MIT Sloan Sustainability Initiative
The Aggregate Confusion Project
Capital markets are moving fast to incorporate Environmental, Social, and Governance factors.
Related Articles
The problem? ESG data are noisy and unreliable.
We found the correlation among prominent agencies’ ESG ratings was on average 0.54; by comparison, credit ratings from Moody’s and Standard & Poor’s are correlated at 0.92. This ambiguity around ESG ratings creates acute challenges for investors trying to achieve both financial and social return.
With our five member companies, the Sustainability Initiative is working to solve this problem through a program of research to improve the quality of ESG measurement and decision making in the financial sector.
We've begun to examine these problems, and potential solutions, in our ongoing suite of research:
Research + Publications
-
Our first paper produced under this project looks into the differences between environmental, social, and governance (ESG) ratings provided by six popular ESG rating agencies. We identify three main factors contributing to the rating divergence: scope, measurement, and weight. Measurement was found to be the biggest contributor, accounting for 56% of the divergence, followed by scope at 38%, and weight at 6%. Our study also detected a "rater effect," where a rater's overall opinion of a company affects the rating of specific categories. The results call for greater attention to how the data underlying ESG ratings are generated.
Florian Berg, Julian F Kölbel, Roberto Rigobon, Aggregate Confusion: The Divergence of ESG Ratings, Review of Finance, Volume 26, Issue 6, November 2022, Pages 1315–1344, https://doi.org/10.1093/rof/rfac033
Awards + Recognition:
- Lead article, Review of Finance
- Editor's Choice, Special Issue on Sustainable Finance, Review of Finance
- Best overall paper award GRASFI 2020
- Best paper award on ESG Standards GRASFI 2020
- Investments & Wealth Institute Governance Insight Award 2021
- Best paper award Liechtenstein Workshop of Sustainable Finance 2021
- Runner up best paper award Gronen Conference 2020
-
The explosion in ESG research has led to a strong reliance on ESG rating providers. This paper documents widespread changes to past ratings from a key rating provider, Refinitiv ESG (formerly ASSET4). Depending on whether the original or rewritten data are used, ESG-based classifications of firms into ESG quantiles and tests that relate ESG scores to returns change.
Berg, Florian and Fabisik, Kornelia and Sautner, Zacharias, Is History Repeating Itself? The (Un)Predictable Past of ESG Ratings (August 24, 2021). European Corporate Governance Institute – Finance Working Paper 708/2020, Available at SSRN: https://ssrn.com/abstract=3722087 or http://dx.doi.org/10.2139/ssrn.3722087
Awards + Recognition:
- John L. Weinberg/IRRCi 2022 Research Paper Award
- Cornell ESG Research Conference 2022 Best Paper Award
- CFA Society Germany Investment Research Award Runner-up
-
How does ESG performance affect stock returns? Answering this question is difficult because existing measures of ESG performance — ESG ratings — are noisy and, therefore, standard regression estimates suffer from attenuation bias. To address the bias, researchers propose two noise-correction procedures, in which they instrument ESG ratings with ratings of other ESG rating agencies. The corrected estimates demonstrate that the effect of ESG performance on stock returns is stronger than previously estimated: after correcting for attenuation bias, the coefficients increase on average by a factor of 2.6, implying an average noise-to-signal ratio of 61.7%. In simulations, our noise-correction procedures outperform the standard approaches followed by practitioners such as averages or principal component analysis.
Berg, Florian and Kölbel, Julian and Pavlova, Anna and Rigobon, Roberto, ESG Confusion and Stock Returns: Tackling the Problem of Noise (October 12, 2021). Available at SSRN: https://ssrn.com/abstract=3941514
Awards + Recognition:
- ARCS 2022 People's Choice Award
-
This study examines the impact of ESG ratings on mutual fund holdings, stock returns, corporate investment, and corporate ESG practices, using panel event studies. We focus on changes in the MSCI ESG rating, which we show is the most relevant rating for the holdings of ESG mutual funds in the US. We document that rating downgrades reduce ownership by mutual funds with a dedicated ESG strategy, while upgrades increase it. We find a negative long-term response of stock returns to downgrades and a slower and weaker positive response to upgrades. Regarding firm responses, we find no significant effect of up- or downgrades on capital expenditure. We find that firms adjust their ESG practices following rating changes, but only in the governance dimension. These results suggest that ESG rating changes matter in financial markets, but so far have only a limited impact on the real economy.
Berg, Florian and Heeb, Florian and Kölbel, Julian, The Economic Impact of ESG Ratings (September 4, 2022). Available at https://ssrn.com/abstract=4088545
-
In “The Signal in the Noise,” we argue that abandoning ESG now would essentially be throwing the baby out with the bath water. ESG, however flawed, is currently the best way to measure the ethical behavior of companies and that ESG data, when deployed with greater transparency, can be an important source of information for investors.
“Given the complexity of what ESG measurement entails, we believe that the only solution to gathering, analyzing, and aggregating the data runs through commercial ESG rating agencies and ESG data providers,” write authors Florian Berg, Jason Jay, Julian Kölbel, and Roberto Rigobon.
The Signal in the Noise. Florian Berg, Jason Jay, Julian Kölbel, Roberto Rigobon, CESifo, Munich, 2023, EconPol Forum 24 (1), 23-27
-
We quantify the financial performance of environmental, social, and governance (ESG) portfolios in the U.S., Europe, and Japan, based on data from six major ESG rating agencies. We document statistically significant excess returns in ESG portfolios from 2014 to 2020 in the U.S. and Japan. We propose several statistical and voting-based methods to aggregate individual ESG ratings, the latter based on the theory of social choice. We find that aggregating individual ESG ratings improves portfolio performance. In addition, we find that a portfolio based on Treynor-Black weights further improves the performance of ESG portfolios. Overall, these results suggest there is a significant signal in ESG rating scores that can be used for portfolio construction despite their noisy nature.
Berg, Florian and Lo, Andrew W. and Rigobon, Roberto and Singh, Manish and Zhang, Ruixun, Quantifying the Returns of ESG Investing: An Empirical Analysis with Six ESG Metrics (February 27, 2023). Available at SSRN: https://ssrn.com/abstract=4367367
Aggregate Confusion Project Scope
Together, we are ready to chart a new course towards more rigorous and coherent methods for ESG integration. Each member organization will have slightly different business objectives, internal capabilities for ESG research, and priorities among ESG issue areas; but broadly we understand there to be a few tasks common to all:
-
Reduce the level of noise in measuring specific ESG categories such as labor treatment, carbon emissions, and product safety;
-
Understand the effect of ESG-driven investment flows on stock price and firm behavior;
-
Develop smarter ways to aggregate ESG factors into composite indices;
-
Reliably assess investor preferences to enable ESG indices to be more customized and attuned to investors’ values.
Building a sustainable investment capability inside your firm is a process of engaging people and building their skills and knowledge. Our intention is that our members will have access to top researchers, and the opportunity to engage with our entire team and suite of activities to build their firm’s capacity.
MIT Sloan Sustainability Initiative
Carbon Confusion
A suite of research within the ACP, we are probing fundamental questions around scalability, additionality, asset pricing, and the evolving regulatory environment of carbon markets.
Learn MoreLeadership Team

Florian Berg
Co-Founder, Research Associate
Learn more.
Jason Jay
Senior Lecturer, Director, Sustainability Initiative, PhD '05
Learn More
Julian Kölbel
Co-Founder, Research Affiliate
Learn more
Roberto Rigobon
Economics, Finance and Accounting
Society of Sloan Fellows Professor of Management
Learn MoreResearchers

Dev Asnani
Research Assistant

Alexia Couyutas Duarte
Research Assistant
Learn more
Richard Chen
Research Assistant
Learn more.
Angela Gales
Research Assistant
Learn more
Justin Grossman
Research Assistant
Learn more
Florian Heeb
Postdoctoral Associate
Learn more. News + Media
-
Financial Times | July 3, 2023 MIT Sloan SI Researchers: Rating the ESG rating agencies
-
The Banker | April 6, 2023 Aggregate Confusion researchers: ESG ratings are still our best option
-
Ideas Made to Matter ESG ratings: Don’t throw the baby out with the bath water
-
Institutional Investor | September 13, 2022 Here Are the Winners of the 2022 Allocators’ Choice Awards
-
The Economist | July 21, 2022 The signal and the noise
-
Foreign Policy | January 10, 2022 Can Global Regulators Save the ESG Movement From Itself?
-
Economic Times India | December 10, 2021 Sustainability-oriented innovations solve public problems
-
Ideas Made to Matter Why sustainable business needs better ESG ratings
-
Ideas Made to Matter 4 strategies for sustainable business
-
World Economic Forum | November 23, 2021 New World Economic Forum "ESG Transformation Map"
-
Wall Street Journal | November 18, 2021 Where Will ESG Investing Be in Five Years?
-
Yahoo Finance | November 3, 2021 Financial firms managing $130 trillion commit to net-zero goals
-
Institutional Investor | October 27, 2021 AQR and MFS Partner With MIT to Solve ESG Data Problem
-
PR Newswire | October 26, 2021 Aggregate Confusion Project welcomes four new investment firms
-
Washington Post | October 10, 2021 Finance Can Save the Planet. It Just Needs Better Data.
-
Fortune | October 11, 2021 Oatly learns that it’s not easy being ‘green’
-
Top1000Funds.com | September 23, 2021 ESG needs better data, better ratings and better products
-
Bloomberg | September 10, 2021 Credit Suisse ESG Head Wants ‘More Pressure’ on Rating Firms
-
Press Rethinking diversity measures in the finance industry
-
Ideas Made to Matter 3 hurdles to sustainable investing — and how to overcome them
-
Wharton | May 25, 2021 How Far Could Pension Funds Drive Sustainable Investing?
-
MIT Spectrum | April 22, 2021 A Business Lens for Sustainability
-
Harvard Business Review | April 2021 Overselling Sustainability Reporting
-
PRI | January 15, 2021 Rewriting history II: The (un)predictable past of ESG ratings
-
London Business School | March 19, 2021 Everything you need to know about ESG investing – for now
-
Top1000Funds.com | January 14, 2021 MIT consortium builds ESG tools
-
Forbes | January 10, 2021 The Curious Case Of Engineering Schools And Sustainable Investing: MIT
-
CG Capital Markets | December 3, 2020 The Hidden Joule | A Conversation with Florian Berg
-
ChangeInc | November 23, 2020 Assessment of company sustainability policy differs per rating agency
-
CIO Magazine | September 17, 2020 The Hodgepodge of ESG Investment Standards Sparks Controversy
-
Wall Street Journal | September 14, 2020 Massachusetts Pension Overseers Join MIT Sloan ESG Initiative
-
Barron's | September 14, 2020 New SEC Rules May Be Good for Business but Not for Investors
-
VP Bank | October 2, 2020 Award-winning: VP Bank presents the Best Paper Award
-
Press Aggregate Confusion Project: Improving the quality of ESG measurement
-
Forbes | August 4, 2020 Do Signatories To The Principles For Responsible Investment Practice What They Preach?
-
GRASFI | September 11, 2020 Aggregate Confusion: GRASFI Best Paper Prizes
-
Forbes | June 8, 2020 Demystifying ESG: Its History & Current Status
-
Financial Times | May 11, 2020 Fund managers struggle to compare ESG apples with oranges
-
Forbes | April 15, 2020 Municipal Credit Ratings And ESG Ratings: Irreconcilable Differences?
-
Grow The Pie | March 14, 2020 The Inconsistency of ESG Ratings: Implications for Investors
-
Reuters | March 11, 2020 'Thanks Larry!' Green accounting project says BlackRock plug gave it a boost
-
CIM Magazine | February 17, 2020 La production de rapports sur la durabilité est-elle devenue intenable
-
AllAboutAlpha | January 20, 2020 ESG: Why Ratings Diverge
-
Bloomberg | December 11, 2019 Conflicting ESG Ratings Are Confusing Sustainable Investors
-
THE IRISH TIMES | DECEMBER 10, 2019 Are you sure you’re investing ethically?
-
GreenBiz | November 18, 2019 ESG ratings are confounding. For CSOs, that’s good news
-
MIT ALUMNI ASSOCIATION | OCTOBER 22, 2019 A Unique PROMISE to Measure and Incentivize Widespread Prosperity
-
FINANCIAL TIMES | SEPTEMBER 16, 2019 Credit rating agencies join battle for ESG supremacy
-
Ideas Made to Matter ESG ratings vary widely. Here's why