The Bottom Line: Differences in ESG ratings highlight that sustainable investing must be active to be successful. This is the conclusion of a study by Dimson, Marsh and Staunton, the authors of “Triumph of the Optimists”, and well-respected academic observers of investment theory and practice.

The Study: “Divergent ESG Ratings” by Elroy Dimson (of Cambridge University’s Judge Business School) and Paul Marsh and Mike Staunton (both of the London Business School). Published in the Journal of Portfolio Management in November 2020.

The Process: The study examines the ESG ratings from three providers (FTSE Russell, Sustainalytics, and MSCI) for six large companies (Facebook, JPMorgan Chase, Johnson & Johnson, Wells Fargo, Walmart, and Pfizer). The authors examine both their overall ratings, and their ratings for each of the three pillars: environmental, social and governance.

They then examine the four factors that give rise to ratings inconsistencies: data discrepancies, different benchmark choices, different approaches to missing data, differences in scoring methodologies, and varying weightings.

The Findings: “[T]here is minimal correlation between ESG ratings from alternative agencies. . . Companies with a high score from one rater often receive a middling or low score from another rater.”

The correlations in each of the three pillars are “extremely low,” with more agreement between Sustainalytics and FTSE than between other pairs of raters.

“Perhaps the most striking finding is the extraordinarily low correlations for governance. Raters disagree even on factual issues.”

“[T]he aggregate ESG ratings show more agreement than is demonstrated on the component parts.” However, the average pairwise correlation is still a very low 0.45.

Differences in the weighting schemes could play a significant role in the varying scores. “Sustainalytics tends to place roughly equal weight on each of the three ESG pillars, whereas MSCI has a greater variability in the factor weightings.”

The Implication: “ESG ratings should not be treated as a black box or used mechanically. Blanket use of ESG scores is not the solution. At best, they are a starting point. Analysis and fund managers need to understand how they are constructed and supplement them with their own scrutiny to build a holistic understanding of a company and hence ensure that their investments best reflect the values of their end-investors.”

For further discussion of the active nature of sustainable investing, see the Active Managers Council whitepaper: Sustainable Investing is an Active Process.