Tips to Overcome ESG Data Selection Challenges

Gediminas Rickevičius, VP of Global Partnerships at Oxylabs


Environmental, Social, and Governance (ESG) guidelines promise better investment outcomes with multiple benefits to citizens, the environment, and the economy. Despite the perceived benefits, challenges continue to emerge with establishing ESG ratings and data collection.

Developing industry-wide standards are part of the solution, however firms can also take measures to establish a framework when analyzing data sets and company rankings. Web scraping is an additional solution that empowers firms with actionable data when checking ESG scores.

What is ESG Criteria?

ESG guidelines attempt to assess investment outcomes with a focus on environmental, social, and corporate governance goals. Environmental criteria are primarily concerned with reductions in waste and greenhouse gas emissions, and management of water supplies. Social guidelines assess businesses on their activities concerned with consumer protection, employee welfare and wellbeing, financial activities (such as lending), human rights, and community impact.

The third component of ESG guidelines includes the promotion of Diversity, Equity, and Inclusion (DEI). These programs and policies aim to promote equal participation of individuals based on group identity factors that include gender, skin tone, culture, age, sexual preference, and religion.

Gediminas Rickevičius

ESG Investing is Growing

According to Bloomberg, ESG investing will reach $53 trillion by 2025. Individual shareholders are also voicing support for social and environmental proposals, with an increase from 21% to 32% between 2017 and 2021.

Currently, global assets under ESG management amount to $35.3 trillion in the United States, Canada, Japan, and Australasia. European firms lead the way with over 50% of investment assets under management following the EU’s 2016 ESG global mandate.

How ESG Ratings are Calculated

ESG scores or ratings are based upon a company’s success in addressing its environmental, social, and governance responsibilities. While some of these responsibilities have material consequences, most are important to stakeholders for non-financial reasons.

ESG ratings are not established by a standard system. Instead, there are several ESG rating agencies that provide investment assessments based on a business’s ESG performance. Most of these agencies have varying methodologies when assigning ESG scores, making it challenging for investors to understand a company’s performance in real terms. In addition, these calculations are typically based on a company’s “perceived” activities, and may not account for all positive practices put into place.

ESG Scoring Factors

ESG scoring factors vary significantly from one agency to another. Factors that contribute to the overall rating typically include:

Environmental Scoring Factors

Environmental scoring factors can include commitments to renewable energy, climate change, water pollution, soil contamination, and waste production.

Social Scoring Factors

Social scoring ratings are primarily based on relationships with employees, shareholders, suppliers, and partners. Factors that determine this rating include employee compensation, workplace safety, health benefits, and workplace conditions of operations located in other parts of the world.

Corporate Governance Scoring Factors

Corporate governance scoring depends onboard activities, regulatory compliance, and legal operations. Factors that influence this rating include compliance with federal, provincial, state, and municipal laws, executive compensation, and if the board of directors comprises individuals from different identity groups.

ESG Data Challenges

The ESG space faces numerous challenges that specifically concern score calculation and regulations. Some of the most frequent criticisms underlying this framework include:

Inconsistent Grading Methodologies and Data Standards

In the absence of a regulatory framework, rating agencies typically use different methodologies, resulting in varying scores for the same company. In addition, there is a challenge in quantifying facts such as biodiversity investment, carbon footprint, and what constitutes a “diverse” management team. Besides the lack of a consistent framework, firms do not use normalized data sets across varying time periods, resulting in inconsistent conclusions about a company’s adherence to ESG standards.

Corporate Greenwashing

Greenwashing is a deceptive marketing practice that attempts to convince investors that an organization is committed to sustainable environmental goals. The overall purpose is to include the company in sustainability indices for inclusion in investment portfolios, and to promote a positive public image.

How to Choose ESG Data

Despite the challenges facing ESG investing, firms can take action to obtain relevant data when assessing sustainable investments. Some tips when formulating a strategy include:

Set firm-specific ESG goals

Ethical investing is more than just a passing trend. Digital innovation enables investors from all demographics to see the effects of corporate activities across the world.

Creating firm-specific ESG goals can be profitable from a retail perspective – particularly with younger “millennial” investors. According to a recent report, millennials drove sustainable investment from $5 billion in 2015 to $51.1 billion in 2020. Investors today are more concerned with the power of their money than ever before – and investment firms are responding by creating funds that align with current global issues.

Evaluate data quality

Firms should vet data for completeness and transparency. Reporting on a limited data set may not provide a complete picture that considers the source and methodology. In addition, some data sets have gaps that make the data inaccurate and unreliable. Choose a provider that continuously tracks indicators to produce timely data that is relevant.

Gather data with ethical web scraping

In the absence of consistent data sets, many investment firms choose to obtain their own data with web scraping – a process that extracts public information from websites.

Web scraping can be used to collect publicly available ESG data from multiple sources, including company websites, online directories, and other sources. Areas of interest to ESG investors can include:

  • Investment in sustainable energy
  • Air quality
  • Health and safety procedures
  • Waste removal
  • Employee compensation and benefits
  • Shareholders’ rights
  • Diversity, equity, and inclusion
  • Board structure and tenure

Businesses opting to gather public data can employ an in-house team or use a web scraping tool customized to extract data from any public source.

Learn more about scraping critical ESG data

Finance firms increasingly use web scraping to extract ESG and alternative data critical to successful investment decisions. Download “Alternative Data Unlocks Key Decisions in the UK & US Finance Industries” – our complimentary white paper that explains:

  • The dominance of alternative data across the global finance industry
  • Regional differences in the UK and US finance sectors
  • Most valuable data extraction methods
  • Data extraction challenges
  • Emerging data collection strategies

ESG investing has grown substantially in recent years and is forecasted to increase. Download our whitepaper to learn about data extraction solutions that drive precision investment strategies and comply with ESG guidelines.

Explore more