Below is a summary of Core Earnings: New Data & Evidence, a paper by professors from Harvard Business School & MIT Sloan and in The Journal of Financial Economics, which proves we provide the best fundamental data in the world.

Get the evidence that legacy fundamental data & research is flawed and unreliable: Lost In Standardization: Effects of Financial Statement Database Discrepancies on Inference and Do Managers Define Non-GAAP Earnings to Meet or Beat Analyst Forecasts?

See Earnings Distortion: The New Value Factor for examples of how to monetize the idiosyncratic alpha in our proprietary data. More on the superiority of our research is here.

Learn more about the best fundamental research

The Problems:

Flawed Fundamental Data


  • “we identified cases where Compustat did not collect information relating to firms’ income that is useful in assessing core earnings.” – pp. 16, 2nd para.
  • “IBSPI and OIADP do not fully adjust for these [non-recurring…] items.” – pp. 15, 1st para.


  • “[New Constructs’] Total Adjustments[1] differs significantly from the items identified and excluded from Compustat’s adjusted earnings measures. – pp. 14, 1st para.
  • “Compustat adjustments explain only 26% to 53% of the variation in Total Adjustments. Jointly they explain 57%, leaving a significant amount of the variation in Total Adjustments unexplained.” – Online Appendix pp. 1, 2nd para.


  • Prior research suggests they [street earnings from IBES] can systematically exclude certain operating and recurring items and may be biased due to managerial incentives." – pp 16, 3rd para.
  • "The ‘three-year rule’, which is still in effect, illustrates a source of hindsight bias in Compustat data‚" – pp. 16, 1st para.

Significant Materiality of the Flaws

Size (large):

  • “average non-core earnings adjustments (Total Adjustments) amounts to an 18-cents-per-share increase in a firm’s Net Income. “– pp. 20, 1st para.
  • “average Total Adjustments represents 19% of average Net Income.” – pp. 20, 1st para.

Frequency (often):

  • “…from 1998 to 2017, the average number of non-core-earnings items identified in a 10-K rose more than 30%...” – pp. 3, 2nd para.
  • “more than 99% [of firms in the sample period] disclose one such [non-core-earnings] item on the face of their income statement. Further, 83% disclose a non-core-earnings item off of the income statement over their histories, either in the footnotes (72%), MD&A (55%), or cash flow statement (60%) sections of their 10-Ks.” – pp 18, 1st para.

Causes of the Flaws

Poor collection policies:

  • "A concern with metrics such as street or pro forma earnings is that managers and analysts choose the composition of items to include versus exclude in a biased fashion." – pp. 1, 3rd para.
  • "larger values of non-core earnings identified in the New Constructs data correspond to larger absolute differences between New Constructs and Compustat adjustments, consistent with systematic differences in data collection protocol or quality." – pp. 14, 1st para.

Inconsistent collection rules:

  • “the ‘majority rule’ that, until recently, was used to determine the contents of Compustat’s ‘special items’ (SPI) states that if a Compustat analyst cannot determine which income statement line an item is aggregated into for a (dollar value) majority of the identified non-recurring items, then no values are recorded for the SPI field (or its relevant sub-components).” – pp. 14, 3rd para.
  • "Street Earnings adjustments are not necessarily complete or comparable across firms." – pp. 16, 3rd para.

Systemic Bias:

  • “manager-reported non-GAAP earnings are identical to the street earnings reported in IBES, which suggests general agreement between analysts and managers on how to adjust GAAP earnings to reflect core operating performance. This consensus raises the possibility that managerial bias that could be reflected in pro forma earnings is also reflected in street earnings.” – pp. 16, footnote.

The Solution:

Technology You Can Trust

Transparency enables trust:

  • The unrivaled transparency we provide for every data point and calculation enabled the authors of this paper (and their peers) to audit and verify the superiority of our data, models and technology.
  • Without this transparency, the authors of this paper would not have considered investing their time or risking their reputations to publish this paper.

Machine learning proven to work at scale:

  • “the machine [NC’s Robo-Analyst technology] learned and replicated human analysts’ judgements based on their prior decisions. It did so with greater speed and scale to produce a database covering a broad cross-section of firms.” – pp. 9, 2nd para.
  • Harvard Business School Case 118-068: Disrupting Fundamental Analysis with Robo-Analysts, February 2018. (Revised March 2018) By: Charles C.Y. Wang and Kyle Thomas

The Benefits:

Better data

More accuracy:

  • “to provide a detailed quality assessment, we hand-checked a sample of roughly 350 unique specific non-core-earnings items New Constructs identified. …In 100% of instances, we perfectly matched the description, value, and disclosure location of that item in the 10-K.” – pp. 10, 1st para.

Fewer omissions:

  • “[Core Earnings] is empirically distinct from commonly used alternatives and is less likely to be susceptible to compositional and selection biases found in analysts’ and managerial estimates.” – pp 6, 2nd para.

Fewer biases:

  • "Because an independent research firm produces the underlying data, [Core Earnings’] key appeal is that the classification of earnings components is less likely to exhibit systematic biases found in street earnings or pro-forma earnings" – pp. 3, 1st para.

Better Models

Better earnings forecasting:

  • “Core Earnings contains information about future performance that is incremental to Street Earnings” – pp. 29, 2nd para.
  • “Core Earnings is a significantly better forecaster of future Net Income than current-period Net Income.” – pp. 26, 1st para.
  • “For each of the performance measures considered, we continue to find incremental predictive ability in Core Earnings, consistent with Core Earnings effectively distinguishing the recurring and non-recurring components of Net Income.” – pp. 27, 2nd para.

Better stock picking:

  • “Generally, net adjustments identified from the income statement, footnotes, and MD&A are strongly predictive of future performance.”  – pp. 27, 3rd para.
  • “Our empirical evidence suggests that non-core-earnings components are informative of future performance.” – pp. 29, 3rd para.
  • “the predictive power of our measures for returns stems from identifying less obvious items that are not reflected in Compustat Special Items or non-operating earnings.” – pp. 34, 2nd para.

Idiosyncratic Alpha

Footnotes offer a “novel”, untapped dataset:

  • “These findings suggest analysts are not efficient in incorporating the implications of noncore earnings into their forecasts, particularly those disclosed in the footnotes of the 10-K.” – pp. 31, 3rd para.
  • “market participants are inefficient in impounding the implications of non-core earnings, especially those stemming from the footnotes of the 10-K, into stock prices.” – pp. 34, 3rd para.
  • “Our evidence suggests that market participants are slow to take into account the implications of non-core earnings. We find Total Adjustments positively forecasts revisions in analysts’ earnings forecasts in the 12 months following a firm’s 10-K filing.” – pp. 5, 1st para.

Trading strategies to exploit this new dataset:

  • “Trading strategies that exploit non-core earnings produce abnormal returns of 8% per year.” – Abstract, 5th sentence
  • We find the mean difference in monthly abnormal returns between the tenth-decile and first-decile value-weighted portfolios to be 0.66%. These monthly excess returns are both statistically and economically significant, equating to an annualized difference of 8.2%.” – pp. 32, 3rd para.
  • “A value-weighted trading strategy that buys firms in the highest decile of Total Adjustments and sells firms in the lowest decile produces monthly excess returns of 66 basis points per month (8.2% annualized)…” – pp. 5, 2nd para.

Access:   Core Earnings Data    Earnings Distortion Scores        Research         Investment Ratings

This article originally published on December 9, 2020.

Disclosure: David Trainer, Kyle Guske II, and Matt Shuler receive no compensation to write about any specific stock, style, or theme.

Follow us on Twitter, Facebook, LinkedIn, and StockTwits for real-time alerts on all our research.

[1] “Total Adjustments” is the paper’s term for our Earnings Distortion.

Click here to download a PDF of this report.

    3 replies to "The Problems with Fundamental Data & the Solution – as Featured by The Journal Of Financial Economics"

    • Derek Geng

      May I ask does the data cover European Equities as well? Lets say benchmark is MSCI Europe, what is the coverage there? and how long does the history trace back?

    • Matt Shuler

      We cover ~2900 currently active U.S. exchange-traded stocks and ~2700 currently inactive U.S. exchange-traded stocks that appear in the historical data. Our coverage universe is updated daily and available here.

    • Demo New Constructs

      We cover 120 European Companies w/ $4.4 trillion of market cap – that’s ADRs.

Leave a Reply

Your email address will not be published.