Morningstar’s ETF and mutual fund ratings have been the go-to standard in the fund industry for decades now, and the company does an impressive job of providing research on a massive number of funds to millions of people.

However, I believe the overreliance on Morningstar is unhealthy for the fund industry. Changes in Morningstar ratings have significant impacts on mutual fund flows, even though the predictive value of their ratings, both the old star ratings and the new analysts ratings, is questionable at best.

Furthermore, while Morningstar provides research on the past performance and strategies of ETFs and mutual funds, it is sorely lacking in quality analysis of the holdings of these funds. The performance of a fund is only as good as the performance of its holdings, and a rating that doesn’t take into account the holdings is not truly predictive.

At New Constructs, our predictive fund ratings are based on our stock ratings of the fund’s holdings as well as the fund’s total annual costs. These ratings are forward-looking and do not rely on past performance, which allows us to spot underperforming funds that are primed to take off and over performers set to revert to the mean, just as we do with individual stocks.

Figure 1 shows several examples from recent years where New Constructs and Morningstar have had contradictory ratings on funds. In most of these cases, Morningstar has had a positive rating and New Constructs has had a negative rating, as Morningstar’s ratings are overwhelmingly biased towards positive ratings. However, there are also cases where Morningstar’s ratings were more negative than ours. In every case, New Constructs’ rating was more predictive of the future performance of the fund.

Figure 1: Excess Returns/Protection from New Constructs Ratings

Screen shot 2014-01-29 at 11.21.50 AMSources: New Constructs, LLC and company filings

In the cases where Morningstar explains the methodology behind its ratings, fundamental analysis is either absent or extremely shallow. In Morningstar’s article recommending Vanguard Small Cap Value ETF (VBR), the fundamental view section contains 1) A track record of small-cap value stocks overall, 2) a paragraph about the U.S. economic recovery, and 3) a price to earnings ratio.

When that article was published on Morningstar, New Constructs had a 2 Star, Dangerous rating on VBR based on our ratings of the ETF’s holdings. Sure enough, VBR has lagged the Russell 2000 index since April 2013.

Past performance clearly has a strong effect on Morningstar’s ratings, even though countless research has shown that past performance is not a good predictor of future returns. For instance, when American Beacon Small Cap Value Fund (AVFIX) earned Morningstar’s 4 Star rating in January of last year, it had a prior 3-year return of 98%.

Once again, our analysis of AVFIX showed a much less rosy picture, and once again it has significantly lagged its benchmark over the past year.

Even when Morningstar’s ratings do look past prior performance, such as their recommendation of LongLeaf Partners Fund (LLPFX) in February 2012, the fundamental analysis is lacking. Even though LLPFX had been basically flat the year before, Morningstar recommended the fund based on the quality of the managers and their “skin in the game”, the fact that they invested their own money in the fund. However, New Constructs saw little reason for optimism in LLPFX’s holdings, and sure enough the fund has delivered only one third of the S&P 500’s return over the past two years.

Looking Forward

One of Morningstar’s recent ratings that we disagree with is its 5 Star rating of First Trust Consumer Staples AlphaDex (FXG). The Consumer Staples sector is our top-rated sector, but FXG is not one of the best ETFs in the sector and only earns our Neutral rating. Other ETFs such as SPDR Consumer Staples Select Sector ETF (XLP) and Vanguard Consumer Staples ETF (VDC) look much more attractive to us in this sector. More on the best and worst Consumer Staples sector funds is here.

Over 60% of FXG’s assets are allocated to Neutral-or-worse rated stocks, and it has above average total annual costs of 0.78%.

Constellation Brands (STZ) is the top holding in FXG at 5.4% and earns my Dangerous rating. The wine distributor has seen some recent growth due to its expansion into the beer industry, but its return on invested capital (ROIC) still stands at an unimpressive 7%. Investors are probably aware of STZ’s $3.8 billion in adjusted total debt, but its $900 million in employee stock option liability not as well publicized. The truly Dangerous aspect of STZ, however, is its valuation. The current valuation of ~$80/share implies that the company will grow after-tax profit (NOPAT) by 21% compounded annually for the next nine years. The wine and beer industry is not exactly a high-growth market, so these expectations seem unreasonably high.

Allocation to Dangerous stocks such as STZ make FXG only a Neutral ETF, and certainly not one worthy of a five star rating. As you can see from Figure 1, our Neutral rating on Allianz Mid-Cap Value Fund (PQNAX) correctly predicted that it would match its index’s performance, even though Morningstar had assigned it a 2 Star rating. In this case, we expect FXG to perform similarly to the market and lag behind many of its fellow Consumer Staples ETFs. FXG has put up impressive returns in recent years, but its holdings are overvalued compared to its peers.

Morningstar’s research can be helpful, but we believe our in-depth analysis of ETF and mutual fund holdings gives our ratings a superior predictive value, and results back up our claim.

Sam McBride contributed to this report.

Disclosure: David Trainer and Sam McBride receive no compensation to write about any specific stock, sector, or theme.