gameguru (33.48)

gameguru's CAPS Blog

Gauging the Usefulness of Star Ratings

Recs

0

Report

December 06, 2006 – Comments (2)

I was looking over the performance of the CAPS1StarIndex and CAPS5StarIndex today and I noticed that both of them are substantially outperforming the market. That is, if you had made outperform picks on all 200 of the stocks in either index, you would have a positive score in the CAPS system. (Note that CAPS1StarIndex currently has a negative score, as it makes underperform calls.)

The following then occurred to me:

The “market” benchmark is defined by CAPS as the S&P500, and therefore consists of 500 stocks, while the CAPS universe currently consists of 8348 stocks, 2858 of which have a star rating. This means there are currently about 572 stocks at each star level. Star levels have no set relationship to market capitalization. (However, one might expect larger-cap companies to be more “popular” and perhaps have higher star ratings. For instance, of the 100 highest stocks by market cap, 18 were 5-Star, 28 4-Star, 25 3-Star, 21 2-Star, and just 4 1-Star, while 4 were unrated. That’s nearly 2x as many 5- and 4-Star stocks as 1- and 2-Star stocks in the group.)

It is therefore possible that every star-rating level (1 through 5) could simultaneously outperform the market (defined as the S&P500).

Although the S&P500 is not simply the 500 largest stocks by market cap, it is limited to larger-cap stocks and primarily domestic U.S. stocks. The necessary conditions for this odd situation, then, are simply that small-to-mid cap and international stocks outperform large-cap stocks.

Of course, the converse is also true – all star rating levels could simultaneously underperform the market (defined as the S&P500) if large-cap stocks are doing better than other market classes.

I’m certainly not suggesting that star levels should have anything to do with market cap.

Rather, I’m wondering how well we can really gauge the usefulness of the star ratings generated by CAPS when comparing against just the S&P500. Should we be gauging against a broader benchmark – the Wilshire 5000 maybe? Or against multiple benchmarks – including international, small-cap, large-cap, and broad-market indices? Ultimately, we will need very specific figures – by how much does each star level outperform each benchmark – to determine whether the star ratings provide any useful information.

When the CAPS administration starts judging the system’s performance (and likely making marketing claims on that basis), I hope they’ll take these points under consideration.

2 Comments – Post Your Own

#1) On December 08, 2006 at 9:15 AM, 5STARsmallCAPS (74.49) wrote:

Interesting point. As an example of the variation between stock market indices:

From January 1999 through 7/2/06, the S&P 500 was only up about 3%, while the Russell 2000 small cap index was up about 70% over the same time period.

It will also be interesting to see how CAPS ratings are affected if/when dividends are included.

Report this comment

#2) On December 08, 2006 at 2:11 PM, gameguru (33.48) wrote:

Exactly. I think that the underperformance of large caps may be currently driving the simultaneous outperformance of both 5-Star and 1-Star stocks (and it wouldn't surprise me if 2-, 3-, and 4-Star stocks were outperforming, too). Ultimately, it may mean that knowing a stock's market cap class may be more informative than knowing its CAPS Star status, which would be a sad state of affairs, in my opinion.

I think the CAPS cognoscenti will eventually have to consider alternative benchmarks in describing/evaluating the performance of CAPS ratings. There may even be a need for normalizing CAPS ratings based on market cap classification and the performance of that class, but that's a complex step that I wouldn't take unless the data bear it out.

By the way, I also can't wait for dividends to be included. You're right - it will be interesting to see if and how that affects the CAPS ratings. I think such effects would happen indirectly, as it would primarily affect performance calculations, which could very well push some players into or out of All-Star status. In turn, that would affect the star ratings of the stocks those players have picked.

Report this comment