Use access key #2 to skip to page content.

TMFSpiffyPop (99.23)

Community Intelligence Works

Recs

148

November 29, 2007 – Comments (54)

With a year of data now crunched, we’re pleased and perhaps even excited to show you how our CAPS star ratings are performing. The headliner: CAPS 5-star stocks exhibited superior outperformance of the market averages, CAPS 1-star stocks exhibited superior underperformance of the market averages, and everything in between did about what you’d expect. Four stars beat two stars, and three stars slightly outperformed the market.  Check out Tim Hanson’s The Top Stocks and the Top Returns article for more detail.   Here’s the short version:

 


Community intelligence works. Especially when it emanates every market day, all year long, from the world’s greatest investment community.

Let’s briefly review how this works. You pick stocks. We score you. We rate you on a scale of 0-100 (just like the SAT), and rank you against your fellow Fools. We also take your picks, and we create ratings for the stocks too -- the star ratings (quintiles) I’m talking about in this blog post. Our CAPS stock ratings are weighted based on player merit -- no beauty contest, and you don’t have to be a talking head on TV. Just straight numbers: If you’re a 90-rated Fool, the stocks you pick will get better ratings than if you were a 10-rated Fool. We update all of our data and ratings and rankings every 15 minutes, so that CAPS star ratings provide you a very rare and rich “Doppler radar” look at the stock market. CAPS ratings are merit-based, forward-looking, and firmly predicated on this community’s broad and deep intelligence.

What’s the bottom line, then? Our star ratings matter. Factor them into your thinking. This is absolutely not to say that every 5-star stock will beat the market, or even beat every 1-star stock. It is to say that these broad categories appear to be predictive and important and noteworthy. And given that they now cover more than 5000 stocks on the market, we almost always have an interesting and valuable second opinion for you on any stock you might research.

There’s always so much more to say about CAPS. I find myself, in this quantitative-focused blog post, wanting to talk about all the amazing qualitative content that lurks behind every ticker across the site. So many Fools have typed in anything from a sentence to a 10-page writeup for their picks. Beyond even its star ratings, CAPS therefore has so many thoughtful and fresh insights for you as you conduct your stock research.

But to close as I began, with best wishes for CAPS in year two, let’s take one more look at the big story:

 


--David

54 Comments – Post Your Own

#1) On November 29, 2007 at 9:53 AM, floridabuilder2 (99.64) wrote:

for my real life purchases (except my phm technical trade) i buy only 4 and 5 star caps stocks.... although i consider myself an expert at homebuilding and can pick a one star stock with no regrets... the other 195 industries out there require a lot more research than i have time for... so if i do common sense thinking, look at the charts and some high level research... i consider the caps community my in depth researchers...  i especially look at what the top 20 fools say on a given stock (you can sort the caps picks on a stock by player ranking) and when they gave their prediction....  i can then go to those time frames and look at what news came out at the time

thus, before i buy a 4 or 5 star stock my seperator is the who and when

Seacrest out

Report this comment
#2) On November 29, 2007 at 10:52 AM, FoolishChemist (98.12) wrote:

Good to see we haven't been wasting our time!  If I'm reading this right that's the return just when the stock is 5 stars, and not just the stocks that were 5 stars a year ago.  Are the returns on a per stock basis, or market cap weighted?  I am curious what the turnover ratio is.  If you form the CAPS mutual fund, the expense ratio should be really low, since we're doing all the work for free :-)

Report this comment
#3) On November 29, 2007 at 10:53 AM, gonzflanker06 (< 20) wrote:

Congrats Dave!

This only confirms for me how harnessing the power of community intelligence can prooduce great returns... I can't wait to see where CAPS takes us next

Phil Cooney

Report this comment
#4) On November 29, 2007 at 11:34 AM, Imperial1964 (97.87) wrote:

We need to start a CAPS 5-star fund!

I'd put money in it. 

Report this comment
#5) On November 29, 2007 at 11:35 AM, TMFCrocoStimpy (93.81) wrote:

FoolishChemist,

Your interpretation is correct - each of these portfolios is constructed by holding stocks while they are in a specific quintile. So, for example, any time a 5 star stock drops below 5 it is "sold" out of the portfolio, and any time a stock becomes a 5 star it is "bought".  The portfolios are weighted on a stock basis, same as the way they are counted in the general CAPS scoring method.  The turnover rate is high, especially because there is a significant degree of churn near the 5-star/4-star border.  Given the strength of predictive power observed over the past year, especially taking into consideration that the beta of these portfolios are essentially 1, we're currently working on identification strategies that can help winnow down these portfolios so that it is possible to create a realistic level of turnover.

Congratulations to Dave and the entire CAPS community!

Xander Twombly

Report this comment
#6) On November 29, 2007 at 11:46 AM, FoolofValue (93.68) wrote:

This is excellent news!

Report this comment
#7) On November 29, 2007 at 11:52 AM, tengrandchicago (37.87) wrote:

awesome

Report this comment
#8) On November 29, 2007 at 12:09 PM, hondo928 (99.64) wrote:

I might have misread the part where you talked about this, but is this backdating performance, of stocks currnetly rated 5 Starts, or When a stock is 5 Stars it is put into that category and dropped when it drops from 5 to 4, etc.?

Report this comment
#9) On November 29, 2007 at 12:16 PM, leanpockets (85.46) wrote:

A few suggestions: 

You should earn more points for contrarian picks (giving an outperform on a 1 star stock and being right). Maybe fame is enough of a reward, but you're not highlighting it.

It would be cool to have a macro indicator where the community can predict which way the market is going.  Maybe the ratio of 5 star to 1 star picks can tell you this?

You should have an API. 

You should publish a list of daily transitions (4 -> 5 star, etc).

 Also, why not include November in your chart?

Report this comment
#10) On November 29, 2007 at 12:24 PM, TMFCrocoStimpy (93.81) wrote:

hondo928,

The star rating portfolios are forward moving performance, not backdating performance.  For example, for the 5 star portfolio shown in the graphs above, all the 5 star stocks on 11/6/06 where added into the portfolio on that date.  Then, as the date marches forward, any stock that drops below 5 stars is sold out of the portfolio on the day it drops in rating, and any stock that becomes a 5 star stock is purchased on the same day as well.  The portfolios are stock weighted (rather than market cap weighted), so the contribution of each stock is its gain/loss relative to its original value on the day of "purchase".

Report this comment
#11) On November 29, 2007 at 3:09 PM, mgiv (99.54) wrote:

I wish they provided rss feeds of stock ratings, people ratings, with portfolios, ended picks, etc.  Instead of trying to quant trade stocks, I would be plugin in this foundation knowledge and manage my trading decisions that way.

Report this comment
#12) On November 29, 2007 at 3:57 PM, TMFSelzhanik (99.35) wrote:

This is impressive news indeed.  30% annual returns.  And what I find really great is while the market (as measured by the S&P 500) has been relatively flat since May at 10%, the 5-stars are up from 20% to 30%.  If only we could see how the top 1% perform, as opposed to the top 20%...

Thanks for the clarification on how the returns were calculated, Xander.

Report this comment
#13) On November 29, 2007 at 4:34 PM, MCKIrobert (65.54) wrote:

 

One quick thought.  Motley Fool should have a way of removing inactive players, so that the ratings don't get stale.  You might have a rule that any player who does not log in for more than 3 months gets a warning email and is then removed (at least from the star calculation).

 

Report this comment
#14) On November 29, 2007 at 5:54 PM, TradeOffTrader (72.49) wrote:

The results are impressive.  However, the value of the CAPS rating could diminish over time should more traders use it for buy and sell decisions. 

Additionally, the value of the ranking of CAPS players could diminish greatly as the density of followers increases amongst the more highly ranked players.  Consider the player statistic 'average pick rating.'  For most of the highly rated players it is a 4.  Now suppose lots of interlopers in CAPS pick 5-star stocks merely because of the current rating.  Then they can get good scores initially.  However, after a certain point the predictive value of the rating disappears because the followers have high scores but no positive contribution to the community intelligence.  As a matter of fact their contribution is negative.

CAPS should fix this by rating players based on leadership.  Scoring points on a 1-star pick should be worth a lot more than scoring points on a 5-star pick.  Maybe a player should get no points for a pick that is currently a 5-star.

 I hope that CAPS addresses this issue before this great system falls into ruin! 

Report this comment
#15) On November 29, 2007 at 6:52 PM, zygnoda (27.19) wrote:

I was wondering if Caps could be changed so that I can add stocks with a market cap of less than 100 million to my watchlist.  I understand the reasons why you guys haven't done anything like that (but thought I would ask anyway).

'This is impressive news indeed.  30% annual returns.'  Woo hoo!

 

Report this comment
#16) On November 29, 2007 at 7:15 PM, TDRH (99.84) wrote:

Appreciate all the hard work Dave.  I am with Imperial1964, though I would call it a CAPS index fund, put 80% in 5 star stocks and short 20% on one star stocks.    I would put money in it. 

 

Report this comment
#17) On November 29, 2007 at 7:35 PM, kristm (99.71) wrote:

MCKlrobert:

I agree - would be nice to see how many people we're actually playing against. I'd be shocked if more than 60% of the supposed 40,000 registered players had logged in since 2006. Shutting out anyone inactive for more than three or four months with less than 10 picks would be a good start...

TradeOffTrader:

Player ratings are determined by the performances of their picks. If the leader picks a 5-star stock and 500 other people do too, their scores will go up along with his (or hers). But if they're not sharp and don't end the pick they'll lose their rating as it falls. Same with underperform calls.  Those who truly know their stuff will make picks and end picks independently, and those lemmings who follow them will generally lag behind. As their scores fall, their picks mean less to the Caps system.

Report this comment
#18) On November 29, 2007 at 8:18 PM, TMFSpiffyPop (99.23) wrote:

Just FYI, picks erode their meaning with time. As they work their way through their timeframes, and as their timeframes expire, they die. The person will still rack up Score and Accuracy -- and one of the beautiful things about CAPS is how we can watch these trees grow over time -- so no, we never close out anyone's pick for them. But do we "close out" their effect on our stock ratings? Yes, it's always been that way.

Fool on,

David 

Report this comment
#19) On November 29, 2007 at 11:02 PM, FleaBagger (34.76) wrote:

When I don't set a timeframe for my pick, how long does it affect that stock's rating, if at all?

Report this comment
#20) On November 30, 2007 at 12:08 AM, TMFSpiffyPop (99.23) wrote:

One year. --D

Report this comment
#21) On November 30, 2007 at 6:36 AM, Gtrinvestor (99.88) wrote:

Great to see some empirical evidence back up what you would already normally believe.  I personally don't make any investments with my own money until I have at least looked to what the CAPS community has to say.  I love CAPS and hope it continues for years to come.

Report this comment
#22) On November 30, 2007 at 2:38 PM, hondo928 (99.64) wrote:

I think it would be interesting to see if you could break down these stocks between Small, Mid, and Large Cap Companies, a lot of solid Large Cap companies like Apple lack the community support while some very small Caps like say ARKR have community support but pretty much pace the market. 

Report this comment
#23) On November 30, 2007 at 11:42 PM, turb0kat (92.58) wrote:

Investors who picked the stocks who did best are used to rate the stocks...resulting in the stocks that are highest rated doing the best... that's like saying 1 = 1.

 

Report this comment
#24) On December 01, 2007 at 12:19 AM, TMFSpiffyPop (99.23) wrote:

The "investors who picked the stocks" were rated before they picked the stocks. They then picked the stocks. Those stocks then did what the graph shows. Take another look and readthrough: This isn't a case of predicting the game after looking at the boxscore. This is predicting the game prior to opening pitch... and the game results then playing out in a way that say something very special about the punditry. --DG

Report this comment
#25) On December 02, 2007 at 9:18 AM, Engr27 (< 20) wrote:

Is CAPS an acronym? What does it stand for?

Report this comment
#26) On December 02, 2007 at 3:02 PM, TMFSpiffyPop (99.23) wrote:

It can be.

Community Association for Picking Stocks?

Or pick your own. :) --David

Report this comment
#27) On December 03, 2007 at 1:26 PM, gonzflanker06 (< 20) wrote:

Was there evidence of the same trend say in the top 100 rated stocks? or the top ten?  Is the turnover ratio getting larger or smaller for these brackets?

Report this comment
#28) On December 04, 2007 at 5:06 PM, jfielhauer (< 20) wrote:

Great work guys!  But now that you have a year worth of data you should be able to fine tune your algorithms to increase performance for the next year.  If the stock rating is based on the weight of picks, then what algorithm for assigning weight will optimize performance?  For example:

Is the algorithm for player rank optimized? (weighting of player accuracy and score)

Is the algorithm for pick weight optimized? (relationship between player rank and pick weight - linear/exponential/polynomial)

Could an inverse correlation between the number of picks by a player and their pick weight improve performance? 

Granted, "past performance does not guarantee blah blah blah," but if you had used a different algorithm, how good could the 5 stars have performed?  The 4 star transition stocks? 

I'm excited to see what CAPS comes out with next based on their beta testing!

Report this comment
#29) On December 06, 2007 at 12:53 AM, mtsafe (98.56) wrote:

I have long held a negative opinion of managed mutual funds. For one, I think that the Churn Concerns paralyze portfolios and diminish performance. How did the CAPS 5-Stars perform against funds over the same period?

Report this comment
#30) On December 10, 2007 at 12:37 PM, FundamentalGamma (46.48) wrote:

I was wrong in my assumption that CAPS is a guessing game.

Report this comment
#31) On December 14, 2007 at 8:45 AM, hattyr (< 20) wrote:

second that fund idea. maybe a closed $1000 share fund that 'replicates returns/leveraged returns on 5 star rated fool stocks'...  sign me up. 3 month maturity so hopefully that would limit turnover

Report this comment
#32) On December 14, 2007 at 12:09 PM, dcstrade (86.73) wrote:

Curious to see how these trends behave with the S&P performance subtracted out.  Actually, it may be appropriate to divide the other plot values by (1+S&P%) where the percentage is just represented decimally (i.e. .01 instead of 1%).  I think you get the idea.  Would show some very nice trends, separating exponentially is my guess.

Report this comment
#33) On December 15, 2007 at 4:49 PM, ChesapeakeSailor (53.18) wrote:

Can we turn this in to a mutual fund or ETF?

Report this comment
#34) On December 23, 2007 at 1:46 AM, mercury14 (< 20) wrote:

I've gotta agree with Lean Pockets (post 9) and TradeoffTrader (post 14) that a weighted system, i.e. 1 picks more valuable, 5 picks, less, would really weed out the followers from the leaders. The other night, I thought it would be valuable to research the top 10 CAP players and search out which 1 and 2 stars they have recently picked that are starting to move agressively. The problem was that very few of them had picked 1 or 2 star stocks to outperform. Of course, most were chasing short stocks, mixed w/ a few 5 stars. If we set the system up to reward 1 star outperforms, we might all benefit in discovering value plays.....

Report this comment
#35) On January 27, 2008 at 12:48 PM, MDVW (73.34) wrote:

Ever though about parterning with an ETF provider to offer the CAPS strategy? 

Report this comment
#36) On February 02, 2008 at 1:47 PM, mvg994 (72.36) wrote:

CAPS ETF

LETS START A PETITION... BETTER THAN THE BANK RIGHT?

Report this comment
#37) On February 06, 2008 at 2:40 PM, waditude (< 20) wrote:

David, like post #22, I would love to see your chart broken down by market capitilization.  If much of the 5-star outperformance is driven by small-caps that I won't buy due to liquidity/spread/friction concerns, this would tremendously affect the usefulness of CAPS for me.

Report this comment
#38) On February 12, 2008 at 9:20 PM, sperlyjinx (95.21) wrote:

Anyone know if there is a way to download historical CAPS data?

Report this comment
#39) On February 25, 2008 at 6:52 PM, desect24d (< 20) wrote:

Someone should make an index fund of 5 star stocks.

Report this comment
#40) On February 29, 2008 at 11:40 AM, Gemini846 (88.55) wrote:

The problem with an "index" fund per say would be that the index is constantly changing. Another problem here of course would be weighting. If GOOG moves 2% it's alot more money than if DELL moves 2%. You'd have to own a lot more DELL per share to keep the % change valid. You'd have to buy a certain amount based on its price relative to something (SPY perhaps) when it went 5 star.

I think what Dave is saying is pretty valid. If you are considering purchasing a stock and it has a 1-2 star rating you might want to find out why.  That's the real purpose of this "game".

Report this comment
#41) On March 28, 2008 at 9:02 PM, nuf2bdangrus (< 20) wrote:

My only question is had anybody tallied up all the "outperform" vs "underperform" picks and see in quantity how they spread across a bell curve?  I am just wondering if that curve would be evenly distributed, or there is a long bias.  of course, the S&P should have a"long" bias, but me thinks me sees to many "outperforms" in quantity, and of course, they can't all outperform.

Report this comment
#42) On April 22, 2008 at 9:06 PM, pytheian (32.63) wrote:

I can't reconcile:

"The star rating portfolios are forward moving performance, not backdating performance.  For example, for the 5 star portfolio shown in the graphs above, all the 5 star stocks on 11/6/06 where added into the portfolio on that date."

With using TTM figures. Does that mean that once a stock becomes 5-star, you add its TTM in to the weighting? Aren't the highest TTM stocks those most likely to be 5-star rated in the first place? What would be the return if you bought the stock immediately after it joined a certain CAPs rating level, then sold it as soon as it changed? Asking because someone demonstrated that you couldn't capture Valueline's claim returns by the time you received the new rating and put your order in.

Thanks!

Report this comment
#43) On April 23, 2008 at 9:15 PM, TMFJake (63.55) wrote:

Pytheian, no we don't add a 5-star stocks TTM into the weighting when it is added to the index.  On the day this time series was started, all 5-Star stocks as of 11/06 were put into the index, and the performance of the 5-Star index was measured from that day forward until the end of the 12-month time series.  If a stock dropped below 5-stars it was removed from the index.  If a stock transitioned from 4 to 5, it was added to the index, and it's performance calculated in the index from the day of transition forward.

Hope this helps,

jk 

Report this comment
#44) On April 29, 2008 at 12:15 AM, pytheian (32.63) wrote:

Absolutely, thanks!

Are there any plans to publish yearly return indices for each class of stock (i.e. adding them when they get upgraded, selling when they drop, etc.)? I can only assume that a portfolio short all 1s and long 5s would have a staggering return. Alot of data to manipulate but would be fascinating.

Report this comment
#45) On May 14, 2008 at 9:08 AM, overley (70.36) wrote:

Now if some genius can offer a mutual fund or etf of 5 star caps picks ( hello vanguard ) we will really be getting somewhere.

Being an obsessive type, I can now check my score ten times per day and leave my portfolio alone, thanks Fools.

Report this comment
#46) On May 17, 2008 at 2:30 PM, jmkk (23.07) wrote:

I am just getting interested in small caps, especially 5 star caps.   One I am especially excited about but will not hold for too long.  How do I get rid of it?

Report this comment
#47) On June 17, 2008 at 11:49 PM, d1david (29.85) wrote:

Thanks for all your hard work on this site.  I agree with TDRH, I would put money on a "caps 5 star " index fund, especially if it shorted a few of the 1 star stocks (I am very bearish right now and will be till the financials come clean)

Report this comment
#48) On June 18, 2008 at 12:16 AM, FleaBagger (34.76) wrote:

Second overley. One of the most useful things about CAPS is that we can get a lot of checking and tweaking out of our system without commission-killing our real money portfolios.

Report this comment
#49) On June 30, 2008 at 5:57 PM, thebeaconrocks (< 20) wrote:

Re: Your "Community Intelligence Works" posting on 11/29/2007

David, 

There is _nothing_ predictive about CAPS ratings!  

The evidence you give is entirely backwards looking: "TTM returns" stand for "trailing twelve months" returns, which means you look backwards and see how well the stock price did over the *previous* 365 days.  Thus, the five-star stock graph is simply an average of the price changes of stocks that the community deemed valuable _after_ it was able to read the earnings reports etc.  In other words:  "Let me show you the company's information, and you'll tell me how well the stock price did in the *last* year".  There is nothing predictive. 

If you really want to prove that CAPS ratings have predictive qualities, I would urge you to publish non-contrived results that indicate this: for example, show us the return graphs of star-rated portfolios: Do something more formal and verifiable than that short little sentence that Tim adds:  "To wit, our researchers found that a portfolio of stocks purchased as they became four-star stocks, and sold when they became 4.5-star stocks, returned 41% -- the best returns any CAPS segment had to offer. " For example, you could create five CAPS players correspond to the five star ratings and they would automatically pick the respective stocks as soon as they enter/exit the respective ratings.  Yes, that would mean some work, including removing the max picks restrictions etc. Are you up to that challenge?

Using TTM is highly misleading and unfair to your customers.

Remark: Please note that I don't deny that "Community Intelligence works" -- dnying this would contain as little information and would be as unverifiable as your claim that it does.  There is certainly some value in allowing CAPS users talking to each other -- and if it's just advertising dollar for your company. 

Report this comment
#50) On July 08, 2008 at 10:56 PM, TMFJake (63.55) wrote:

I don't think there's a contradiction at all. And, in fact, we created the CAPS ratings quintiles measured in this time series exactly as you propose in your challenge to us. :)

Each rating quintile is treated as an index and measured over the 11/06-11/07 time series. To use the 5-star stocks as an example, we are NOT looking at stocks that recently achieved 5-star status and looking at their TTM returns. We ARE measuring the performance of stocks that have already achieved 5-star status over a time series. That is to say, on 11/06, all stocks that had achieved 5-star status are included in the 5-star index. Throughout the time series, stocks would be added or removed from the 5-star index depending on whether they crossed into or below the 5-star rating threshold.

Tim used the TTM term to refer to the performance of the CAPS ratings quintiles over the first year of our public launch.

Hope this clarifies and helps.

Foolishly,

jk

Report this comment
#51) On July 31, 2008 at 9:49 PM, LordZ wrote:

WOW...

impressive...

 

Report this comment
#52) On October 03, 2008 at 11:50 AM, bostoncelitcs (73.66) wrote:

We need an update on the statistics from when the S&P 500 peaked on 5/19/2008 at 1426.

Report this comment
#53) On October 16, 2008 at 2:33 PM, menelyik (< 20) wrote:

The whole concept of CAPS is to pick stocks which will "outperform" or "underperform" the market as a whole.

This is such a useful metric when the whole market is tanking ....

Report this comment
#54) On October 15, 2009 at 12:44 PM, Denverhoss (44.14) wrote:

Sorry for a new post in an old blog, but there was a good question asked above.  Can you download historical (or current) caps ratings for stocks?  I'm using an excel spreadsheet and I can easily pull data from Yahoo.  But when I want to add Caps ratings and Under/Out Perform info from Fool I have to enter it manually.  Anyone able to pull this from within Excel, or at least copy and paste from a web site and get numbers instead of the star graphics?  Thanks!

Report this comment

Featured Broker Partners


Advertisement