Don't pay too much attention to CAPS rankings
Yesterday I was perusing the blogs and came across a comment I've heard too many times. An individual was citing another individual's CAPS score and accuracy, implying that the values in some way lent credibility to the opinions they were voicing. It's quite distressing to read things like this, not because I care what people do on CAPS, but because I worry that people may make decisions with their hard earned money based only on recommendations from Top Fools. I'm going to explore in detail the accuracy component of the CAPS ranking, but first a note on score.
Browse through the profiles of the leader board and you'll see two tickers on almost every single one. GMGMQ.PK (yes, that's GM equity), and PXCE.OB. It is near certain that GM equity is worthless, and PXCE is a shell of a company that has most likely had it's price manipulated by "pump and dumpers." So, to rack up points, you simply red thumb these (in some cases more than once) and it's an easy 100 points plus an additional "correct call." This sort of thing is clearly not actionable in practice and so is irrelevant from an investing stand point, yet in some people's mind, the points it generates lends credibility to one's arguments. More important than a person's score is how they have achieved such a score. Have they made picks that are actionable in practice, and supported their pick with thoughtful commentary, or have they gamed the system?
Accuracy is critically important to CAPS scores, but is it really important? I'll often hear comments like "individual X has been right on Y% of their calls, therefore they should be listened to." Oh really? I will submit that it is not at all relevant, and in fact, a computer could beat anybody's accuracy. Let's explore.
It is well known (well enough that I'm not going to source this statement) that stock prices typically follow a random walk about some longer term trend. With a margin of "success" of only 5%, how well would you do if you simply made picks at random and closed them once the score exceeded 5? To do this we simply need to run a random walk simulator and compare the results to what human players have done.
Lognormal daily stock prices projected for 252 days, or approximately one year (not attempting to model intra-day). The price is not in absolute terms, but in terms of price relative to the S&P 500, so it begins at 1, and a price of 1.05 gives a success on an outperform pick. The pick, either outperform or underperform is chosen at random (this part isn't actually relevant). Volatility is fixed throughout the projection. I projected 10,000 picks, and closed each as soon as it generated a score greater than 5. No pick is closed with a negative score. Esentially, I'm trying to replicate with a simulator the kind of strategy most of the top CAPS players use.
Clearly the volatility assumption is critical. The higher the volatility, the more likely it will at some point be successful. So what is our assumption? I looked at three widely followed companies, and took a standard deviation of the difference in their daily returns and that of the S&P 500. Since prices in our model are relative, so too should volatility. The period of time is 1st quarter 2009.
BAC - 11.4%
XOM - 1.3%
AAPL - 1.9%
A 2% volatility during this period seemed like the most reasonable assumption to me for a real life (not CAPS) portfolio. Clearly in CAPS you can take on more risk than you would with your brokerage account. The results may surprise some people. Fully 85% of the picks were recorded as a success. That means, if you simply picked stocks at random, at some point, 85% of them would have beaten the S&P by 5%. Increase volatility to 5%, and the success ratio is 92%. If we use BAC's volatility, which in truth is the kind of pick a lot of CAPS players have been using over the past 9 months, (think triple levered ETFS like FAZ and FAS) the success ratio is a whopping 95%. So, in actuality, CAPS players, even the best, are underachieving the accuracy a computer could generate. Chalk it up to the picks they actually believe in and hold for a longer period of time.
I'll conclude by saying what I think everyone already knows, but perhaps at times forgets. When it comes to investing, it's not what you do, but why and how you do it. The Fool with a rating of 99 point something, score over 10K, 85% accuracy and average score of 5 is doing nothing more than playing the CAPS game, and should not be taken any more seriously than the Fool with a score <20. The strength of a person's research and argument is what's really important, and is the true value of this site.