Use access key #2 to skip to page content.

TMFJake (60.61)

New Ratings Idea for CAPS

Recs

23

May 12, 2010 – Comments (34)

Recently, we tested a new ratings idea for CAPS. The idea is that it would be valuable to rate companies by categories in addition to future price performance.  As we do our research, many of us screen for companies that appear to be undervalued, or exhibiting strong growth... What is it that distinguishes these candidate companies from one another?  For many of us, it's understanding the quality of a business and its people... So, we identified key factors that we think distinguish great companies.  We rolled these factors into we might call the "5-Ps," People, Profitability, Position, Potential, and Purpose.

I tested the 5-P ratings with an email survey to anyone who has rated one of the ten most rated stocks on CAPS: Apple, Altria Group, Cisco, Johnson & Johnson, Google, Ford, Microsoft, General Electric, Bank of America, and Citigroup.  You can read my article on the results here:  5 Secrets of Investing Greatness.

Below is a summary of the results.  Each rating category is measured on a scale of one to five, with five as the most positive rating:

Key No. 1: People
Specifially, we asked members to rate companies on Shareholder-Friendliness, Management, and the CEO.

Key No. 2: Profitablity
Some of the indicators we feel are important include whether a company has more cash than debt. We look for companies that generate consistent revenue growth and deploy capital effectively, with return on equity greater than 10%. 

Key No. 3: Potential
Here we asked members where the business can go in years to come. Is the company at a relatively early stage in its maturity cycle? If the company's products are at a late stage in their maturity cycle, are they the best offerings available? Can you envision multiple futures for this company? Does this company benefit from economies of scale?

 

Key No. 4: Position
We asked members to evaluate competitive advantage--e.g. companies with cost advantages that can't be replicated, or companies that convey social status in the use of their products or services.

Key No. 5: Purpose
It's critical for a business to be surrounded by enthusiasm and to serve a larger mission. Ask yourself the question, "Would the world be worse off if this company didn't exist?"

Finally, here's how companies rank when we aggregate their scores across each of the 5-P Categories (People, Profitability, Potential, Position, and Purpose):

34 Comments – Post Your Own

#1) On May 12, 2010 at 5:18 PM, Melaschasm (88.39) wrote:

A more detailed rating system has the potential to collect valuable information.  However, it also means that the people doing the rating need to spend more effort to make their picks. 

This is an interesting idea, and if it was a secondary option within the existing rating system, It would definately appeal to me.

Report this comment
#2) On May 12, 2010 at 5:33 PM, chk999 (99.98) wrote:

The interesting question is "is this rating system predictive or postdictive". Meaning that if we used this now and then look at total returns in 5 years, how much did it predict.

Report this comment
#3) On May 12, 2010 at 6:59 PM, MegaShort (99.95) wrote:

I think you guys should improve the current system instead of making a new one.  People on CAPS have been making suggestions for the last 4 years, and there is some consensus around deemphasizing accuracy and emphasizing score/time and score/pick.

If you do a multivariable system I think simpler ways of organizing it might result in better performance for the model.  For example:

'Potential' and 'Risk' (something like estimating the alpha and beta for each stock).  Currently CAPS doesn't have any way to look at risk.  There are very risky 5 star stocks alongside very conservative ones although they are sure to behave differently.

Or, expanding it a bit more, 'Growth' (a guess on forward earnings growth), 'Value' (a price target), 'Rating' (5* strong buy to 1* strong sell), 'Risk'

Report this comment
#4) On May 12, 2010 at 7:00 PM, MegaShort (99.95) wrote:

PS please fix the stock screener.

Right now it will not screen for unrated stocks.  This sucks.

Report this comment
#5) On May 12, 2010 at 7:29 PM, ChrisGraley (31.89) wrote:

I'm not that enthused.

It would need to be weighted like the star system to be worth anything and I have a feeling that it will simply be a measure of past results.

There have been much better suggestions from other fools. I wish just a few of those were implemented or at least acknowledged. 

Report this comment
#6) On May 12, 2010 at 7:34 PM, portefeuille (99.77) wrote:

I think you guys should improve the current system instead of making a new one.  People on CAPS have been making suggestions for the last 4 years, and there is some consensus around deemphasizing accuracy and emphasizing score/time and score/pick.

I very much agree. Should the consensus have been forgotten it can quite easily be extracted starting here.

Finding the true gurus in CAPS

Report this comment
#7) On May 12, 2010 at 8:16 PM, MegaShort (99.95) wrote:

By the way porte what do you think of BBVA?  I know you are a fan of Santander (at least before their recent 20% pop) but I think BBVA may be the cheapest big European bank.  They have even stronger projected earnings and less leverage. I guess the biggest drawback compared to Santander may be less exposure to South America.

Report this comment
#8) On May 12, 2010 at 8:37 PM, ChrisGraley (31.89) wrote:

I'm a little ticked off that nobody picked up on my Satander joke.

Report this comment
#9) On May 13, 2010 at 1:33 AM, APJ4RealHoldings (26.14) wrote:

Uh that was wierd to include 5 year returns in that last chart. 

My personal thoughts, it seems a bit much & will likely add higher collection of misinformation as there will be much more uninformed people than those informed that will override rating data in a particular stock.  There's no check/balance on actual due diligence the rater has done and many are likely to either follow the crowd or go with their gut or go simply based on what they like/dislike on the "cover page" of the financials

Anyways, I agree with comments above from MegaShort, Portefeulle, and Chk999. 

portefeuille (99.97) wrote:

I think you guys should improve the current system instead of making a new one.  People on CAPS have been making suggestions for the last 4 years, and there is some consensus around deemphasizing accuracy and emphasizing score/time and score/pick.

I very much agree. Should the consensus have been forgotten it can quite easily be extracted starting here.

Finding the true gurus in CAPS

Report this comment
#10) On May 13, 2010 at 7:28 AM, XMFSinchiruna (28.57) wrote:

MegaShort

There is definitely no consensus from me regarding de-emphasizing accuracy. I now play for accuracy more than score, and my silverminer portfolio has benefitted handsomely from the approach. To change the rules of the game midstream would be patently unfair to those who selected their strategies accordingly.

Accuracy is a critical factor in identifying the best stock pickers in the game.

I sure hope no one is seriously considering such a fundamental change to the scoring system.

Report this comment
#11) On May 13, 2010 at 9:23 AM, portefeuille (99.77) wrote:

#7 I think it is a good idea to have shares of both, I still prefer Banco Santander. I do not have shares of either one (just indirecty via EURO STOXX 50 index certificates). Banco Santander will probably be my next buy. A recent list of some of my current positions is here.

Report this comment
#12) On May 13, 2010 at 9:29 AM, portefeuille (99.77) wrote:

Accuracy is a critical factor in identifying the best stock pickers in the game.

The "caps" game "accuracy is more or less useless in "identifying the best stock pickers in the game". see this post and this post.

 

Report this comment
#13) On May 13, 2010 at 9:30 AM, portefeuille (99.77) wrote:

"accuracy

"accuracy"

Report this comment
#14) On May 13, 2010 at 9:32 AM, SUPERMANSTOCKS (50.81) wrote:

I say stop messing with things you cant fix or don't need fixing

Report this comment
#15) On May 13, 2010 at 9:37 AM, portefeuille (99.77) wrote:

in "identifying

for the purpose of

Report this comment
#16) On May 13, 2010 at 9:40 AM, portefeuille (99.77) wrote:

#14 The "caps" game needs fixing and can be fixed.

 

fixed version of comment #12 above, hehe ...

The "caps" game "accuracy" is more or less useless for the purpose of "identifying the best stock pickers in the game". see this post and this post.

 

Report this comment
#17) On May 13, 2010 at 11:32 AM, edwjm (99.83) wrote:

I still think the most needed change is a provision to remove inactive players.  After a period of time, inactive players should be send a warning, and, if there is no response affirming that they still agree with all their picks, they should be removed.  The number of players does not matter as much as the currency of their picks.

 

A second change I would like to see is the removal of the S&P 500 comparison feature.  When my stocks go down, it is no consolation to know that the S&P 500 has gone down also.

Report this comment
#18) On May 13, 2010 at 12:30 PM, JaysRage (94.20) wrote:

The thing that I'd most like to see is a clear in or out for a particular stock.    For those stocks that bounce in and out of the 100 million cap range, I'd like to see them either in or out.   Set the range higher if you must but once a stock is in, I think it always should be in.    The way it currently is, it biases against green thumbs and gives advantages to those who scan constantly for the pump and dumps.   

Report this comment
#19) On May 13, 2010 at 12:34 PM, JaysRage (94.20) wrote:

If you are going to have five reasons why you pick a particular stock to do well and if current valuation isn't one of them, then I don't think it's practical.   

Report this comment
#20) On May 13, 2010 at 12:43 PM, portefeuille (99.77) wrote:

-------------------

Finding the true gurus in CAPS

Did someone already write this post? Hopefully not. I’m still a little young to be the guy who tells you the story that you just told him yesterday. This is an attempt to coalesce in a thread some thoughts that highly contributing members have had about improvements we would like to see on CAPS. TMFJake, I’m aware you are on the case and I appreciate your efforts. But after the last “upgrade” I think it would be best for our preferences to be on the record before any changes are made. For those of like mind, please keep comments on point so that we can maintain this thread and eventually arrive at a mutually satisfactory slate of prospective improvements to present to TMF. As attention spans are short, I’m going to make the suggestions first and then rant afterwards.

 

I. A screening tool to identify players that outperform according to individualized standards. Leave the rating system as is, just allow us to search within it for our own purposes. Stock and member screeners are already in place, unfortunately the data inputs are generally useless (risk tolerance??). My personal preference would be to look at score only, isolating outperform pitches on non-OTC stocks and eliminating the S&P comparator. I could go either way on leveraged ETF’s, but I’d definitely like the option to eliminate red thumbs on unshortable ETF’s

a. Option to isolate green thumbs or red thumbs

b. Option to eliminate .OB and .PK

c. Option to eliminate accuracy

d. Option to eliminate S&P comparator

e. Option to eliminate leveraged ETF’s

 

II. Make CAPS more friendly to those of us who like to write about stocks of actual companies, and easier for us to find each other and read each other’s work. Many of us follow sectors or a small collection of stocks and they are like our naughty, misbehaving, unpredictable children.

a. Emphasize comments, or provide preferences to see more comments on the stock page. Raising the comments and moving the news down to the bottom would be an improvement. I come to CAPS for collective intelligence, not comprehensive company data.

b. Put rec boxes back in the top bull/bear spots.

 

III. We want to be score leader on our pet stocks. Our ratings will inevitably plateau if we don’t resort to playing games with the scoring, so we want to be able to compete for something.

a. Change the leaderboard to reflect the players with the top cumulative scores across all their picks, the way that the score leader is chosen. We don’t care so much about that guy who randomly picked it at the low point two years ago because he liked the ticker symbol and racked up a double. I want to know the guy who got positive scores on it up and down nine times in a year.

b. Add a score leader ranking. Who are the score leader leaders? 

 

IV. Clean the game up just a little.

a. Why can we cancel picks during the 20 minute delay? I understand the delay, but not the cancel. You’re encouraging people to sit in front of their computers all day picking and canceling as they feel for a high or a low. My brokerage won’t let me do that, and I don’t really want to do it on CAPS either.

b. It’s irritating and misleading to see stocks that blew up maintaining 5-star ratings while they trade in pennies, because they’re under the threshold and there’s no incentive for anyone to close their outperform picks (if they’re still even around).

c. What’s the point of having a player score graph that only goes back a few months? If you can’t take it back to inception you might as well put that space to better use.

d. There are not 66000 people playing CAPS. People who were here for a week in 2006 shouldn’t be in the ratings. If they haven’t picked or blogged in three months, they’re most likely gone, let go. Send them an e-mail, if they don’t answer cut them out of the ratings. I don’t want to be an “AllStar” in a game where the journeymen all retired last year.

e. People who only registered to pump or flame and don’t even have picks shouldn’t be among that number either. They also shouldn’t be allowed to post anything if they’re too lazy or insecure to rate the stocks they’re yapping about. Let them stay on Yahoo.

f. Is there any way to fix the lags and errors that plague the scoring? If you’ve tried and you can’t, that’s OK. But it’s annoying to finally close a pick after a year with a 7.8 score and see it show up in the Ended column as a 4.9. I know, I’m the guy who doesn’t think Accuracy matters. But still …

 

And now the rant …

 

I come to CAPS to practice virtual investing in actual companies, and to read pitches and blogs by those who do that much more skillfully and profitably than I do. In return, I’ve contributed everything I could as I have learned, and I’ve been as honest as I can about my limitations and errors.

 

What could be better than being a top 10 ratings leader on CAPS? Actually, just about anything. Not the least of which would be if CAPS helped me make actual money in the market.Unfortunately, the current system fails to identify players who can be followed into outperform picks with real money with any expectation of not losing one’s shirt. High ratings themselves are particularly unhelpful as predictors of success, although the low ones are fairly useful as predictors of failure. When I first joined CAPS I took the word of anyone with a rating over 99 as gospel. Once I hit 99 myself and climbed as high as the 99.7’s, I continuously had to raise that bar as I was painfully aware of my own inability to grow rather than vaporize my own brokerage account. Eventually I realized that CAPS rewards longevity and obsessive-compulsive disorder far more than it does the ability to make money in the market.

 

Can CAPS identify true outperforming investors for those of us that want to follow them into long investments in real companies, where we can actually buy shares at their CAPS start price? It is very possible that it cannot. After all, those investors are a small minority, and why would they waste time on CAPS or any other stock site when they could be quietly enriching themselves? Then again, maybe they are out there, maybe they’re very proud of themselves and want everyone to see what they can do, or maybe they just want to share. But if they are in CAPS, most likely scattered around the top 200, I want to find them. 

 

Apparently, there are some who believe that CAPS was not designed for this purpose and we are somehow misusing the system by seeking such a benefit from collective intelligence. If so, please carry on as you were. Our requests should not affect your CAPS experience in any meaningful way. If you feel compelled to write a “Love it or leave it” type response, I would encourage you to conserve energy and move on to another blog post instead. To the highest-rated, I don’t intend to denigrate your efforts. Some of you would certainly be among the highest-rated on my individualized list as well. But it’s been a rough couple of years and I’m ready to make a little money. Past performance may not guarantee future results, but I’m willing to weigh that risk. CAPS, are you ready to help?

-------------------

Report this comment
#21) On May 13, 2010 at 1:05 PM, Pennyperson (< 20) wrote:

Go port..I agree

Report this comment
#22) On May 13, 2010 at 1:10 PM, TMFJake (60.61) wrote:

Thanks for all the comments.  I do appreciate the reminder that adding new features shouldn't substitute for improving the existing service.

For those who want to reduce accuracy:  We monitor this regularly and haven't found any convincing evidence that reducing the accuracy weighting improves the predictive value of CAPS ratings.  We are looking at a number of ways to imrove the rating system, but  accuracy hasn't bubbled up in our analysis as a compelling change.  I do believe we will change the market cap / price threshold to one based on average dollar trading volume--for those of you who have advocated for this, we hear you and our tests indicate this would be a good improvement.

For those who want to find a better way of finding great stock pickers on CAPS.  I agree with you. Lots we can do here.  One simple example is when we implemented the Stock of The Day feature, we identified the people who seem to have similar interests as you and who are higher rated.  We draw from their picks to populate your Stock of The Day.  But we don't tell you who those people are!!)&)&$%$  I hope to correct this in the near future...

Also, FWIW, these new ratings categories are meant to complement individual valuation work, not substitute or replace that type of analysis.

Fool On!

 

Report this comment
#23) On May 13, 2010 at 1:10 PM, portefeuille (99.77) wrote:

a related new post.

Report this comment
#24) On May 13, 2010 at 1:17 PM, portefeuille (99.77) wrote:

We monitor this regularly and haven't found any convincing evidence that reducing the accuracy weighting improves the predictive value of CAPS ratings.

You will not find convincing evidence until you implement the appropriate changes (get rid of that ridiculous "caps" game "accuracy"), as the change will induce a change in the players' "behaviour". This post demonstrates just how riduiculous that "accuracy" is.

Don't pay too much attention to CAPS rankings

Report this comment
#25) On May 13, 2010 at 1:25 PM, YodaBuffett (< 20) wrote:

I like accuracy, but I feel that average score per pick is a better metric.

Realistically someone could be dead on and only outperform the market 5% or so per pick versus someone who's less accurate but has an average return above market of 30% across their entire portfolio...

Report this comment
#26) On May 13, 2010 at 1:33 PM, portefeuille (99.77) wrote:

riduiculous

ridiculous

Report this comment
#27) On May 13, 2010 at 1:50 PM, capri25 (24.11) wrote:

I have come to realize that my "ideal" and "desired" rating is between 35-45%. The current method of scoring does not take into account risk or volatility. Funny, but in my real life - and in my CAPS choices, I take account of both. And, I use my picks to monitor my performance in a number of ways. I could probably do better vis-a-vis other members were I to load up on one particular type of stock. But then, I'd be at the top of the heap one month and at the bottom the next month. I would rather have STEADY performance. And the CAPS rating system ignores that, so I use the technology but ignore the ratings.

Report this comment
#28) On May 13, 2010 at 3:11 PM, APJ4RealHoldings (26.14) wrote:

TMFJake (89.02) wrote:

"I do believe we will change the market cap / price threshold to one based on average dollar trading volume--for those of you who have advocated for this, we hear you and our tests indicate this would be a good improvement."

 

THAT WOULD BE AWESOME & would be the BEST, & single GREATEST improvement to the current CAPs system in place.

CAPs is the best transparent, free, security picking website in the world right now ; and it is refreshing that there are still yet some (very few) great ideas out there to even improve upon this great system & website!! 

The interface is amazing, intuitive, easy to use, and easy to DRILL down to - ie go to players page, go to security, go to specific pitch, go to tag, go to industry, see top picks in tag/industry, see player's sector information, see players' blogs.

AMAZING.

I think one of the most awesome, underrated CAPs features is SECTORS tab for each player - that is POWERFUL information - and you guys have made that tab PERFECT over the last couple of years.

Report this comment
#29) On May 13, 2010 at 3:21 PM, portefeuille (99.77) wrote:

#28 I have written a few times that the $1.50 threshold, the 20 minute delay cancel ability and the "caps" game "accuracy" are the 3 most absurd features. So one of those may finally be fixed. a semi-ironic yeay!!!!

Report this comment
#30) On May 13, 2010 at 4:56 PM, APJ4RealHoldings (26.14) wrote:

#29, whatever happened to your 20min time delay player? how did that go?

Report this comment
#31) On May 13, 2010 at 5:32 PM, portefeuille (99.77) wrote:

I have not used that player (now called portefeuille12) for its intended purpose except for the 9 calls I made on 04/01/09. I thought it was not really my thing to prove something by experiment, I am a theoretical physicist, so I thought the algorithm posted in comment #10 to that post by zzlangerhans was enough (as I wrote in comment #1 to the "follow-up" post by zzlangerhans), hehe ...

Report this comment
#32) On May 14, 2010 at 12:03 AM, TMFJake (60.61) wrote:

Thanks APJ4RealHoldings!  Appreciate the kind words.

And Porte, always appreciate your passion even if I don't always agree...

Report this comment
#33) On May 14, 2010 at 4:59 AM, megalong (< 20) wrote:

TMFSinchiruna (98.46) wrote:

"There is definitely no consensus from me regarding de-emphasizing accuracy. I now play for accuracy more than score, and my silverminer portfolio has benefitted handsomely from the approach. To change the rules of the game midstream would be patently unfair to those who selected their strategies accordingly."

Honestly I don't care that much about fairness to existing players' scores. I think the performance of the model is way more important.  But since some people feel strongly about it, it could easily be dealt with - for a few months while a new scoring system is being tried out, both scores could be displayed, or a new scoring system could be opt-in.

TMFJake (88.27) wrote:

"For those who want to reduce accuracy:  We monitor this regularly and haven't found any convincing evidence that reducing the accuracy weighting improves the predictive value of CAPS ratings.  We are looking at a number of ways to imrove the rating system, but  accuracy hasn't bubbled up in our analysis as a compelling change."

Thanks Jake, that is interesting and it is good to hear that you guys are talking about changing the system.  I just wish TMF involved us more on this issue.

Deemphasizing accuracy might not improve system performance, but I doubt it hurts it significantly, right?  And of course port's point about it influencing behavior (negatively) is right on.  I hope you also run some numbers and consider other important measures like performance/pick, performance/time, risk adjusted performance and other variables mentioned here.

Report this comment
#34) On May 18, 2010 at 3:55 PM, FleaBagger (33.65) wrote:

Thanks for all your help, TMFJake. Did you get my note about the "related articles" on HEAT's page not being related? If there's an easy, effective way of fixing that (and fixing/preventing it on other pages), I would sure appreciate it. It's a frustrating waste of time to click on related articles on a stock page and look for info on the company, and find nothing.

Report this comment

Featured Broker Partners


Advertisement