Use access key #2 to skip to page content.

How Green is our Valley ( ie CAPS) : A journey in Sentiment Analysis



January 24, 2010 – Comments (12)

I started this last weekend – couldn’t complete – also I was expecting an impending reversal or “corrective” – so we can maybe add some additional datapoints.  And there were some serious data challenges with the MF Screener

 This is a bit of an off-shoot from my previous post 

Is a CAPS based Fund plausible? Attempt at Quant based strategy”

First of all – do not get too much worried – this is January – as usual new year enthusiasm and all – I’ll not be furiously blogging. Reasons as follows:

(a) Inertia
(b) Lack of time
(c) Lack of interesting topic

Additionally, I will be gone for a while in Februrary and won’t possibly be accessing CAPS.

Anyway  the previous post was essentially a “Long Oriented” or Bullish leaning study – to understand what could work from a CAPs rating perspective. As I was doing it – especially in the backtest of the 9/19/2008 Long portfolio – I noticed that I could only find 27 stocks while I found about 110 in early March ie Fools rated more stocks positively in late Feb – not so much in Sep 2008. This got me thinking – could we use CAPS rating/bias towards understanding market direction?

Binve , the smart guy he is – posed the counter question – can there be a Signal – Bearish ( You know his slant  : )   ) – I will focus on the signal part.

So here it is – my study ( You draw the conclusions – and it should be pretty obvious its no Holy Grail – well if it was – I wouldn’t be blogging about it – would I?)

2 sections

(a)    CAPS Bullish/Bearish Index

(b)   Compare Contrast with general Investor Sentiment Data


A.      The CAPS Bulls/Bear Stock Rating Index

Looking at the table –you’ll see its pretty straight forward –

1.       Take all the Rated Stocks Day 1 of the month –

2.       Classify 4+ as Buys < 3 as sells and 3 as holds.

3.       # of Buys/# Sells is the Bull/Bear Spread



Pretty easy: Does CAPS or Can CAPS react in anticipation of moves – and on a comparative basis to the general investing public ( mentioned here very affectionately ..ahem as the “herd” or “sheeple” etc)

Issues with the Data: I had to hand run everything – making it tedious – the simple fundamental problem with this kind of data is “survivorship bias” – I was hoping to avoid dealing with it – however CAPS forced me to .  In the month of April 09 – about 80+ stocks disappeared as per the CAPS database in terms of rating – it’s an error – the stocks in this list are like TOL etc…..and then in July the # of rated  Stock  count goes up by about 90 stocks! The set is not 1-to-1 swap – but most stocks comeback in the interim period – however majority was in that month. ( 4C Controls one of our old Red thumb favs amongst them) . I really had to take a hard look back to figure out how to adjust – the period is very critical – the first rally and the first corrective happened in these months. Anyway I simply forced survival in these months – ie ensured that the denominator of the index was consistent.



I wish I could say “TADA!!”. Well it opens up more questions and thoughts of introspection . My basic observations are on the chart – so I will not repeat most of it.

Generally CAPS Stocks ( read the distinction – since this is stocks rating based – its not CAPS members – the stock ratio which is being measured) – just went for a toss mid-2007 and looks like has remained so.

However – this Diffusion index as I see it – is essentially a Delta of 100 – ratio of Buys/Sells . So a negative number would have meant that there were more sells than buys – this has NEVER happened in CAPS. Even at the very bottom in April 2009 – Buy rated stocks held a small advantage.

My inferences/observations




(i)                  CAPS has maintained a fav or “select few” list of buys thru this period …this is the main reason why  I believe the Buys/Sell ratio has not climbed all that much. Not many new entrants and old sells still languishing

(ii)                Because if you care to click the Skeptic charm in CAPS – it shows about 2140 survivors – of about 70,000 players ( and some of them could be skeptics red thumbing Ultra etfs) – that’s less than 5%. Additionally the 100th percentile CAPS Score threshold has steadily moved up – when I started – about 1800 CAPS points meant you were Lord Gallagher or something – that number has edged up to about 4600 I think now.



(iii)               There’s no doubt CAPS promotes ( in the sense of rating etc) – ultra speculativeness. And then due to accuracy – it promotes ultra egotisticism and adamancy. I do not need to elaborate too much ( you do not need to look further – looks at one of my picks called PALM – I wish it provided much needed shade to me!!!)

(iv)              Well you can say this makes no sense – yes and no – because CAPS has a defense – Stock ratings are weighed towards Star Player picks….but Star players can maintain fiefdom thru the Accuracy counter ( my dear friend Tastylunch is unabashedly doing this currently for survival – and so do I – because mine’s pretty low in any case). This has resulted in non-closure and maintenance of old Red Thumbs

(v)                In a perverse way CAPS scoring system – encourages the ULTIMATE  - ACT & HOLD strategy. If you manage to catch a stock at the bottom – and it s a multi-bagger and good chance of not faltering – do not close it ever – the incremental score contributions of those picks can be astronomical. We all have learnt this first-hand haven’t we since Mar 2009?


On the other hand if you manage to Red Thumb at the peak – and that stock fell 70/80% - you can easily stay put even in a bull cycle – because you will lose points – but unless the stock registers close to an all-time high – marginal effect on your accuracy is zilch . There are a lot of players like this – especially the dormant – “open and shut” inactive players.


 SO WHAT”s USEFUL? ( Things to watch out for)

Its always the marginal movement – that’s what counts – not the absolute most of the time.

(a)    Bearish Signal: Another silde in the index - Run for cover – especially new low

(b)   Bullish Signal: If the index breaks above the Aug 2009 high of 4.1% - significantly – ie decent jump.  CAPS is not so good at timing – that should be obvious – but I think it would be a nice – directional indicator

The above statement is extremely opinionated and judgmental – there is no DATA/BACKTEST to prove this. So take it with whatever skepticism you can muster.


THE EGO-BUILDING PART: Comparison to general investment sources


Coastaltrader (CT as we call him in CIL – – pointed me to this Hewitt ( I am sure a lot of your own Company 401Ks are administered thru them ) site – where they publish data around 401Ks they manage. It’s a good side check it out.  (

What we wanted to understand was the extent of the so called “money-on-the-sidelines” [ There is definitely some element of truth to this – all the CAPS surviving bears – can’t or couldn’t have survived in the real-world without covering – but given their slant – my bet would be 100% cash for them. BTW – if you were wondering – I was a restrained long-especially post the Jul09  S&P 905 and currently just about smidge long – mostly cash]

Anyway , here’s what I did – another very painful round of data entry – I am afraid. Took the monthly data available from Aug 1997 to Dec 2009 and smoothed it in Excel and plotted it ( For definition of the index look here:

You have to understand this index – its truly the “traded” portion of it – ie the “movement” in 401Kfunds. Dormant money doesn’t play any role in it. So % Fixed Income days – is simply count of days as a percentage for the month – when more money ( trades) flew into Bonds


Its much better if you smooth the data for 3SMA and plot it against the S&P.




(i)                   There is some “Negative Divergence” ( TA term) going on – since 2003 between % trades in and out of the 401Ks. The  bottoms  steadily rose thru 2003-2007. So people who trade or rebalance 401Ks became steadily more defensive

(ii)                The opposite happened thru the bear – less and less people traded into Bonds with each leg of the Bear!!!

(iii)               Since Mar 2009 – there has been not a single “flight-to-safety” period ( going above 60%) – people have hung on – similar to the 2003 late period.First sign there was 2004. A scenario likely to repeat or possibly repeat in 2010 – IMHO!

(iv)              However the KEY thing is that  - its based on trades….Very few people do that. Here’s the plot of these “movements” as a % of Total balance on a Monthly Average



                Very few people touch their 401Ks…..if that happened – Markets will move in a way – which is UNTHINKABLE!!!

Thus 401K movements mainly come from 2 sources

(a)    Trades or Rebalancing

(b)   New money/Contributions

So the second piece – is the contributions. There are 2 available – one is the all in ( including Employer match ). Since most employer match is equity – I focused on the employee piece of it. You may very well look at the total balances – its available also – I personally believe – its slightly auto-correlated….since most people hold – and if they follow a decent balanced allocation – Bull Cycles will increase Equity Balance portions and Bears deplete the same – I have a bunch of comparisons – keep that thought in mind.

Here’s the current last 2 months pic

NOV 2009

DEC 2009

We are currently seeing about 27-28% of the “new” money going to “Safer” instruments. Lets look at some trending of this

Peak 401k Bullish Activity month :Feb 2007

It was around 21% - and almost the same in Oct 2007 – market peak month.

Now compare to the “scare the bejesus “ months ie

NOV 2008

Mar 2009

So about a 14% ie 66% additional movement. At current level – we are about 50% - almost midway between  peak to trough allocations. Pretty much the same retraces in Indices also – (Strange isn’t it?)

Now I tried to compare to the last bear ie the 2000 tech crash. The allocation data is not available there – but the balances are – and its good because it provides a view AT THE TOP!

Mar 2000

Ok ! Ballpark same number 22% - same as the 2007 early allocations – which means 80/20 rule applies here – people aspire to that Balance.

Next is Feb 2003 ( bottom)

And we also have the comparable allocation ( ie new money) here

Ok about 38% in Balances here – and 28% Allocation. (Comparable to the 35% allocation in Mar 2009). Seems to me there’s a rhtym of 7% cycle here…..2009 was clearly 1 amplitude higher!!!

We can also compare the Balances for 2007 and 2009 – here they are

Oct 2007:

Mar 2009:

And the balances as of Dec 2009:

The total balance allocation to cash/bonds went to 44.5% ( about 7% higher : ) than 2000 bottom) in Mar 2009. And it’s back – obviously 50% from there – to about 34%


Last section – can any class actually ANTICIPATE?

First a case in White-Noise or randomness….. S&P Index and 401k Index delta’s plotted together.

Its clearly random…..Ran the correlation analysis at 1 interval lag ( ie to see predictiveness)….Here’s the results for 401K “Traders” and CAPS “Raters”

Both are negative – ie counter-trend anticipatory – the 401k is basically useless <5% significance. The CAPS one- its better than nothing – but wont’ stand a model productiveness test – about 14%



Let me start with one chart borrowed from GV ( GoodVibes – blog


(a)    It shows investor – I call “Trader” sentiment. Clearly its bullish

(b)   The 401K Trader Index is also basically “bullish” –since no one essentially seemed to have abandoned stocks – during both the Jul and Oct corrections. Usually corrections are marked by at least 60% trading into bonds etc month!

(c)    CAPS All-Stars are BEARISH – on a stock rating sense. Its calling this a Bear Market Rally

(d)   Overall CAPS players I guess are Bullish - <5% Skeptics currently

(e)   Money on the sidelines and “new money” contributions: This is neutral – 50% retracement towards peak – in line with Index retraces

(f)     Also based on a Ultra-Bullish allocation of about 21% cash – there room to go as far as 401K investors


My feeling is there will be correction here ( possibly started) which will force that move to “safety” – but given the allocations – there seems to be a distinct semblance of truth about money in sidelines.




Should be pretty obvious this was a painstaking blog. It would interesting to see the number of recs :) – LOL!


12 Comments – Post Your Own

#1) On January 24, 2010 at 8:57 PM, CoastalTrader (94.59) wrote:

Keep at it AC.  Just becuase this data isn't particularly predicative in terms of anticipating turns, it may well prove valuable in other ways such as marking "irrational exhuberance" or excessive bearishness.

Report this comment
#2) On January 24, 2010 at 11:04 PM, UltraContrarian (31.41) wrote:

Great post as usual AC.

I ran a similar backtest to your last thread.  Looked up stocks that rose to 4 stars between 2/1/2009 and 3/5/2009 vs those that rose to 5 stars between 2/1/2009 and 3/5/2009. Further categorized those into "good" fundamentals and "bad".  (Bad being characterized by negative 2008 earnings, negative tangible book, debt to equity over 1, current ratio under 1, or negative 2 year revenue growth.  I checked by hand to confirm this data would have been current as of 3/5, not current data being applied back.)

It turns out that the "Rising 5 Stars" significantly outperformed the "Rising 4 Stars" (which was contra TMF's previous note on intra-star changes.)  And not surprisingly, the stocks with "bad" fundamentals significantly outperformed the stocks with "good" fundamentals during the "junk rally".  If you look back at the previous 4 years, these same stocks had significantly underperformed, setting them up as good values.

10 Month Returns

Portfolio Maps

Portfolio Maps

I'll post more later.

Report this comment
#3) On January 24, 2010 at 11:51 PM, Tastylunch (29.52) wrote:

 There’s no doubt CAPS promotes ( in the sense of rating etc) – ultra speculativeness. And then due to accuracy – it promotes ultra egotisticism and adamancy...

but Star players can maintain fiefdom thru the Accuracy counter ( my dear friend Tastylunch is unabashedly doing this currently for survival – and so do I – because mine’s pretty low in any case). This has resulted in non-closure and maintenance of old Red Thumbs

yeah exactly. That's very well said.Although I waould add it also result in non-closure of greenthumbs as well and may even be worse there since you can lose far less points!

CAPS doesn't have enough "pain" as I call it  and lets you engage in behavior that would make you broke in real life.

One solution I've thought is having a CAPS players rating be a one year rolling rank  as a opposed to all time (or some combinatio there of). If they had that a player couldn't do what I've done with my CAPS port over the past year and still be an allstar. They've got to make players more accountable I think.

As awful as it looks I have picked up about 6-7 points of accuracy doing what I've done.In the long run it should be pretty beneficial.

You know what's funny is how much your first graph jives with my own feelings about this palce. The CAPS blogosphere and pitches as ana aggregate felt much much smarter fall-07 to spring 08 than it does now .

I dont know why that it is. Seems like a bunch of top players have left. certianly there are just more bloggers period so there may be more noise to signal in the blogosphere and there is the whole politicial mess which seems to be bringing the inner ideologue in everyone.

Alos the ticker pages have changed in a  way that I think really discourages people from making picthes now, which I think may dumb down CAPS a bit. TSIF, Ultralong and ZZlangerhans are pretty much the only guys consistently bangin out really good pitches now.

CAPS is so young you have to wonder if it will recapture that dynamism the place had two years ago.  And whether that factors into what you see in your first graph.

I personally don't think the diffusion index will likely ever go negative unless you have true depression level apathy bottom. CAPS has all skill level and people's natural instincts and the wall street system is naturally encourgaing of bullishness.

Kinda funny doens't look like to many CAPs players listened to Goodvibe's bottom call does it?

can’t or couldn’t have survived in the real-world without covering 

Right I was actually already out of all my long term shorts near  the mrahc bottom (I was still bearish but couldn't get a good read on how things were going legislative wise, was worried baout the stress tests) actually swing traded HBAN and FITB long in March and April. I did have a disastrous episode with FAZ where I made 25% or so on options expy but fooolishly held through the weekend and promptly lost the entire gain.

Anyway my net portfolio exposure to shorts was about 2-3% at the time.

RE: Money on the Sidelines- I'm not necessarily a big fan of the conept. If I can remember the site I'll link a smart money/dumb money indicator I think is more useful.

FWIW it has gone to slight top mode lately.

great blog Anchak! I have feeling this will be the top blog tomorrow. It definiely should be!

outstanding stuff, I hope the Fool compensates you for this somehow!  This ust ahve taken you a long time to do!

Report this comment
#4) On January 24, 2010 at 11:52 PM, UltraContrarian (31.41) wrote:

By they way anchak, have you read "The (Mis)Behavior of Markets"? I just finished it, it's really good.

Report this comment
#5) On January 25, 2010 at 2:11 AM, Tastylunch (29.52) wrote:

A bit related , I've been spending most of the day investigating Decision Moose and Zeelotes Riding the Wave (and the MarkW variant)

 my math is so rusty it's going to take me awhile to figure out how to do it but I think you'd get the concepts in a snap (if you you haven't researched them already)

both to me seem to have superior results to any market timing/rotation system I've ever seen~ 40-50% CAGR

(Decision Moose has been in action since 1989)

I think they may actually be able to be improved upon as well, bit seem to make the assumption that noncorrelation is enough, I imagine having a few negatively correlated choices could furtherjucie their returns.

Report this comment
#6) On January 25, 2010 at 6:27 AM, portefeuille (99.44) wrote:

... which seems to be bringing the inner ideologue in everyone.

At first glance I read that as "inner monologue". And I really liked the idea, it is "probably very true". People use the blog posts to write down their inner monologues. Most of them shouldn't do that (my inner monologues appear to be centred on linguistics and Düsseldorf, hehe). 

Report this comment
#7) On January 25, 2010 at 10:16 AM, binve (< 20) wrote:

AC, man, yet another fantastic post!!

Again, thank you very much for the props!

I am still pondering what you wrote, and like you mention above, no "Tada!" conclusion jumps out at me either.

But I think the true use and value of this post comes from your thoughts and your reasoning about why the trends are, and why they are subtle. Fantastic thought processes!

The bullish bias is definitely evident. So many complain that "Caps is so bearish", but it is really not supported by the data. A few top-players are, and a few loud-mouthed bloggers (such as myself :) ) are. But the data supports knife catching on the way down, but *not* a huge bullish spike at the bottom or the last few months.

Why is that?

I think you hit on it. Bullishness / players who are bullish on the issues in the index are maxed out. This very much goes with the "money on the sidelines" theorey. I tend to agree with Tasty, I don't really buy it. This post is a very good look at money flows from a *lot* of different perspectives (click on the link at the bottom of the post, a lot of great charts - ContraryInvestor: Of Mountains And Molehills - )

So I think your other thought, which was to look at this data as a sentiment indicator, or a "smart money vs. dumb money" indicators (they way Guy Lerner at the Technical Take does) is *much* more useful. I know that the data looks very noisy/random, but I know you are on the right track. I think there is something to be uncovered there.

But as always, you write the *best posts*! I love your thoughts. Thanks man!..

Report this comment
#8) On January 25, 2010 at 12:15 PM, UltraContrarian (31.41) wrote:

Just when I despair about the lack of interesting content in the CAPS blogs, anchak delivers with more great research.

For your first bull/bear sentiment graph, I think you may be making a mistake.  Isn't the CAPS rating system designed to have an equal number of 1, 2, 3, 4, and 5 star stocks?  I believe they rebalance the cutoffs regularly (perhaps weekly) to keep these equal.  So your stock rating index is an artifact of stocks that have been delisted through bankruptcy, merger, etc.  It should naturally fall over any period of time since 1-2 star stocks are more likely to be delisted than 4-5 star stocks.  The similarity to the 2008 crash is pure coincidence.  Correct me if I'm way off base.

Report this comment
#9) On January 25, 2010 at 12:26 PM, anchak (99.86) wrote:

I'll reply to comments in detail later.


Quick one on #8.

UC just added you as a fav.....but then I hope this is the right profile - I have been here now for 2+ years, you know :) I miss IBleedConcrete, and now you have Streetflame

They don't as far as I know- that would be a little contrived......They weigh the pick by your current rating though

The survivoirship bias - ie delisting issue is definitely real. ONe has to adjust for it.....I would have put that extra effort in - in terms of doing cohort studies - if I saw something like TADA!. Also it would take ,me much more effort and data extraction - which really cant do with just the screener

My goal was plain and simple - try to get an alternative indicator


Report this comment
#10) On January 25, 2010 at 12:37 PM, UltraContrarian (31.41) wrote:

"They don't as far as I know- that would be a little contrived"

Ha, of course the whole thing is contrived and ad hoc. It's amazing that it works at all.

Not only are 1-2 star stocks more likely to get delisted, they are more likely to slip into no rating territory as well.  I appreciate the desire for an alternate indicator but I don't think this is it.

Report this comment
#11) On January 26, 2010 at 11:46 AM, floridabuilder2 (99.35) wrote:

great post...  When I first came on here there were a lot of top cap players blogging about how bad the economy was going to get including housing and they were correct.  However, then a lot of these same bloggers switched to peak oil, commodities, etc.... 

I for one looked at commodities as a trade but just like the housing bubble, didn't think commodity bubble was sustainable.  I think a lot of people got killed when that thing tanked. 

Then most of 2009 was bulls laughing at bears, rightly so for thinking that the market would go down forever (the exact opp of the housing bubble and commodity bubble line of thought).  When I look at the blogs now, I don't see a lot of really strong opinions about the market in the sense that everyone is pounding on the same subject or debate the same subject.  I'm not on caps as much so maybe I am wrong. 

I take this as a sign that the market has peaked and everyone is waiting to see what happens next

Report this comment
#12) On November 22, 2010 at 4:14 PM, Mary953 (62.05) wrote:

AC - I wanted to tell you that You are one of my very favorite Fools!  I know that you already know that, but I have left a Thanksgiving blog and wanted you to find it.  Can't believe I had to come back this far to find your last real blog.  No wonder everyone thinks we just decamped!

Mary Ann

Report this comment

Featured Broker Partners