Use access key #2 to skip to page content.

anticitrade (99.57)

Backtesting Investment Strategies

Recs

6

July 13, 2009 – Comments (8) | RELATED TICKERS: ICI , TRAD

When I first developed my investing model I was ignorant about my level of ignorance.  Despite my ignorance^2, I had the opportunity to meet with several large investment firms and venture capitalists about my creation.  One question that was consistently asked was:

Who the hell do you think you are?

However, another more relevant question was:

Have you done backtesting? And what were the results?

My answer at the time was, “Backtesting is hard.  Even if I did backtesting, you would then question the accuracy of my backtesting in the same way you question the accuracy of model.  Thus, it would be an ineffective means to validate my tool.” 

I was right about backtesting being hard, and probably right about it not providing real validation for my tool (this may be the subject of a future post).  However, as a user of my own tool, I thought it would help me sleep easier if I had a pretty good idea how my model would have performed over the last few years.

So I built it.  It took 2-3 months of thinking about it, and 2 weeks to actually build it.  To load an investment strategy takes about 8 hours, to gather all the price data takes about 12 hours (only needs to be done once for each year tested), to execute at varying portfolio sizes only takes about 5 minutes.

As I may refer to the results of my backtesting tool (I still need a cool name for it, any ideas?) I thought I would discuss its limitations upfront.

1. It does not factor in quarterly financial statements.  The results are based solely on the end of year financial statements from the 5 previous years

2. It can only make end of day decisions since price data is most available in this form (this amount of data alone is sufficient to bring my system to its knees).

3. I can not use 2/10 price determinants that I use on anticitrade.  So it does not perfectly model the investment decisions of my full system (for better or worse).  

4. Many modern stocks do not have an adequate history for consideration in my model, consequently the quantity of viable investments is lower.  

5. Many stocks that would have been a possible investment have since gone out of business, so their financial data was also missing from the analysis (probably inflates the results of my backtesting).

6. It does not account for transaction costs (although it will tell you the # of transactions).

That’s all the big flaws I have discovered so far.  As some of you may conclude, creating a system that automatically makes investment decisions based on fundamental analysis and relative pricing was not a trivial task.

If anyone has a fundamental strategy they would like me to test out let me know.  If I think it is worth 8 hours of my machines time, I will give it a run.

I don’t think I will be releasing the results of testing anticitrade for a variety of reasons.  The two biggest are that the testing is incomplete, and I gain nothing from releasing it (since validating my own system does not add to its credibility).

8 Comments – Post Your Own

#1) On July 13, 2009 at 1:30 PM, kaskoosek (99.70) wrote:

It is funny how I look at the income statement from 2005 to make a buy or not.

 

However, I also look at debt levels in 2008, because I do not like to get wiped out either through dilution or bankruptcy. Increase in leverage like GE is a no-no.

Report this comment
#2) On July 13, 2009 at 1:45 PM, portefeuille (99.74) wrote:

Could you maybe give some details of the algorithm behind your rating system? Just the basics, you can of course keep the secrets!

What goes in and what is the decision/evaluation process, "flow chart" ...

Report this comment
#3) On July 13, 2009 at 2:01 PM, anticitrade (99.57) wrote:

First I gather 10 years of historical income and balance sheets for every company I analyze.

I use that data to calculate 10 different prices for each stock.  I then use those 10 prices to determine a single "conservative" price, and "fair" price for each stock.

I then compare the calculated prices against the current price to determine if it is under or over valued.

That explanation is almost simplistic enough to make it totally useless.  After hearing the community response to Goldman Sach's algorithm, I have decided that I need to take security a little more seriously.

Report this comment
#4) On July 13, 2009 at 2:10 PM, portefeuille (99.74) wrote:

Yes, a little too general, but I can understand the secrecy problem.

Do you really use no quarterly (or more current) data? That would mean that a stock that the fair value your system assigns is a step function with 1 year steps unless you change the algorithm. So the only steps occur when new year end results enter your system and when you change the algorithm. Is that so?

Report this comment
#5) On July 13, 2009 at 2:20 PM, anticitrade (99.57) wrote:

In backtesting I do not use quarterly statements, but in  my working model I do (for the reasons you state).  However, the prices are not static eithey way since my model essentially adjusts the valuations to remove the effects of movement of the overall market.

So the valuation of a stock changes everyday dependent on the movement of every other stock in the market.  These changes are usually very subtle.

Report this comment
#6) On July 14, 2009 at 10:42 AM, lemoneater (84.46) wrote:

How about calling your investing model something like "Tradedition."

Report this comment
#7) On July 16, 2009 at 12:50 PM, mikecart1 (99.28) wrote:

Anticitrade is one of the smartest people I have come across on the internet.  A lot of people like to talk a lot of stuff but never back it up.  But Anticitrade has a unique system with his own site and isn't afraid to share his thoughts on stocks.  Basically Anticitrade is what Jim Cramer's Mad Money should of been.

Report this comment
#8) On July 16, 2009 at 3:02 PM, anticitrade (99.57) wrote:

Mike,

I think I might print your comment out and hang it on my wall.  It's not often I get a compliment, and even rarer that it's from a nuclear engineer with an MBA. 

One of the problems on the internet is everyone is eager to make wild claims, and no one is willing to hold themselves accountable for them (at least the ones that go bad).  This bothers me because there is SO much to be learned from mistakes.  I figure that right or wrong I am going to hold myself accountable, so I better watch what I say.  The ability to admit you are wrong is powerful.

Anyway, thanks for your comment.  I really liked your blog about how sick you are of all the bs advice (I remember nodding my head a lot when I originally read it a few months ago).

Report this comment

Featured Broker Partners


Advertisement