Systematic Relative Strength

When the facts change, I change my mind. What do you do, sir? -John Maynard Keynes

Although their theories seem to be no more or no less accurate, British economists always seem to have better quips than other economists. At the same time, Keynes brings up an important point. What do you do when the facts change-and how do you know if the facts have changed?

CSS Analytics has an interesting post on this subject today. The author points out a problem with forecasting:

The strangest thing about the forecasting world is not that it is a dismal science (which it is) but rather that forecasters share some remarkably primitive biases. Whether you look at purely quantitative forecasts or “expert/guru” forecasts, they have one thing in common: they rarely change their opinions or methods in light of new information. In fact, what I have noticed is that the smarter the person is and the more information they seem to possess, the less likely they are to change their mind. Undoubtedly this is why many genuinely intelligent and knowledgeable experts have blown up large funds or personal trading accounts.

It wouldn’t be so sad if it weren’t true. This lack of adaptability wiped out even a couple of Nobel Prize winners at Long Term Capital Management. The problem really is one of properly structuring one’s decision framework. CSS Analytics observes:

Ask a person to give you an opinion on where a market is going, and then notice what happens when the market goes dramatically the other way along with news announcements that seem to conflict with their thesis. Most of the time this person will tell you that they have not changed their mind, and in fact that it is an even better price to buy (or short). Models or systems suffer from the same problem–they typically do not adjust as conditions or regimes change…

Here, the decision framework is fixed. Since humans are naturally wired to seek out confirming evidence and to ignore disconfirming evidence, that is what happens. All evidence begins to be massaged to support the opinion, which is assumed to be correct. The story from CSS Analytics below is sadly familiar to most investors, almost all of whom start out as value investors:

I spent my early investing experiences as a “value investor” and let me tell you that I learned the hard way many times that the market was more often right than wrong. It was uncanny how well future fundamentals were sometimes “forecasted” by price. At the time, I had no knowledge of technical analysis and lacked the intellectual framework to synthesize a superior decision-making method. Of course, I would ride that “under-valued” stock with a price to book ratio less than 1 all the way to being a penny stock before I gave up. I also sold many of my winners far too early because their P/E indicated they were no longer undervalued. Some of these stocks went on to go up 400% or more, while I was content to make 25% profit. I did the exact opposite with overvalued stocks or stocks with crappy fundamentals. I was heavily short Fannie Mae and Freddie Mac as well as General Motors in early 2007! Of course, I got my clock cleaned and got margin-calls long before they plunged almost to zero. This was a case of being right, but too early to fight the sentiment of the crowd.

As a wise man once said, “Being early is indistinguishable from being wrong.” Even knowing that we should avoid a fixed decision framework, however, doesn’t really get us any closer to knowing how to handle the forecasting problem. CSS Analytics describes their epiphany:

It finally dawned on me one day that good forecasting (or decision-making) was a dynamic process involving feedback. In fact, the actual information used to make the initial decisions need not be complex as long as you are willing to adjust after the fact.

I put the whole thing in bold because I think it’s that important-although I don’t think our systematic relative strength process is forecasting at all. It’s simply a decision-making framework that incorporates a specific kind of feedback: the most important feedback to an investor, which is price. Price change tells you whether your decision is working out or not. If something isn’t working, you kick it out of the portfolio-that’s adjusting after the fact. Price is not complex at all, but it is the one mission-critical piece of feedback that is needed because that’s how every investment is measured.

Investing is one of those weird fields where clever opinions are often more valued than results. Certain gurus still get media exposure and sell hordes of newsletter subscriptions because subscribers agree with their bullish or bearish opinions, even though an outside service like Hulbert can demonstrate that their actual performance is abysmal. In professional sports, if you suck, you get released or sit on the bench. If you really suck, you don’t make the roster in the first place. Sometimes, in the investment field, if you really suck, you get to be on CNBC and have well-coiffed, polite hosts take your opinions seriously.

I think the description of good decision-making as a dynamic process involving feedback is very concise. Incorporating feedback is what makes a model adaptive. Models that are based on historical data ranges blow up all the time for this very reason-they are, in effect, optimized to the historical data but cannot always incorporate feedback. Continuous measurement of relative strength is not unlike former New York City mayor Ed Koch’s greeting, “How am I doing?” If the answer is ”not well,” then it’s time to ditch that asset and replace it with another one that has the prospect of better performance. The beauty of a relative strength model is that when the facts change, it changes its mind.

Man versus machine, art versus science, intuition versus logic—all of these are ways of expressing what we often think of as contradictory approaches to problem solving. Should we be guided more by data and precedent, or is it more important to allow for the human element? Is it critical to be able to step aside and say, with the benefit of our judgment, “maybe this time really is different?”

The Harvard Business Review recently took on this topic and a few of their points were quite provocative.

A huge body of research has clarified much about how intuition works, and how it doesn’t. Here’s some of what we’ve learned:

• It takes a long time to build good intuition. Chess players, for example, need 10 years of dedicated study and competition to assemble a sufficient mental repertoire of board patterns.
• Intuition only works well in specific environments, ones that provide a person with good cues and rapid feedback . Cues are accurate indications about what’s going to happen next. They exist in poker and firefighting, but not in, say, stock markets. Despite what chartists think, it’s impossible to build good intuition about future market moves because no publicly available information provides good cues about later stock movements. [Needless to say, I don't agree with his assessment of stock charts!] Feedback from the environment is information about what worked and what didn’t. It exists in neonatal ICUs because babies stay there for a while. It’s hard, though, to build medical intuition about conditions that change after the patient has left the care environment, since there’s no feedback loop.
• We apply intuition inconsistently. Even experts are inconsistent. One study determined what criteria clinical psychologists used to diagnose their patients, and then created simple models based on these criteria. Then, the researchers presented the doctors with new patients to diagnose and also diagnosed those new patients with their models. The models did a better job diagnosing the new cases than did the humans whose knowledge was used to build them. The best explanation for this is that people applied what they knew inconsistently — their intuition varied. Models, though, don’t have intuition.

We can’t know or tell where our ideas come from. There’s no way for even an experienced person to know if a spontaneous idea is the result of legitimate expert intuition or of a pernicious bias. In other words, we have lousy intuition about our intuition.

It’s easy to make bad judgments quickly. We have many biases that lead us astray when making assessments. Here’s just one example. If I ask a group of people “Is the average price of German cars more or less than $100,000?” and then ask them to estimate the average price of German cars, they’ll “anchor” around BMWs and other high-end makes when estimating. If I ask a parallel group the same two questions but say “more or less than $30,000″ instead, they’ll anchor around VWs and give a much lower estimate. How much lower? About $35,000 on average, or half the difference in the two anchor prices. How information is presented affects what we think.

We’ve written before about how long it takes to become world-class. Most studies show that it takes about ten years to become an expert if you apply yourself diligently. Obviously, the “intuition” of an expert is much better than the intuition of a neophyte. If you think about that for a minute, it’s pretty clear that intuition is really just judgment in disguise. The expert is better than the novice simply because they have a bigger knowledge base and more experience.

Really, the art versus science debate is over and the machines have won it going away. Nowhere is this more apparent than in chess. Chess is an incredibly complex mental activity. Humans study with top trainers for a decade to achieve excellence. There is no question that training and practice can cause a player to improve hugely, but it is still no contest. As processing power and programming experience has become more widespread, a $50 CD-ROM off-the-shelf piece of software can defeat the best players in the world in a match without much problem. Most of the world’s top grandmasters now use chess software to train with and to check their ideas. (In fact, so do average players since the software is so cheap and ubiquitous.)

How did we get to this state of affairs? Well, the software now incorporates the experience and judgment of many top players. Their combined knowledge is much more than any one person can absorb in a lifetime. In addition, the processing speed of a standard desktop computer is now so fast that no human can keep it with it. It doesn’t get tired, upset, nervous, or bored. Basically, you have the best of both worlds—lifetimes of human talent and experience applied with relentless discipline.

A 2000 paper on clinical versus mechanical prediction by Grove, Zald, Lebow, Snitz, & Nelson had the following abstract:

The process of making judgments and decisions requires a method for combining data. To compare the accuracy of clinical and mechanical (formal, statistical) data-combination techniques, we performed a meta-analysis on studies of human health and behavior. On average, mechanical-prediction techniques were about 10% more accurate than clinical predictions. Depending on the specific analysis, mechanical prediction substantially outperformed clinical prediction in 33%–47% of studies examined. Although clinical predictions were often as accurate as mechanical predictions, in only a few studies (6%–16%) were they substantially more accurate. Superiority for mechanical-prediction techniques was consistent, regardless of the judgment task, type of judges, judges’ amounts of experience, or the types of data being combined. Clinical predictions performed relatively less well when predictors included clinical interview data. These data indicate that mechanical predictions of human behaviors are equal or superior to clinical prediction methods for a wide range of circumstances.