Rules in the Textbooks, Guidelines in the Trenches
A study three years ago figured out how to save more than 100,000 lives a year.
If you stop there, you think we’ve found a miracle. But dig deeper and you see why miracles are hard to achieve.
Good, smart strategies that are worth promoting can work for many people and backfire on others. Those “other” people have to abandon time-tested rules and figure out what works for them.
For decades, keeping your systolic blood pressure (the top number) under 140 was the accepted target for maintaining health. But the Systolic Blood Pressure Intervention Trial showed that 140 wasn’t low enough; 120 should be the goal.
The lower level led to fewer heart attacks, fewer strokes, fewer deaths. If all U.S. adults over age 50 with elevated blood pressure hit 120 or below, 107,500 lives would be saved every year.
The American Heart Association changed its guidelines based on the study. “Few other medical interventions are currently available that could have such a large and immediate public health impact,” one analysis wrote.
But there’s a reason they call these guidelines, not rules.
More than half of patients in the study could not push their systolic blood pressure below 120, even with “intensive treatment.” And those intensive treatments – aggressively pushing blood pressure down with medication – can have wild side effects.
One in every 62 patients in intensive treatment got an acute kidney injury, about double the rate of the control group. About one in 100 fainted, 50% higher than the control. Electrolyte abnormalities were a third higher than the control. The study tallied the rate of these side effects with a high bar – whether they caused the patient to go to the emergency room.
If intensive treatment gives you ER-worthy side effects while keeping you short of the target blood pressure, you and your doctor will view the study as a guideline, not a rule.
You followed good, evidence-based advice. But everyone is wired different, and this strategy didn’t work for you. It backfired. So you have to find another solution. Maybe fewer medications and more diet. Maybe stress management. One analysis said that, for many people, “it would be prudent to aim for a more conservative goal” than the target of 120.
Rules are fun because they exempt you from second guessing your decisions. But in the trenches, there are only guidelines.
The beauty of big data is the ability to spot trends that anecdotes never revealed. Crunch the numbers, read the data, and do what it tells you. That’s how you make better decisions.
But rules that come from data rely on a few things. The data has to capture every nuance of whatever you’re measuring. The analysis has to be unimpeachable. The interpretation has to be contextualized. And, most important, whatever trend discovered in past data has to continue in the future, without room for what you’re measuring to adapt.
Which isn’t always the case. Depending on what you’re measuring, it’s almost never the case.
Jeff Bezos recently explained why he listens to individual Amazon customers in addition to broad data:
The thing I have noticed is when the anecdotes and the data disagree, the anecdotes are usually right. There’s something wrong with the way you are measuring it.
I can imagine meetings at Amazon HQ go something like this:
Business analyst: Mr. Bezos, our data shows that delivering packages a day late doesn’t impact a customers’ willingness to renew their Prime membership.
Bezos: I don’t know about that. Jim from Chicago just emailed me saying he canceled because his package was late.
The hard here part is balancing evidence-based decisions with the flexibility to realize that data is fallible. That balance is more art than science.
But of course it is.
The reason economies grow is because people get more productive at doing stuff. And they get more productive because they stop doing old things, start doing new things, adapt, evolve, and grow new tastes. Investment opportunities exist because people do those things at unpredictable times and for unpredictable reasons. So even if your data and analysis is excellent, it will never perfectly answer what people will do next, because what they do next isn’t what the data measured them doing in the past. It takes someone like Bezos – who views data as guidelines, not rules – to navigate that kind of land.
Here again, the business data isn’t bad. Like the blood pressure study, its conclusions are fantastic and they should be promoted.
But every situation is different. There will be side effects that are more serious in some cases than others. New solutions will arise. There is a huge difference between data serving as a guideline of where to direct your attention vs. treating its conclusions as unshakable gospel.
This happens in investing.
Ben Graham is one of the greatest investors of the last century. Most of his success is owed to one investment that broke his own rules. And the specific investing rules laid out in his best-selling books were constantly thrown out and updated in later editions.
What do we make of this?
Maybe Graham didn’t have the discipline to stick with his rules, and got lucky when he veered from them. Or maybe he realized that rules work on average, but individual opportunities in the trenches are not averages. Every opportunity has its own nuance that isn’t cleanly measured, which is why opportunities exist to begin with.
Factor investing – an investing style that is now used in public, venture, and private equity investing – strips investing of subjective decision-making in a way I find fascinating. It’s a step forward toward turning the investing industry into a science.
But one of the most important questions in factor is investing is, “How do you know when a trend that worked in the past doesn’t work anymore?” It is a difficult question. There are ways to answer it analytically. But in the trenches it’s only answered by an investors’ and their clients’ willingness to endure underperformance without knowing whether it’s temporary or a new paradigm.
The irony is that factor investing is designed to remove the emotions of subjection decision-making. But its most important part – getting people to believe in a strategy – is as subjective and emotional as it gets. So in the trenches a good portfolio manager might use data as more of a guideline than a rule. Maybe a strategy uses less concentration than the data recommends. Maybe it’s multiple strategies to mitigate concentrated volatility. Or an emphasis on the importance of client communication, which is something data on its own doesn’t reveal.
The point is that medicine, business, and investing will always be part art part science, even if you can describe their success in purely scientific ways. Data that captures averages over a big set of examples won’t always apply to the day-to-day nuance and opportunities that practitioners deal with.
Bruce Lee had this figured out.
His theory of martial arts was to learn as much traditional style as he could, so he had the tools to improvise any situation he needed. He was never winging it. And he was never following the textbook. He used traditional style – call it data – as a guideline, with the understanding that unique situations require unique strategies. “The truth is outside of all fixed patterns,” he said. Good advice.