How Normal Distribution Shapes Stock Picks: Separating Skill from Luck

In medical school I read a fantastic book called The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow, a Caltech professor. Recently, when cleaning out our basement, I found it again and started re-reading it. The whole premise of the book is laying out how we can work to understand our world better through a better understanding of chance, probability, and statistics. Not surprisingly, there are a lot of parallels here to investing and especially the “art” of stock picks.

So let's go down the rabbit hole…

The biggest reason we make mistakes

In everyday life, we face countless decisions. And most of those decisions we make based on a very limited data set of prior events, perceived skill, and hunches. Because that is often all we have to work with.

For instance, even as doctors, we do this on a daily basis. A new cancer treatment has recently been approved and its core study demonstrated benefit for a certain group of patients. We usually set statistical significance set at p=0.05 in medicine. But even then, we still accept an at least 5% chance that the findings of the study were due to random chance and we are giving our patient a treatment with all risk and no benefit. Even in this example, we assume that the study was correctly powered. And the statistical assumption is that all patients included were “similar.” However, we know that no two patients are really the same. And yet we have to make the best decision we can.

The problem, however, is that most of life's decisions are not this well laid out. And even our evidence based decisions in medicine are still flawed. Again, within the medical world, look at the process of matching residents. We look at a limited data set of their accomplishments, test scores, and letters of recommendation. These are flimsy and limited data points upon which we have to choose how we rank and match them.

The takeaway here is that, as humans looking to explain the world around us, we too often find cause and correlation where events are better explained by randomness and chance.

The bell curve and chance

We are all familiar with the standard bell curve or Gaussian curve. It seems very basic.

The bell curve came about as various scientists observed that for really any measure, data clustered around a central value after which values above and below this central value tapered off, very sharply at first before leveling out. As the number of data increased, this formed a smoother and smoother curve. This curve could then be approximated by a mathematical equation. For any set of data that follow a normal distribution, for which the results all have an equal probability of outcome (like say, rolling a die), 68% of observations will fall within 1 standard deviation from the central value or average, 95% within two standard deviations, and 99.7% within 3.

But when you look at it in historical context, this bell curve was an amazing discovery. This curve allowed humans, for the first time, to take a limited set of observations and assess the probability that our determination of the “true value” of our observations is correct.

Again, this is what we are asked to do in everyday life, over and over again. But still, even with the bell curve, we do such a poor job at it. Most often due to a lack of awareness. The “fast” thinking part of our brain takes over and is quick to attribute cause to any identified patterns. Instead, we need to allow the “slow” thinking part of our brain really think about what we are looking at and how much could be explained by random chance.

Where does investing and stock picks come in?

The holy grail of investing in the stock market is the ability of a stock manager to pick particular individual stocks that will do better than others and to time when those stocks will rise and fall in value to buy and sell them appropriately. If someone possesses this skill, they will be able to “outperform” the overall stock market. And for the sake of this argument, the returns of the overall market can be likened to the central value of this data – the returns most managers and investor will realize.

Needless to say, such a skill would make a whole lot of money for the stock manager and their investors.

But is such a skill at stock picks possible?

Let's look at an example using normal distribution

First, forget about the stock market.

Let's say we ask 300 college students to guess the results of flipping a coin 10 times. And we track how well they did. Obviously, flipping a coin has a 50/50 chance of landing heads or tails. Now, let's plot their results on a graph. Imagine a graph where the x-axis is the number of correct guesses and the y-axis is the number of student who guessed that many correctly.

Not surprisingly, we find that the curve is bell-shaped. The central value is 5 correct guesses (which makes sense). The curve then falls to 2/3 of its maximum height at around 3.5 correct guesses on the left and 6.5 correct guesses on the right. In all, this represents a bell curve typical of a random process measuring events of equal likelihood, such as a coin toss.

Now, back to the stock market

Look at the graph from the book below.

The line with the squares represents the coin toss experiment and its bell curve confirming a random process.

The line with the circles represents another experiment. This line represents the performance of 300 mutual fund managers, with the x-axis representing the number of years out of 10 that a manger outperforms the group average (i.e. the performance of the overall stock market).

So, what does this mean?

It means that the performance of mutual fund managers and of actively investing in the stock market at all looks a whole lot like a random process.

If some fund managers were just better than others, we would not see a random distribution like this. Instead, we would see some consistent peaks at the far right and left of the graph. These peaks would represent managers doing better and doing worse, based on their skill. But we don't see this at all.

So, what conclusion can we draw?

Well, we certainly would not take a college student who happened to correctly guess all 10 coin flips and bet out life savings on him or her guessing the next one correctly, right?

But many investors often hand over their life savings to a fund manager who happens to have a few good years of outperformance in a row.

Given what we now know, that this is a random process, why would we do that?

Because it's only human.

Investing in the stock market seems way more complicated than calling a flipped coin. So we perceive skill can be present where it isn't. Plus, it takes (minimal) work to invest your own money. So we offload it and outsource it. Plus, the (unlikely) chance of our fund manager making the correct stock picks year and year to the benefit of our investment accounts is exciting! Even if the odds of them performing consistently 3 standard deviations outside of the central value are just 0.3%.

Combine these factors with our predisposition to place significance where there is none in our vain efforts to explain the randomness of the world around us and we can start to understand why so many investors fall victim to this fallacy.

The alternative to random stock picks

We face a fork in the road: We can either:

Keep making stock picks by essentially basing our decision on a monkey throwing darts at a dartboard, or
Chart a different path forward

I think most of us will agree that the wise thing to do is to find a more consistent, reliable alternative.

But, if active stock picking and timing of the stock market doesn't work and not investing at all isn't a good option, then what is the alternative?

Well, it's simple. Instead of trying to pick individual stocks and time their value fluctuations, just buy all the stocks and invest for the long term. This way, you buy all the great stocks (at the expense of all the not so great ones) and buy all the rises in value (along with the dips). You do this because you can't predict ahead of time which stocks will be good or bad and when they will change in value one way or the other. But the overall trend of the market is upwards over time.

And this is exactly the strategy of passive investing. And passive investing has been proven to work better than active investing 80% of the time with lower fees and taxes. Our laws of measurements and bell curve help demonstrate and prove why this is the case.

Investing passively via index funds is how I invest my money (here is my portfolio's return last year). And how people a lot smarter than me invest theirs.

For actionable tips and tricks to invest your money the right way, check out these posts:

What do you think? Do you pick stocks? Or invest passively? Why? Does the bell curve sway you at all towards one or the other? Let me know in the comments below!