Summary & Key Takeaways
Key Takeaways
Almost all outcomes in life are due to a combination of luck and skill. Outcomes that have lots of variability, involve more luck than skill. Outcomes that have very little, if any, variability, involve more skill than luck.
The best test for finding out how much luck is involved in a given activity is answering the question: “Can you lose on purpose?” If you can, then it’s mostly skill, if you can’t luck plays a larger role.
The paradox of skill is that as participants in a field increase their skills, luck has more of an effect on the final outcome because participants converge on skill (less of a difference between the best and the worst, decreasing variability). Sports are a good example of this phenomenon.
The best thing to focus on when dealing in fields where luck as a large presence in outcomes is to focus on the process, not the outcome.
If you are in a competition with others, know that if you are the most skilled, you want to simplify things so that your skills come through. On the other hand, if you are the least skilled, the more randomness and complexity you can introduce, the better off you’ll be.
Notes
Always consider the base rate
“The paper argues that intuitive judgments are often unreliable because people base predictions on how well an event seems to fit a story. They fail to consider either how reliable the story is or what happened before in similar situations. More formally, Kahneman and Tversky argue that three types of information are relevant to statistical prediction. The first is prior information, or the base rate.”
Always consider luck
“Thinking explicitly about how luck influences our lives can help offset that cognitive bias.”
Know what you can know AND know what you cannot know
“Knowing what you can know and knowing what you can’t know are both essential ingredients of deciding well. Not everything that matters can be measured, and not everything that can be measured matters.”
Process > outcome
“When luck has little influence, a good process will always have a good outcome. When a measure of luck is involved, a good process will have a good outcome but only over time. When skill exerts the greater influence, cause and effect are intimately connected. When luck exerts the greater influence, cause and effect are only loosely linked in the short run.”
“In investing, as in other activities that are heavily probabilistic, process trumps outcomes.”
Test for measurement of luck involved
“There’s a quick and easy way to test whether an activity involves skill: ask whether you can lose on purpose.”
We draw lessons from the past that are wrong
“While we are capable of contemplating a future pulsating with possibility, we quickly forget that our experience was one of many that could have been. As a consequence, often we draw lessons from the past that are wrong.
Humans love stories
“The need to connect cause and effect is deeply ingrained in the human mind.”
“We re-create events in the world by creating a narrative that is based on our own beliefs and goals. As a consequence, we often struggle to understand cause and effect.”
Skills are not that transferable
“His research shows that organizations tend to overestimate the degree to which a star’s skills are transferable.”
The environment we work in matters on performance
“The organization or environment in which a CEO works also has an influence. The evidence shows that employers systematically overestimate the power of an individual’s skill and underestimate the influence of the organization in which he or she operates.”
Small samples are dangerous in luck influenced areas
“In activities that are strongly influenced by luck, a small sample is useless and even dangerous. You’ll need a large sample to draw any reasonable conclusion about what’s going to happen next.”
The paradox of skill
“As skill improves, performance becomes more consistent, and therefore luck becomes more important.”
“When everyone in business, sports, and investing copies the best practices of others, luck plays a greater role in how well they do.”
Note: the more skillful participants are in a field, the more luck matters
Ingredients of an outlier
“Great success combines skill with a lot of luck. You can’t get there by relying on either skill or luck alone. You need both.”
Rate of reversion to the mean
“Understanding the rate of reversion to the mean is essential for good forecasting.”
Time is just as important as the size of your sample
In skill influenced events chances matter
“The more chances they have to score, the more influence skill has.” (Talking about the NBA, a sport where skill plays a larger role)
“A slight advantage in skill gives you a tremendous overall advantage if you have enough chances to exercise that skill and to thereby nullify the effects of luck.”
Note: on the opposite side, this is why sports that have very little scoring chances (like hockey and soccer) have outcomes that are more influenced by luck
Paradox of skill in sports
“If the population of players to choose from expands and the size of the league remains constant, the average level of skill will tend to increase.”
Note: can apply beyond sports as well
Natural tendency to think fast, rather than slow
“The natural tendency when solving problems is to rely on cognitive mechanisms that are fast, low in computational power, and require little concentration, rather than recruiting those mechanisms of the mind that are slow, computationally intensive, and that require effort.”
Explore vs exploit
“Finding the best balance between exploration and exploitation depends on the rate of change in the environment. When change comes slowly, the balance can tilt toward exploitation. When it comes quickly, an organization must dedicate more resources to exploration, since profits are quickly exhausted.”
What generates power laws
“Whenever people can judge the quality of an item by several different criteria and are allowed to influence one another’s choices, luck will play a huge role in determining success or failure.”
Path dependent events are very sensitive to initial conditions
“A path-dependent process is one in which what happens next depends on what happened before. It’s a process that, in effect, has memory. These processes are very sensitive to initial conditions and lead to phenomena such as the rich getting richer and the poor getting poorer.”
“In these kinds of systems, initial conditions matter. And as time goes on, they matter more and more.” (Talking about academia, and how getting a good initial job can heavily influence the rest of the career of an academic)
The tipping point
“A phase transition occurs when a small incremental change leads to a large-scale effect. This is known colloquially as a tipping point.”
Increased pay for top performers
“Increased competition for talent is another factor that creates outsized pay for top performers.”
Quality and value are not necessarily correlated
“The relationship between quality and value is not at all clear.”
Qualities of useful statistics
Persistent: what happens in the present is similar to what happened in the past
Predictive: they can be used to accurately predict the goal you seek (validity)
Measure these by examining the coefficient of correlation
“You have to translate a theory of cause and effect into quantities that you can observe and measure.”
“The simple test of persistence and predictive value goes a long way in judging how practical a measure is likely to be.”
How to measure which statistics are due to luck and which to skill
“If a statistic accurately measures a player’s skill, you would expect the values to be similar from one season to the next. On the other hand, if the statistic varies a great deal from year to year, you can assume that luck plays a large role in that outcome.” (Talking about looking at different statistics in baseball players like batting average vs on-base percentage vs strikeout rates)
Note: skill statistics have much lower volatility rates
“In general, the more a statistic relies on the interaction of teammates, the lower the coefficient of correlation.”
Using nonfinancial measurements
“You can use nonfinancial measures to improve your performance.”
“Carelessly chosen nonfinancial measures are even worse.” (Than poor financial measures)
Portfolio measurement: active share
“Active share measures the fraction of a portfolio that is different from the benchmark index.”
Becoming an expert
“You can become an expert if cause and effect are clear and consistent in what you do and if you practice intensely and are guided by accurate feedback.”
“For activities that take place in environments that are stable and in which luck plays a small role, deliberate practice improves skill.”
Value of checklists
“In most cases, checklists don’t contain anything that you don’t already know. They just ensure that you actually perform all the tasks you’re supposed to perform...whenever there are distractions, checklists help direct and manage your attention.”
Note: especially work well in high pressure tasks
“Checklists have never been shown to hurt performance in any field, and they have helped results in a great many instances.”
Improving performance
“The combination of structure and effort are the key to advancing beyond plateaus in performance.”
“Your performance improves only if you receive accurate and timely feedback.”
A skillful process
“A skillful process will tend to have three parts: analysis, psychology, and the influences exerted by your organization.”
On being contrarian
“It’s also hard to distinguish yourself if you’re doing the same thing as everyone else.”
“If you have formed a conclusion from the facts and if you know your judgement is sound, act on it—even though others may hesitate or differ.”
“The goal is to be a contrarian when it allows you to gain an edge, and the calculator helps you ensure a margin of safety.”
Strategies when dealing with luck and skill
“If you are the favorite, simplify the game. If you are the underdog, make it more complicated.”
The need to overcome organizational traditions
“Part of the reason combatants don’t switch is that when training and equipment are developed for one strategy, it’s often costly to shift to another. Leaders or organizational traditions also stand in the way of adopting new strategies. This type of inertia often prevents an organization from pursuing the strategy that offers the best chance of winning.”
Note: similar to why big companies fail
Measure and react
“Watts suggests that instead, we ‘measure and react’; that is, we do carefully controlled experiments and let our actions be guided by the results.”
How to deal with high uncertainty and change
“Taleb recommends that we avoid optimization and allow for redundancy.”
“Optimization in a changing system is a setup for failure. If you are an animal that evolved in a cold climate, you might die if the climate becomes too warm.”
Note: this is the issue in machine learning and computer science called overfitting. A good discussion on this topic exists in Ch 7 of Algorithms to Live By
“Taleb’s message is that if you are dealing with events in the fourth quadrant, you should seek payoffs that are equivalent to buying options, even though they have a small but steady cost and you don’t know if or when you will get a big payoff.”
“In other words, know what you don’t know. The second step is to take steps to make sure that whatever exposure you do have in the fourth quadrant is the result of buying, or acquiring, options and not the result of selling options.”
“The key is to sidestep activities that have small gains and large losses and to gain exposure to payoffs that have small costs but large gains.”
Understanding the limits of models is as important as using them well
“Understanding the limitations of models is as important as applying them properly.”
Reversion to the mean
“The more general principle is that whenever there is an imperfect correlation between two scores, in this case from one test to the next, you will have reversion to the mean.”
Understanding reversion to the mean
“But just because you observe reversion to the mean, that doesn’t suggest that outcomes are converging toward the average. You must be careful to distinguish between a change of the system and change within the system.”
“The coefficient of correlation between two variables determines the rate of reversion to the mean and provides valuable guidance for forecasting.”
“A high correlation suggests weak reversion to the mean. A low correlation suggests strong reversion to the mean.”
Three types of information for a statistical prediction
The prior information, or base rate
The specific evidence about the individual case
The expected accuracy of the prediction
The trick is determining how to weight the information.
Always consider a null hypothesis
“You should always compare outcomes to what a simple model would generate.”
Measuring skill
“The difference between actual outcomes and outcomes defined only by luck provide a concrete way to quantify the contribution of skill.”
Avoiding hindsight bias
“One way to avoid hindsight bias is to engage in counterfactual thinking, a careful consideration of what could have happened but didn’t. If we accept that x played a role in causing y, then we have to consider how events would have unfolded had x not happened.”
“To maintain an open mind about the future, it is very useful to keep an open mind about the past.”
Improving skill
“We can hone our intuitions if we practice intently in environments that remain stable and linear.”
“Measurement provides the basis for the feedback that allows for the continual improvement of skill.”
Decision journal
“Whenever you make a decision, write down what you decided, how you came to that decision, and what you expect to happen. Then, when the results of that decision are clear, write them down and compare them with what you thought would happen.”
Good processes
“Good processes must be well grounded analytically, psychologically, and organizationally. The analytical part requires finding a discrepancy between value and price that gives you an edge…the psychological requirement means learning about the common biases that plague all of us and developing techniques to mitigate them...the organizational requirement is often the result of a conflict between a principal and the agent acting on the principal’s behalf.”
Importance of measuring carefully
“If we measure the wrong things, we will not achieve our goals.”