The average is the most commonly used metric we use to understand data and infer insights - it is also the most dangerous.
![beware_average](//images.ctfassets.net/vwq10xzbe6iz/12kJw6jBqfvjjSCHyMxUNm/2dbf7edf379236641095dffe09c670fb/beware_average.png)

# The Pitfall of the Average
The US Air Force began life in the late 1940s with a high rate of aircraft accidents and pilot fatalities (excluding combat losses) with 23.6 aircraft destroyed per 100,000 flying hours. This dramatically reduced to 4.3 by the end of the 1960s driven largely by the work of one man, Lieutenant Gilbert S. Daniels who saw the pitfall of using “the average” to design solutions.

> The tendency to think in terms of the “average man” is a pitfall…it is virtually impossible to find an “average man” in the Air Force population.
[Gilbert S Daniels (1952 US Air Force Report)](https://apps.dtic.mil/sti/citations/AD0010203)

US Army engineers had standardised the design of pilot cockpits to conform to the average dimensions of a 1926 pilot. This was revisited in the 1950s by Lt. Daniels who measured the bodily dimensions of more than 4,000 pilots and found that zero pilots matched the average dimensions.

The US Air Force responded and demanded that its manufacturers design its cockpits to be adjustable to individual dimensions vs the standardised average. As a consequence, both pilot performance and accident/ mishap rates dramatically improved.
![us_airforce](//images.ctfassets.net/vwq10xzbe6iz/6mtEbxaIXNU60Svdu9uGsO/3c61d041d241f4040e7269db2552b8eb/us_airforce.png)

# A Modern Example
A modern example of the pitfall of the average would be to look at user activity. Imagine we build software that helps users perform inspections and we are tasked with improving user engagement.

Looking at the data we find that the average count of monthly inspections performed by our active users is 32 inspections per month:

![table_inspections](//images.ctfassets.net/vwq10xzbe6iz/3SfYJ8CwfGsBPFptTVL3jB/44bd12fd8ec35a7fc0c8efe608a47ed3/table_inspections.png)

Knowing this, we can deduce that our users on average perform 32 inspections per month or approximately one inspection a day. We begin building solutions such as daily notifications to help remind our users to perform their daily inspection.

__Stop.__

Reducing a large data set into a single number can often mask useful information and give us the wrong answer. Instead we can look at the distribution of our data and graph the same data as a histogram to reveal how often each different value occurs in a dataset.
![histogram_inspections](//images.ctfassets.net/vwq10xzbe6iz/1UixJwtgp0xcrWIvV6I6J2/ce429dc02a478f3c49abb41f28b113bc/histogram_inspections.png)

Now looking at the distribution of our data we can see that none of our users perform the average number of 32 inspections per month. Instead we see that 60% of our users perform 10 or fewer inspections per month and a large group (+30% of users) perform 70 or more inspections per month.

The distribution of our data could suggest that the usage of our software differs widely by different cohorts of users. One cohort could be using it to perform monthly scheduled safety checks vs another cohort using it to perform factory line QA checks. 

Next time when using the average, pause before making any conclusions and ask: "How many instances of the data actually reflect the average?". 

__References:__
- [THE “AVERAGE MAN”? - Gilbert S Daniels (1952 US Air Force Report)](https://apps.dtic.mil/sti/citations/AD0010203)
- [Trends in U.S. Air Force Aircraft Mishap Rates (1950-2018) - Thomas Light, Thomas Hamilton, Spencer Pfeifer](https://www.rand.org/content/dam/rand/pubs/research_reports/RRA200/RRA257-1/RAND_RRA257-1.pdf) 
- [The End of Average - Todd Rose (2015)](https://www.penguin.com.au/books/the-end-of-average-9780141980034) 


The average is a myth. Understand the pitfalls of the average and how it can conceal useful information. How the US Air Force threw out the average and saved lives. Use distributions to better understand your data.

How the US Air Force Ditched the "Average" and Saved Lives

Cognitive biases can negatively impact your ability to think critically and make informed decisions. Once you are aware of them, it will help you identify and navigate them.

Here is an explanation of the 4 main types of cognitive bias, some examples and how you can navigate them:
1. [Survivorship Bias](#survivorship)
2. [Confirmation Bias](#confirmation)
3. [Framing Bias](#framing)
4. [Groupthink](#groupthink)

## <a name="survivorship"></a>1. Survivorship Bias
> Focus on the traits of the successful whilst ignoring the failures.

![survivorship-bias](//images.ctfassets.net/vwq10xzbe6iz/t9fd2snGayd6dZM2Dn9Js/5e615d3df42fa017582a58b9f06a0e57/survivorship.JPG)
This involves limiting our analysis to instances that have passed a success hurdle and not analysing those that didn’t pass the hurdle. This commonly occurs when we adopt key learnings from successful case studies yet remain blind to the unsuccessful instances who may have also followed the same learnings. 

### Examples
The famous [WWII case study](https://www.researchgate.net/publication/254286514_Abraham_Wald's_Work_on_Aircraft_Survivability) involves mathematician Abraham Wald who was tasked with identifying where to place more armour on fighter planes. The officers data of bullet holes in returning planes supported placing more amour on the fuselage vs the engine. However, Wald’s insight was to ask where the bullet holes weren’t - i.e. the engine. The surviving planes had fewer hits to the engine because those that were hit didn’t return.

A modern example would be to look at a customer acquisition funnel with a 10% activation rate. Common analysis would be to take key learnings from the behaviour and personas of this group to further lift activation. However, the greater insight would be to analyse the other 90% and ask why aren’t they activating? The latter is more challenging and often involves less access to data, but the payoff is greater.

### How to Navigate?
Ask questions about what other data and groups are missing. Identify the success hurdle and ask why others have not passed it.

## <a name="confirmation"></a>2. Confirmation Bias
> Focus on data to promote our existing beliefs.

![confirmation-bias](//images.ctfassets.net/vwq10xzbe6iz/3Fwt3vluPTrXkA41Yfj6Dx/585fec98fc5647b58a14f4d568b973c7/confirmation.JPG)
This involves seeking out information that validates our existing views and prior commitments. We also discount other information that may invalidate our views. Confirming evidence makes us feel good and people do what makes them feel good. The counter approach is the scientific method, which attempts to ask questions first before gathering evidence to either support or reject a hypothesis. The former is easier and faster, the latter is harder and time consuming.

### Examples
The CEO has a new idea to launch a new product and instructs a team to conduct market research and competitor analysis on the feasibility of this product. The market research team is limited in scope and risks being urged to validate the success of the CEO’s new product idea. A better task for the team would be to investigate what the best product would be to solve a particular problem.

### How to Navigate? 
Try and identify if there is a pre-existing belief (yours or otherwise). Reflect on information you have rejected. Ask others with a different viewpoint.

## <a name="framing"></a>3. Framing Bias
> Solving a problem based on the context it is presented.

![framing-bias](//images.ctfassets.net/vwq10xzbe6iz/1RJWOsPkQawJKKxYk8oIQM/6ee3c51d2e488ae5ecd40e1abd233596/framing.JPG)
The specific context a problem is framed/ presented can influence the options and choices available.

### Examples
The problem of low company profitability can be either framed as a need to cut costs or to increase revenue. The way the problem is framed will influence the options available (i.e. fire and restructure or enter new markets).

### How to Navigate? 
Ask what is the ultimate problem we are trying to solve. For example, ask what the outcome would be if we solved this problem. See if there are other solutions to achieving the same outcome.

## <a name="groupthink"></a>4. Groupthink

> The desire for group harmony leads to poor decision making.

![groupthink](//images.ctfassets.net/vwq10xzbe6iz/3BvESluOEJuRmYVODq44U1/12de08589e23649ad287ff3abba0912b/groupthink.JPG)
People tend to avoid conflict, particularly in the presence of a majority and people of authority. People assume others have made informed decisions and fear the risk of looking un-knowledgable in front of others. 

### Examples
The term groupthink arose from the [case study](https://www.bassberry.com/wp-content/uploads/Effective_SEC_Rulemaking_Letters2.pdf) of the failed US Cuba Bay of Pigs invasion. In 1961, the CIA presented to President Kennedy a Cuba invasion plan formulated under the previous administration. Kennedy accepted it and early objections were dismissed and eventually no objections or alternatives arose. Kennedy learnt from this and a year later during the Cuban Missile Crisis intentionally absented himself from initial meetings to encourage free discussion. Two teams were created to evaluate the options before presenting to Kennedy. 

### How to navigate? 
Be clear on the objective and values. Encourage other views to solve the objective.


Cognitive biases can negatively impact your ability to think critically and make informed decisions. Here is an explanation of the 4 main types of cognitive bias, some examples and how you can navigate them.

Howie Mann
Startup & Investing Advice
About Articles Stocks

Articles > data (2)

How the US Air Force Ditched the "Average" and Saved Lives

4 Types of Bias Affecting Your Decision Making

Howie MannStartup & Investing AdviceAboutArticlesStocks

Articles > data (2)

How the US Air Force Ditched the "Average" and Saved Lives

4 Types of Bias Affecting Your Decision Making

Howie Mann
Startup & Investing Advice
About Articles Stocks