Cherry Picking: When People Ignore Evidence that They Dislike

Cherry Picking

 

Cherry picking is a logical fallacy which occurs when someone focuses only on evidence which supports their stance, while ignoring evidence which contradicts it. For example, a person who engages in cherry picking might mention only a small number of studies out of all the studies which were published on a certain topic, in an attempt to make it look as if the scientific consensus matches their stance.

Cherry picking is important to understand, since it’s frequently used in many domains, and can affect everything from how people present misleading rhetoric to how they conduct their internal reasoning process. As such, in the following article you will learn more about cherry picking, see examples of how people use it, and understand what you can do in order to respond to its use by others, as well as what you can do to avoid using it yourself.

 

Examples of cherry picking

Cherry picking in the media

Cherry picking is often used by the media, particularly in the case of less reputable media bodies, when they present only one side of a story, or give it disproportional coverage while ignoring facts that could support alternative viewpoints.

For example, a notable study on climate change examined thousands of scientific papers on the topic, and found that approximately 97% of them support the consensus position that humans are causing climate change. However, when reporting on the topic, a reporter who engages in cherry-picking might say the following:

Reporter: A recent study found that there are plenty of scientists who believe that humans aren’t causing climate change.

This statement represents a classic example of cherry picking, because it only mentions the fact that a recent study found that some scientists disagree with the idea of human-caused climate change, while ignoring the fact that the study in question also found that the vast majority of scientists do believe in human-caused climate change.

Accordingly, cherry picking is a rhetoric technique that is frequently used in the media by denialists, in an attempt to mislead people and discredit concepts that they disagree with, even when there is an overwhelming scientific consensus which proves that they’re wrong.

 

Cherry picking in politics

Cherry picking is also frequently used in political discourse. For example, consider the following statement:

Politician: our new educational program was highly successful, since it improved students’ math scores!

This would be considered cherry picking in a situation where the new program also led to a decrease in students’ scores in various other domains (such as English and science), or in a situation where the improvement was only evident in a small proportion of the schools where it was implemented (e.g. only in 1 out of 30).

Another common example of cherry picking in modern political discourse appears in situations where politicians cherry-pick information regarding the success or failure of policies which are used in other countries, when attempting to argue in favor or against the implementation of those policies in their own country.

 

Cherry picking in research

Cherry picking is an issue not only in terms of how people conduct discourse, but also in terms of how they think. As such, cherry picking is a prevalent phenomenon in some communities, such as the scientific community, where it affects the way people conduct research.

For example, cherry picking can be a part of the problematic HARKing process (hypothesizing after the results are known), in cases where people search through data in order to find the measures, analyses, samples, or interpretations that offer the strongest possible support for their initial hypothesis, despite the fact that doing so affects the validity of their research. Evidence of this issue has been found in many cases, and research has shown, for instance, that cherry picking can significantly influence the results of systematic reviews of randomized clinical trials.

The idea of ignoring conflicting evidence which contradicts a researcher’s main hypothesis been referred to in the scientific community as Occam’s broom, which is “the principle whereby inconvenient facts are swept under the carpet in the interests of a clear interpretation of a messy reality”. This represents a misapplication of the well-known Occam’s razor, which is a principle that suggests that all things being equal, you should prefer the hypothesis that requires the fewest assumptions.

 

The problem with cherry picking

The problem with cherry picking is that it involves analyzing and presenting existing information in a misleading way, that fails to take all the available information into account.

For example, cherry picking might cause someone to conduct an improper analysis of the scientific literature on a certain topic, if they take into account only the few studies which support their preexisting stance, while ignoring all the studies which contradict it. Similarly, cherry picking might cause someone to paint a misleading picture of the outcomes of a scientific study, if they mention only one of the possible interpretations for those outcomes, while ignoring all the others.

In this regard, the use of cherry picking violates the principle of total evidence (sometimes also referred to as Bernoulli’s maxim), which denotes that, when assessing the probability that a certain hypothesis is true, we must take into account all the available information.

 

Why people cherry-pick information

People engage in cherry picking both intentionally, when it comes to their discussions with others, as well as unintentionally, when it comes to their internal reasoning process.

Intentional use of cherry picking is driven by the rhetorical value of this technique. Essentially, this means people that use intentionally cherry picking in their arguments because doing so makes their arguments more persuasive, and helps them support their stance.

Note that engaging in cherry picking for this reason carries the risk of backlash if people discover the omission of evidence. However, the rhetorical value of this technique often outweighs this potential risk, especially in cases where there is a low likelihood that people will discover the cherry picking or care about it.

Unintentional use of cherry picking is driven by the flawed manner in which humans process information and make decisions. Accordingly, unintentional cherry picking can occur due to different causes in different situations, and people can engage in unintentional cherry picking even when they are aware of this concept and understand the issues which are associated with it.

One of the main reasons why people engage in unintentional cherry picking is the confirmation bias, which is a cognitive bias that causes people to process information in a way which confirms their preexisting beliefs. Essentially, this means that since people want to feel that they’re right, when they encounter new information or remember old information they tend to focus on information which confirms their beliefs, and ignore information which contradicts them.

Furthermore, another notable reason why people unintentionally cherry-pick information is that doing so is generally easier, from a cognitive perspective, than processing all the available information and therefore also more appealing. Simply put, when people are faced with a large body of evidence, it’s often easier to simply pick a few bits of information that stick out of that overwhelming mass and then base your stance on them, than it is to carefully analyze all the available information.

 

The use of cherry picking together with other fallacies

People often use cherry picking in conjunction with other logical fallacies.

For example, people sometimes use cherry picking together with strawman arguments, which are arguments that misrepresent an opposing stance in order to make it easier to attack. A common way to create a strawman argument is to cherry-pick a few specific quotes out of everything that someone said, and then repeat them out of context, in order to misrepresent what that person actually intended to say, a technique which is sometimes referred to as quote mining.

Similarly, cherry-picking is also sometimes used as part of arguments which rely on anecdotal evidence in a fallacious manner. This happens in cases where people base their arguments on a few anecdotal examples which supports their stance, while ignoring a more comprehensive body of evidence which contradicts that stance.

Note: the cherry-picking fallacy is often referred to using other names, such as the fallacy of incomplete evidence, the fallacy of exclusion, argument by selective observation, argument by half-truthcard stacking, and suppressing evidence.

 

How to respond to cherry picking

There are two main ways to respond to the use of cherry picking:

  • Call out the fallacious reasoning. Specifically, point out the fact that your opponent is ignoring crucial information which should be taken into account, and explain why this is a problem.
  • Bring omitted information into consideration. Specifically, discuss the information which was omitted, and show how taking it into account changes the situation at hand.

When doing this, it’s important to remember that people’s cherry picking might be unintentional. As such, as long as it’s reasonable to do so, you should implement the principle of charity, and assume that the person who engaged in cherry-picking did so unintentionally.

This means that you should generally attempt to ask the other person to justify their choice to omit crucial information, rather than attacking them for it outright. Doing this can help you identify situations where your opponent neglected to mention important information not because they chose to ignore it, but because they were unaware of it, and it can make the other person more willing to listen to what you have to say, and therefore more likely to internalize the information that you’re presenting them with.

Furthermore, doing this can also be beneficial in cases where people engage in cherry-picking intentionally. This is because doing this can often put your opponent in a defensive position, where they need to justify why they omitted critical information from the discussion, and because this approach can show to the audience, if there is one, that you’re committed to conducting a proper discussion, unlike your opponent.

 

How to avoid cherry picking

As we saw above, people can engage in cherry picking unintentionally, when they are driven to do so by a motivator such as the desire to confirm their preexisting beliefs. This is important to remember, because it means that you might also engage in cherry picking, even in situations where you don’t mean to do so.

To reduce the likelihood that this will happen to you, the most important question you need to ask yourself is: “is there any additional evidence or possible interpretations of existing evidence that I should be using in my analysis?”. If the answer to that question is “yes”, then try to modify your analysis by taking the additional information into account.

To improve your ability to do this, you can use techniques which are meant to reduce the likelihood of experiencing the confirmation bias. An example of such a technique is to avoid forming a hypothesis too early on, before you’ve had a chance to look at all the available information.

In addition, you can also use general debiasing techniques, such as slowing down your reasoning process, which will further reduce the likelihood that you will end up cherry-picking information.

 

Summary and conclusions

  • Cherry picking is a logical fallacy which occurs when someone focuses only on evidence which supports their stance, while ignoring evidence which contradicts it.
  • For example, a person who engages in cherry picking might mention only a small number of studies out of all the studies which were published on a certain topic, in an attempt to make it look as if the scientific consensus matches their stance.
  • People engage in cherry picking various situations, both intentionally, when it comes to their discussions with others, as well as unintentionally, when it comes to their internal reasoning process.
  • When responding to someone’s use of cherry picking in a discussion, you should point out the information that they’re ignoring, explain why ignoring it is an issue, and make sure that that information is taken into account during the discussion.
  • To avoid cherry picking information, you should make sure to check that you’re taking into account all the relevant information which is available to you; you can improve your ability do this by using various debiasing techniques, such as slowing down your reasoning process and avoiding forming a hypothesis too early on.