Fallacy Fallacy (Formal Fallacies Part 2)

Formal fallacies are arguments that are always wrong, regardless whether the argument's premises (statements claimed as fact) are true or false. In the previous installment, the appeal to probability, a claim that because something could happen, therefore it will happen is false even if it's true that the something in question could indeed happen.

Argument from Fallacy

If an argument contains a fallacy, what does that say about the conclusion? Actually, it doesn’t say very much. Excessively pointing to fallacies can itself trigger a fallacy of its own: the argument from fallacy, or the fallacy fallacy.

The argument from fallacy is the error of concluding that if an argument can be shown to be fallacious, that means its conclusion necessarily must be false. The form of the argument is:

If P, then Q
P is a fallacious argument.
Therefore, Q is false.

Take, for example, the following claim: “I speak English, therefore I am an American citizen.” That’s a fallacious argument, because many people who speak English are not American citizens. To conclude, however, that because the argument is fallacious, you must not be an American citizen, is taking the claim a step too far. A conclusion can be right even if the argument supporting it happens to be wrong.

If you can show that a particular argument is fallacious, the only thing that means is that the particular argument can’t be used to prove the proposition. The opposite argument, that the fallacious argument itself disproves the proposition, is also a fallacy.

The argument from fallacy is also known as the argument to logic (argumentum ad logicam) and the fallacist’s fallacy. It’s part of a group of fallacies known as fallacies of relevance.

Base Rate Fallacy
Conjunction Fallacy

The base rate fallacy and the conjunction fallacy also fall into the category of cognitive bias, and were both treated earlier in this blog and in my compilation of cognitive biases, published separately.

When 1+1=3 (Part 19 of Cognitive Biases)

Our 19th installment of Cognitive Biases covers the status quo bias, stereotyping, and the subadditivity effect.


Status Quo Bias

Sigmund Freud suggested that there were only two reasons people changed: pain and pressure. Evidence for the status quo bias, a preference not to change established behavior (even if negative) unless the incentive to change is overwhelming, comes from many fields, including political science and economics.

Another way to look at the status quo bias is inertia: the tendency of objects at rest to remain at rest until acted upon by an outside force. The corollary, that objects once in motion tend to stay in motion until acted upon by an outside force, gives hope for change. Unfortunately, one of those outside forces is friction, which is as easy to see in human affairs as it is in the rest of the material universe.

Daniel Kahneman (this time without Amos Tversky) has created experiments that can produce status quo bias effects reliably. It seems to be a combination of loss aversion and the endowment effect, both described elsewhere.

The status quo bias should be distinguished from a rational preference for the status quo in any particular incident. Change is not in itself always good.

Stereotyping

A stereotype, strictly speaking, is a commonly held popular belief about a specific social group or type of individual. It’s not identical to prejudice:

• Prejudices are abstract-general preconceptions or abstract-general attitudes towards any type of situation, object, or person.
• Stereotypes are generalizations of existing characteristics that reduce complexity.

The word stereotype originally comes from printing: a duplicate impression of an original typographic element used for printing instead of the original. (A cliché, interestingly, is the technical term for the printing surface of a stereotype.) It was journalist Walter Lippmann who first used the word in its modern interpersonal sense. A stereotype is a “picture in our heads,“ he wrote, “whether right or wrong.“

Mental categorizing and labeling is both necessary and inescapable. Automatic stereotyping is natural; the necessary (but often omitted) follow-up is to make a conscious check to adjust the impression.

A number of theories have been derived from sociological studies of stereotyping and prejudicial thinking. In early studies it was believed that stereotypes were only used by rigid, repressed, and authoritarian people. Sociologists concluded that this was a result of conflict, poor parenting, and inadequate mental and emotional development. This idea has been overturned; more recent studies have concluded that stereotypes are commonplace.

One theory as to why people stereotype is that it is too difficult to take in all of the complexities of other people as individuals. Even though stereotyping is inexact, it is an efficient way to mentally organize large blocks of information. Categorization is an essential human capability because it enables us to simplify, predict, and organize our world. Once one has sorted and organized everyone into tidy categories, there is a human tendency to avoid processing new or unexpected information about each individual. Assigning general group characteristics to members of that group saves time and satisfies the need to predict the social world in a general sense.

Another theory is that people stereotype because of the need to feel good about oneself. Stereotypes protect one from anxiety and enhance self-esteem. By designating one's own group as the standard or normal group and assigning others to groups considered inferior or abnormal, it provides one with a sense of worth, and in that sense, stereotyping is related to the ingroup bias.

Subadditivity Effect

The subadditivity effect is the tendency to judge probability of the whole to be less than the probabilities of the parts.

For instance, subjects in one experiment judged the probability of death from cancer in the United States was 18%, the probability from heart attack was 22%, and the probability of death from "other natural causes" was 33%. Other participants judged the probability of death from a natural cause was 58%. Natural causes are made up of precisely cancer, heart attack, and "other natural causes," however, the sum of the latter three probabilities is 73%. According to Tversky and Koehler in a 1994 study, this kind of result is observed consistently.

The subadditivity effect is related to other math-oriented cognitive biases, including the denomination effect, the base rate fallacy, and especially the conjunction fallacy.

More next week.

To read the whole series, click "Cognitive bias" in the tag cloud to your right, or search for any individual bias the same way.

Risk Management, Cognitive Bias, and the Global Warming Debate

The debate on global warming tends to revolve completely around the science. Is it good? Is it bad? Is it meaningful? Is it corrupt? Everyone has an opinion on the quality of the science, and once those opinions are formed, they’re almost impossible to shake.

A wide variety of potential cognitive biases complicate the picture. Notice there’s enough here for everybody — no one’s being singled out.

• Base rate fallacy — ignoring statistical data in favor of particulars

• Confirmation bias — interpreting information to confirm your preconceptions

• Experimenter’s bias — with about sixty subsets

• Focusing effect — putting too much emphasis on a single aspect of a situation or event

• Framing — viewing through a perspective or approach that is too narrow

• Hyperbolic discounting — the preference for more immediate payoffs over the long term

• Irrational escalation — making irrational decisions based on rational decisions in the past, or to justify actions already taken

• Information bias — seeking more information even when it cannot affect action or decision

…the list goes on. Recognize some of these biases? If you’re like most of us, you recognize them in the other side more than you see them in yourself or those who agree with you.

Part of the reason why cognitive bias is at work is that the question isn’t really clear. We’re all arguing about the science, though few of us are truly entitled to an educated opinion on the subject.

But what’s the question?

It’s not about whether a scientific opinion is correct or incorrect. That sort of thing only interests specialists. No, the question has to do with what (if anything) should we do about it, based on the potential cost and consequences.

In other words, it’s a question of risk management. And to the extent that it’s a question of risk management, it’s phrased wrong.

A risk, as you’ll remember, is a future event with some probability of happening that if it happens will have a meaningful impact on your situation. If the impact is negative, it’s a threat. If the impact is positive, it’s an opportunity.

Risks, like Gaul, can be divided into three parts. The first part is probability. How likely is it that the risk will happen?

The second part is impact. If the risk should happen, what would be its effects?

Those two parts combine in the formula R = P x I to calculate a risk score, the value of the risk.

We care about the value of the risk because that helps us make a rational decision about the third element: the cost of reducing or eliminating the negative risk, or the cost of obtaining or exploiting the positive risk.

Probability

The argument about the science of climate change is at root an argument about probability. The process of science involves collecting data, discovering patterns in that data, and developing and testing hypotheses and theories about that data. Over time, the process of peer review creates a consensus in the scientific community, and at any moment in time, that’s the state of scientific knowledge.

Let’s sidestep the discussion about whether the consensus of current scientific knowledge is accurate or inaccurate, and merely assess how our own feeling and opinions influence our judgment of probability. Taking as a guide the legal standards of proof, we might fall somewhere on the following spectrum. For a rough calculation, I’ve put in some percentages.

Degree of Belief (Probability You Think It's True)

True beyond any doubt (99+%)

True beyond a reasonable doubt (95%)

True by the preponderance of the evidence (75%)

Unable to tell (50%)

False by the preponderance of the evidence (25%)

False beyond reasonable doubt (5%)

False beyond any doubt (>1%)

This is about what you believe about the science, and a corresponding figure that relates to how likely it is that the threat is true. If you don't like the choices, add one of your own and choose your own probability number.

Notice that the evidence won't stand still. Over time, science will inevitably get better, regardless of your perspective. Either the evidence of catastrophic global climate change will mount so high no sane person can deny it, or global warming will become the Comet Kohoutek of crises, a non-event. Or maybe something in between.

The problem is by the time the facts become incontrovertible, the moment for decision will have passed. If we guess wrong, there are two possibilities: (a) we will be in a significantly worse position to deal with the resultant impact, or (b) we will have wasted significant resources.

Impact

This leads us to the second item, the question of impact. Impact is the effect of the threat or opportunity if it happens — even if you believe the chance is remote at best. So we have to set probability aside temporarily. We’ll come back to it in a moment.

In addition to arguments about how likely it is that the scientific consensus on global warming is in fact correct, there is a range of opinion as to what that means in practical terms: a range of impact. I've specified a set of potential impact levels and set costs for each. Remember, the issue isn't whether these are going to happen. They're simply descriptions of the potential level of impact that different parties suggest are possible.

So choose from the list below. What, in your opinion, is the worst possible potential outcome if global warming happens?

• Catastrophic. Global warming effects will kill tens or hundreds of millions of people directly and indirectly, wipe out tens of thousands of species, and be an economic and social catastrophe to those who survive. Repair or rebuilding may or may not be possible. (Cost = $Quadrillions)
• Serious. Major weather events, such as hurricanes and tsunamis will be more prevalent, tens and hundreds of thousands will die, economies will suffer. (Cost = $Trillions)
• Moderate. Managing environmental issues will be a consuming issue, but better management and improved technology will make this a background costs. (Cost = $Billions)
• Minor. Insignificant costs. (Cost < $Millions)

Notice the impact could also be positive.

Value of the Risk

Just because you aren't convinced the evidence in favor of a risk is certain doesn't mean you don't act on it. We take everyday precautions to avoid low probability or highly uncertain risks with potentially high impact all the time — every time we drive on a freeway, for example. But there's a limit. How does the value of the risk compare to the cost of mitigation?

The value of the risk, as we’ve noted, is the probability times the impact. From our earlier work, we can construct this table. The risk score in each case is what you should reasonably be willing to spend if necessary to mitigate the degree of risk you personally believe is present.

Catastrophic

95% confident, $Quadrillions

75% confident, $Low Quadrillions

50% confident, $1 Quadrillion

25% confident, $High trillions

5% confident, $Low trillions

Serious

95% confident, $Up to 1 Quadrillion

75% confident, $750 trillion

50% confident, $500 trillion

25% confident, $250 trillion

5% confident, $50 trillion

Moderate

95% confident, $Up to 1 Trillion

75% confident, $750 billion

50% confident, $500 billion

25% confident, $250 billion

5% confident, $50 billion

Minor

95% confident, $Possibly a few billion

75% confident, $Less than a billion

50% confident, $500 million

25% confident, $250 million

5% confident, $Low millions

Cost of Mitigation

The value of the risk is what you’re willing to spend if necessary. Depending on how you assessed probability and impact, you ended up with some amount of money (perhaps $0) that's appropriate as a maximum to spend on the risk.

Of course, you need to compare that to the cost of mitigating or eliminating the risk. Sometimes, it’s not worth it. If I offered to save you from a $1,000 risk in exchange for $2,000, it’s not much of a deal. In general, if the cost of getting rid of the risk exceeds the cost of living with it, you’re better off living with it.

On the other hand, if I can save you from a $1,000 risk (say, a 25% chance of losing $4,000) for only $500, that's a pretty good deal. If the risk happens, you've saved $3,500. But if the risk doesn't happen, you're still out $500.

It's true that not all costs of a risk (or costs of a risk mitigation) can be easily translated into dollar terms — or even should be. That doesn’t change the basic principle, though: the cost of dealing with the risk has to be less than the cost of living with the risk.

There’s an important qualification when it comes to risk mitigation. Some risks you can get rid of altogether if you’re willing to pay the price. Other risks you can reduce, but not eliminate. You can lower the probability of the event occurring, or you can lower the impact if it should occur.

That’s not a bad thing, mind you, but you have to take into account the residual risk when deciding if the strategy is worth it. The value of that risk is the difference between the cost of the original risk and the cost of the residual risk.

The Right Question

To have a reasoned discussion on the subject of global warming, you have to figure out where you are on five issues, not merely one.

1. How correct is the scientific consensus on global warming?

2. What is the impact of global warming if it should occur?

3. What is the value of the risk (probability times impact)?

4. What is the cost of mitigating or eliminating the risk, and how much residual risk would remain?

5. In balance, what level of action on global warming (if any) is warranted?

To change someone’s opinion, you have to change that person’s evaluation of at least one of these issues.

As people on all sides have found, it’s nearly impossible to change anyone’s evaluation of the quality of science, which is our probability benchmark. There’s often more consensus of what global warming might mean if it happens, which is why it’s so important to separate discussion of probability from the discussion of impact.

But the real opportunity has to do with the issue of cost. The best current framing of the debate comes from the argument that dealing with global warming and environmental issues can be relatively low in cost, or ideally profitable.

If the cost to deal with global warming is low enough, it's a good idea even for those who think the probability is low.

“Looking for the Pony” — Cognitive Biases, Part 2

Welcome back to part two of our discussion of cognitive and decision-making biases. The series begins here.

Everyone's subject to cognitive biases of one sort or another. None of us is capable of pure objectivity; we cannot see reality without distortion. But we can try.

There are around 100 different identified cognitive and decision biases, and some of them have subsets, as we'll see shortly. Today, we'll cover three more: the base rate fallacy, congruence bias, and everyone's traditional favorite, experimenter's bias.

Base rate fallacy. There are 100 terrorists trying to sneak through airline security for every one million non-terrorists. TSA has set up an automated face recognition system that has 99% accuracy. The alarm goes off, and trained Homeland Security agents swoop down. What is the probability their captive is really a terrorist?

Well, if the failure rate is 1%, that means there’s a 99% chance the person is a terrorist, and a 1% chance that he or she is not, right? That justifies a significant assumption of guilt.

But this actually gets it backward. The chance the person isn't a terrorist is far greater — in fact, it's 99.02% likely that the new prisoner is completely innocent!

The mistake that leads to the first conclusion is called the base rate fallacy. It occurs when you don't notice that the failure rate (1 in 100) is the not the same as the false alarm rate. The false alarm rate is completely different, because there are, after all, far more non-terrorists than terrorists. Let's imagine that we walk everyone — 100 terrorist and 1 million non-terrorists, for a total of 1,000,100 people — in front of the face recognition tool. A 1% failure rate means it's going to ring incorrectly one time for each 100 passengers, 10,099 times in total. It will catch 99 terrorists and miss one, but it's also going to catch 10,000 non-terrorists. The ratio is actually 99:10,099, or a miniscule 0.98%, that the person caught is actually a terrorist.

This does not argue against the value of screening. Screening might be perfectly reasonable. Overreaction, however, is not. If you’re 99% sure you’ve caught a terrorist, you will behave differently than if you’re only 1% sure.

To avoid the base rate fallacy, look at the “prior probability.” If there were no terrorists, what would the face recognition system produce? With a 1% failure rate, it would never pick a real terrorist (there would be none), but it would trigger 10,000 false positives. Now you’ve found the missing fact.

(Footnote: Notice that the base rate fallacy only produces incorrect analysis when the scale is unbalanced, as is our case with 100 terrorists in city with a population of 1 million. As the populations approach 50/50, the failure rate and false alarm rate would converge. Mind you, we'd have different problems then.)

Congruence bias. In congruence bias, you only test your hypothesis directly, potentially missing alternative explanations. In the famous Hawthorne experiment, Frederick W. Taylor, father of Scientific Management, wanted to test whether improved lighting in factories would increase worker productivity. He performed a direct test: he measured productivity, installed better lighting, and measured productivity again. Productivity went up. If you are falling into congruence bias, you’re done. Experiment confirmed; case closed.

But Taylor avoided the trap. He tested his hypothesis indirectly. If improved lighting increased productivity, he reasoned that worse lighting should lower it. So he tested that proposition as well. He took out a lot of lights and measured again: and to everyone’s surprise, productivity went up! A deeper analysis revealed what is now known as the Hawthorne Effect: when people feel others are paying attention to them, their productivity tends to go up, at least temporarily. (It’s a huge benefit of management consultants; just by showing up, we’re likely to make things better.)

To avoid congruence bias, don’t be satisfied with direct reasoning alone. Direct confirmation asks, “If I behaved in accordance with my hypothesis, what would I expect to occur?” Indirect confirmation asks, “If I acted in conflict with my hypothesis, what would I expect to occur?” If Taylor had stopped with the first question, we’d all be fiddling with the lights. Only the second question allowed him to discover the deeper truth.

Experimenter’s bias. This bias is well known to anyone in scientific fields. It’s the tendency for experimenters to believe and trust data that agrees with their hypothesis, and to disbelieve and distrust data that doesn’t. It’s a natural enough feeling; there’s a price to pay if we’re wrong, even if it’s only a hit to our egos. It’s impossible for any human being to be completely objective. Our perceptions and intelligence are constrained, and we are looking from the inside, not the outside.

Experimenter’s bias can’t be avoided; it has to be managed instead. Last week, we discussed the “bias blind spot,” the recursive bias of failing to recognize that you have biases. Self-awareness helps. Another good technique is the “buddy system.” I frequently work with co-authors so I have someone to challenge my thinking. That reduces the problem, though it doesn’t eliminate it — wherever my co-author and I see it the same way, the risk remains.

The best technique is to understand the components of the bias. A 1979 study of sampling and measurement biases listed 56 different experimenter’s biases: the “all’s well” literature bias, the referral filter bias, the volunteer bias, the insensitive measure bias, the end-digit preference bias, and my favorite, the data dredging bias, also known as “looking for the pony.”

More next week…