Our fifth Unitarian Universalist source is:

Humanist teachings which counsel us to heed the guidance of reason and the results of science, and warn us against idolatries of the mind and spirit.

There is a Shakespearean tragedy called Othello. In it, Othello is married to Desdemona, and is friends with and commander of Iago and Cassio. Iago manages to convince Othello that Desdemona is cheating on him with Cassio. Othello kills Desdemona, and then when it is revealed that Iago had been lying to him and Desdemona was innocent, Othello commits suicide.

Now, Iago is definitely the villain here. If he had not manipulated people, it would not be a tragedy. Also, since the society they lived in thought that murder for infidelity was acceptable, the society was also to blame.¹ But one more problem is Othello ended up thinking the world was not the way the world actually was. Had Othello correctly realized Iago was lying and Desdemona was innocent, the play would not have been a tragedy.

Having read the Othello play and looked only at what Othello knew, I am not sure Othello could have reliably figured this out. However, I think he missed several things which should have made him uncertain as to the state of the world.

Knowing what is real is important. If you don’t know where the ground is, eventually you will walk off a cliff.

Human minds are not optimal, because evolution does not have a mind so the results tend to have kludges instead of well-thought-out designs. Jonathan Haidt argues human reasoning evolved to convince other people. By default, our minds are lawyers, not judges or scientists.² Our own mind’s default mode of operation is to find reasons why our current views are right, not what the truth actually is. That is a scary thought.

Eliezer Yudkowsky thinks this problem is so bad that learning more knowledge or more about human bias can make someone stupider.³ He calls this problem motivated skepticism. In it, if you have more knowledge, then you can pick and choose the pieces of evidence to only use the ones that support your view. If you know more about human biases, then if you are a motivated skeptic, you can find more flaws in the arguments you don’t like. Human brains can defeat themselves. By default we use our intelligence to defeat our search for truth. Basically, if you want the truth, you have to search as hard for flaws in beliefs you want to be true as you search for flaws in beliefs you want to be false.

Today, I am going to talk about a specific piece of guidance given us by science and mathematics, called multiple hypothesis testing. Basically, it is useful when there could be multiple possibilities, for what is happening in the world, and we have to figure out which could be true. So we need to look at the available evidence, and compare it to each possibility, and then try to figure out which of the possibilities is correct. There are mathematical formulas for doing multiple hypothesis testing, and if you have the probabilities needed or can determine them, the math is the correct way to do so. Unfortunately in our lives, calculating the probabilities may be extremely difficult. I think even without the exact probabilities, we can do better in our lives.

I am speaking not because I am perfect at multiple hypothesis testing, but because I have failed at it. I have failed despite spending plenty of time trying to figure out the truth and deeply caring about finding the truth. I have made my mistakes and I hope I can fail less in the future.

So the first step to finding the truth is to look at the existing evidence. Writing this down can be very helpful. Try to think of all the evidence, including the pieces that are hardest to explain. If your mind is misleading you, it will think of the strongest pieces of evidence for your favorite hypothesis.

As Charles Darwin wrote:

Once you have the current evidence listed, next you need to determine which hypotheses should be considered. This can be one of the hardest parts. For Othello, three possible hypotheses might be:

1. Desdemona doesn’t care at all about Cassio.
2. Desdemona considers Cassio a friend.
3. Desdemona is having an affair with Cassio.

With those hypotheses in mind, then as each piece of evidence comes in, look at each hypothesis and decide how likely it is the evidence would happen if the hypothesis is true. For example, after Cassio loses his military command and Desdemona asks Othello to consider helping Cassio regain his command, that behavior would be most likely if Desdemona considers Cassio a friend or lover. So those two hypotheses would increase in likelihood and the hypothesis that she doesn’t care would decrease.

Another example is when Iago tells Othello that Desdemona gave her handkerchief to Cassio. Othello then asks Desdemona where her handkerchief is and she says she doesn’t have it with her, and then tries to distract Othello by bringing up Cassio’s military command. Distracting Othello by mentioning Cassio seems rather unlikely if Desdemona had given Cassio the handkerchief or was having an affair with him, so Othello should have decreased the probability of the hypothesis that Desdemona was having an affair with Cassio.

Also, any piece of evidence must have one of three relations to a hypothesis:

1. Evidence for the hypothesis
2. Evidence against the hypothesis
3. Neutral evidence about the hypothesis

If the evidence does not exist, then the relation must be reversed. So if you think seeing a particular piece of evidence is evidence for a hypothesis, then not seeing that particular piece of evidence must be evidence against the hypothesis.

Iago’s wife Emilia swears on her life Desdemona has not sneaked off with Cassio and Desdemona is honest, chaste and true. If Othello does not think this is evidence Desdemona is honest, chaste and true, then to be logically consistent, Othello would have to think Emilia saying Desdemona was dishonest, unchaste and untrue was evidence of Desdemona being honest, chaste and true. A good test of a piece of evidence is to think: what if the opposite had happened, would the opposite be evidence against the hypothesis?

Also, the more unexpected a piece of evidence, the more it should shift your beliefs. I am going to drop a coin. I am pretty sure it will fall. If it falls, falling should slightly increase my belief gravity always works and will cause it to fall. If it floats in the air, floating should massively decrease my belief in gravity always working. (Try dropping a coin.) My belief in gravity always working has just gone up a bit.

Now, how can you tell when your hypotheses are accurate? One good sign is being able to make accurate predictions. If what you think will happen happens, then that is a good hint that you might be correct in your beliefs. On the other hand, if you are surprised, then the surprise is a strong sign your hypotheses might be wrong. Writing down your predictions before hand is a good way to avoid cheating. Many times in life, it is easy to gather more evidence. Try to figure out what evidence would tell your hypotheses apart, and gather it.

There is one more major step we need to talk about. How do you tell when you have the wrong hypotheses? If you have a piece of evidence that is very unexpected for all the hypotheses, then that is an indication none of the current hypotheses are correct. Alternatively, if you get a sequence of several pieces of evidence, and each hypothesis receives at least one piece of evidence that would be very unexpected for that hypothesis, that can be an indication that none of the hypotheses are correct. At this point, you will need to find new hypotheses, which can be far trickier than just looking at the evidence and deciding between several hypotheses.

For Othello, on one hand he has Iago saying Cassio confessed that Desdemona and Cassio had lain with each other. On the other hand, Emilia is swearing on her life that Desdemona is innocent. In the hypotheses I originally listed for Othello, they are all about Desdemona, but the problem is coming from Iago, so until some hypotheses about Iago start entering Othello’s mind, at best Othello will be confused. If Othello had ever thought to put aside and ignore everything Iago had said, he might have realized that the rest of the “evidence” for Desdemona being unfaithful was very weak.

Noticing you are confused is usually the first step. Sometimes it is just a small bit of confusion, a hm, that is odd, in the back of your mind. It is easy to miss it in the rest of the noise in our minds. Also, in my experience, confusion doesn’t always feel like confusion. Sometimes it feels like anger, I suspect because it was easier for evolution to evolve the neural pathways that lead to anger than evolving the neural pathways that create the logical thought that leads to the evolutionarily advantageous action. I think a lot of mistakes could be avoided if, every time I was angry, I stepped back mentally and thought, could this anger be from confusion on my part. Othello could have prevented a tragedy if he had followed this rule. Sometimes confusion feels like surprise. Sometimes the surprise feels like being hit by a flash flood that knocks you off your feet.

So, when you are confused, or angry, or surprised, take some time to think, am I missing a possibility? Those are hints it is time to think harder about what is happening.

In order for Othello to have realized the truth, he needed to realize, despite knowing Iago for years and Iago having been honest so far as Othello could tell, Iago was lying to Othello. Iago lying was not something that ever really crossed Othello’s mind until too late and until the evidence was overwhelming that Iago was lying to and manipulating Othello.

I think it would have been a tragedy even had Othello realized in time. In which case the tragedy would be Othello had been betrayed by a friend he thought honest. I think part of the tragedy of life is, here in real life, not just plays, there are people who pretend for years⁵ to be friends, but are really trying to manipulate someone. And the only way to detect this is to always keep the hypothesis active that maybe this person is playing games or being dishonest.

I think Othello could have done much better than he did in the play. I am not sure he could have realized Iago was manipulating him, but I think he could have realized the evidence was not adding up, that he should not have been certain enough to make an irreversible act. I think the Tragedy of Othello could have been turned into the much less tragic Confusion of the Handkerchief. I think we humans can learn to be wiser, I think we can learn to be more rational. I think we can learn to make fewer mistakes. I think we can learn to heed the guidance of reason.

I hope we gain the wisdom to turn our tragedies into comedies.

1 Probability Appendix

1.1 Basic Equations

This describes some of the math that can be used for multiple hypothesis testing. Note there is a lot more on the subject, this just gives the basic equation and an example.

First of all, the notation $P\left(A\right)$ is the probability that $A$ is true and goes from 0 or infinitely impossible, to 1 or infinitely certain. $P\left(AB\right)$ is the probability that both $A$ and $B$ are true. The notation $P\left(A|B\right)$ is a conditional probability. It means the probability $A$ is true if $B$ is true.

The Product Rule⁶ can be used for deriving other equations:

The Sum Rule defines the relation between $A$ and $\neg A$ (Not $A$):

The next equation⁷ defines how to update a hypothesis based on new data.

where:

In the above equation $P\left(H|X\right)$ is the original probability. $P\left(D|HX\right)$ is the probability of the data $D$ if the hypothesis is true. $P\left(D|X\right)$ is just the probability of the data $D$.

1.2 Multiple Hypothesis Testing Example

To show how to use the formula, here is an example. There are two ball painting machines $M_1$ and $M_2$ Each machine will be randomly loaded with one of three possible colors, red, green, and blue. There are six possible hypothesis. If there is the same probability of getting each paint, then the probability is 1/9 (or 1/3 * 1/3)for any particular combination, and since it doesn’t matter if you change $M_1$ and $M_2$ the inverse combinations have 2/9 probability. The table summarizes the prior information on the hypotheses.

The first data found is that one of the machines produced a red ball (or $D=r$ for short). The first number we need is $P\left(D|X\right)$, in this case: $P\left(r|X\right)$. Since the probabilities of the three possible colors are identical, this will be $P\left(r|X\right)=\frac{1}{3}$.

For Hypotheses 1 ($H_1$), the next number we need is $P\left(D|HX\right)$, in this case: $P\left(r|H_{1}X\right)$. Since both machines produce only red balls,⁸ the probability is 1. So the math is:

For Hypothesis 2 ($H_2$), the number we need is $P\left(D|HX\right)$, in this case: $P\left(r|H_{2}X\right)$. Since neither machine produces red balls in this hypothesis,⁹ the probability is 0. So the math is:

Hypothesis 3 ($H_3$) will also be 0 for the same reason as $H_2$.

For Hypothesis 4 ($H_4$), the number we need is $P\left(r|H_{4}X\right)$. The machine produces red balls half of the time, so the probability is $\frac{1}{2}$. So the math is:

For Hypothesis 5 ($H_5$), the probability of producing a red ball is 0, so $P\left(r|H_{5}X\right)=0$:

Hypothesis 6 will be $\frac{1}{3}$ with the same calculation as $H_4$. The final results are:

1.3 Conservation of Expected Evidence

If you have a hypothesis $H$ you should not expect the probability of it to be increased or decreased with additional evidence.¹⁰

Two comments about this. One, if you have a high expectation to see some evidence, it will not increase the probability much more for your hypothesis. Two, if you think seeing something is evidence of a hypothesis, than not seeing that evidence must be evidence against the hypothesis.

Alternatively, if you have two mutually exclusive hypothesis, $A$ and $B$, so $P\left(A\right)+P\left(B\right)+P\left(\neg \left(AB\right)\right)=1$ then some related conclusions are

Basically, if something is evidence for $A$ and $B$, then it must decrease the probability of $P\left(\neg \left(AB\right)\right)$. The numbers have to add up.

2 Appendix on Evidence Othello sees from The Tragedy of Othello

2.1 Act I, Scene II

Iago says the magnifico will divorce Othello or cause the law to go against him.

Brabantio accuses Othello of stealing his daughter.

2.2 Act I, Scene III

Othello says he thought Desdemona was attracted to his tales of danger.

Desdemona requests to go with Othello to Cyprus when Othello goes to war.

Brabantio says: “She has deceived her father and may thee.”

2.3 Act II, Scene III

Othello states Iago is honest.

Othello comes back to find Montano injured. Cassio refuses to explain what happened. Iago says that Cassio injured Montano.

2.4 Act III, Scene III

Othello comes in to a room with Emilia and Desdemona and Cassio. Cassio leaves without speaking to Othello.

Iago says he doesn’t like that Cassio leaves guiltily.

Desdemona pleads on Cassio’s behalf and requests Othello call him back.

Iago tells Othello to watch Cassio and Desdemona closely and implies something is going on between them.

Desdemona drops her handkerchief at home and Othello tells her to just leave it instead of picking it up.¹¹

Iago says he and Cassio were sleeping together and Iago heard Cassio say “Sweet Desdemona” during his sleep then Cassio kissed him and put his leg over Iago during his sleep.

Iago says Cassio had Desdemona’s handkerchief.

Iago pledges to kill Cassio.

2.5 Act III, Scene IV

Othello asks Desdemona for the handkerchief. Desdemona says she doesn’t have it with her, but it is not lost. Othello is upset about this. She asks Othello to see Cassio again.

2.6 Act IV, Scene I

Iago says Cassio confessed to lying with Desdemona.

Iago say he will go talk to Cassio about Desdemona. Othello sees Cassio come and start talking to Iago but cannot hear the words. Bianca, a lover of Cassio, joins Iago and Cassio. Bianca gives Cassio Desdemona’s handkerchief.

Desdemona says she would do much to atone Othello and Cassio for the love she bears Cassio.

2.7 Act IV, Scene II

Emilia swears on her life to Othello that Desdemona has not sneaked off with Cassio and is honest, chaste and true.

Desdemona says if her father is the cause of Othello losing his command then her father is lost to her.

2.8 Act V, Scene I

Othello enters and sees Cassio injured and Iago nearby.

2.9 Act V, Scene II

Desdemona denies loving Cassio and giving him the handkerchief.