Even More About Probability

 When one thing causes another it tends to have a linear pattern. Someone stomps on your toes, you scream. The harder the stomp, the more you scream. If you graphed this pattern, you’d find that pressure and loudness looked somewhat like a diagonal line. It would only be somewhat straight because high pressure doesn’t always hurt more than light pressure, it just usually does.

In contrast, if there were no relationship between pressure and loudness, the pattern would be more like a circle. The relationship between chance variables has its own pattern: no consistent pattern. Chance tends to change. It might look exactly like a circle, then somewhat like a circle, somewhat like a straight line, and then back to looking like a circle. The trick with chance is inconsistency.

The reason we replicate studies is to show consistency. We know that the results of one study might look like causation but the next study will show opposite results. The results of a single study might be due to chance. So we conduct a series of studies in the area, publish the results, and let other researchers try to replicate our findings.

NULL HYPOTHESIS

Our other protection against chance is we use a null hypothesis. We start with the hypothesis that what we see is going to be due to chance. And we discard that hypothesis only when there is a substantial reason to do so. We assume that a variable has no significant impact on another. Unless we have evidence to the contrary, we accept the null hypothesis. We need a significant amount of evidence for us to reject the null hypothesis, and to say that there is a significant relationship between variables.

Our approach is equivalent to presuming the innocence of a criminal suspect. People are assumed to be innocent until proved guilty. And if they are not proved guilty, they are not declared innocent; they are “not guilty.” Similarly, the lack of a significant finding in research doesn’t mean that a causal relationship doesn’t exist, only that we didn’t find it.

ALPHA LEVEL

We also limit the amount of error we are willing to accept. That is, we decide ahead of time how far away from being a circle (how close to being a straight line) does the data have to be before we say it is significant. It’s good to set the standard beforehand. Otherwise, we’d be tempted to change the standard to fit the data. To avoid that problem, we typically only allow an error rate of 5%, which is the same cutoff score we used for z-scores and correlations.

We could use a higher standard: 3% error or 1% error, but we use a relative low standard of 5% because our numbers are fuzzy. Social science research is like watching a tennis game through blurry glasses that haven’t been washed in months. We have some understanding of what is going on—better than if we hadn’t attended the match—but no easy way to summarize the experience. So 5% is a good level for us.

DECISION ERROR

The kind of error we are limiting to 5% is decision error. We’re setting the point in the distribution beyond which we’re going to discard our null hypothesis (nothing is going on) and accept our alternative hypothesis (something is going on). If we set the standard too low, everything we test will look significant. Seeing significant finding in random events is the equivalent of a statistical hallucination. We only want to see the relationships that exist and not see additional ones that live only in our heads. Decisions which produce conclusions of relationship that don’t exist are called Type I errors.

If Type I error is statistical hallucination, Type II error is statistical blindness. It is NOT seeing relationships when they do exist. Not being able to see well is the pits (I can tell you from personal experience) but it’s not as bad as hallucinating. So we put most of our focus on limiting Type I error.

We pick an alpha level (amount of Type I error) and look up its respective critical value in a table. If the value we calculate is smaller than the critical value, we assume the pattern we see is due to chance. And we continue to assume that it is caused by chance until it is so clear, so distinct, so significant that it can’t be ignored. We only accept patterns that are significantly different from chance.

NOW YOU CHOOSE:
    Day 7: Probability
    Bit More About Probability
    Even More About Probability
    Even More About ANOR
    Calculate ANOR
    Practice Problems
    More Practice Problems
    Word Problems
        Sim1       Sim2        Sim3
    Basic Facts About Probability
    Vocabulary
    Formulas
    Quiz 7
    Summary