Even More About 1-Way ANOVA

It is called 1-way because there is one independent variable is this design. It is called an ANOVA because that’s an acrostic for ANalysis Of VAriance. An 1-way analysis of variance is a pre-test to prevent Type I error.

Although we try to control Type I error by setting our alpha level at a reasonable level of error (typically 5%) for one test, when we do several tests, we run into increased risk of seeing relationships that don’t exist. One t-test has a 5/100 chance of having Type I error. But multiple t-tests on the same data set destroy the careful controls we set in place.

We can use a t-test to compare the means of two groups. But to compare 3, 4 or more groups, we’d have to do too many t-tests; so many that we’d risk finding a significant t-test when none existed. If there were 4 groups (A, B, C and D, we’ll call them), to compare each condition to another you’d have to make the following t-tests:

                  AB
                  AC
                  AD
                  BC
                  BD
                  CD

The chances too good that we’ll find one of those tests to look significant but not be. What we need is a pre-analysis of data to test the overall design and then go back, if the overall variance is significant, and conduct the t-tests.

 

A Ratio

The premise of an ANOVA is to compare the amount of variance between the groups to the variance within the groups.

The variance within any given group is assumed to be due to chance (one subject had a good day, one was naturally better, one ran into a wall on the way out the door, etc.). There is no pattern to such variation; it is all determined by chance.

If no experimental conditions are imposed, it is assumed that the variance between the groups would also be due to chance. Since subjects are randomly assigned to the groups, there is no reason other than chance that one group would perform better than another.

After the independent variable is manipulated, the differences between the groups is due to chance and the independent variable. By dividing the between group variance by the within variance, the chance parts should cancel each other out. The result should be a measure of the impact the independent variable had on the dependent variable. At least that’s the theory behind the F test.

 

The F Test

Yes, this is the same F test we used doing an Analysis of Regression. And it has the same summary table:

                               SS              df              ms
Between
Within
Total 

Notice that the titles have changed. We know talk about Between Sum of Squares, not Regression SS. The F test (named after its author, R.A. Fischer) is the ratio of between-group variance (called between mean squares or mean squares between) to within-group variance (called within mean squares or mean squares within).

 

What To Do

After you calculate the F, you compare it to the critical value in a table of Critical Values of F. There are several pages of critical values to choose from because the shape of F distribution changes as the number of subjects in the study decreases. To find the right critical value, go across the degrees of freedom for regression (df between) and down the df within.

Simply compare the value you calculated for F to the one in the table. If your F is equal to or higher than the book, you win: what you see is significantly different from chance. The table we most often use is the .05 alpha level because our numbers aren’t very precise, so we’re willing to accept 5% error in our decisions. In other words, our alpha level is set .05 (the amount of error we are willing to accept). Setting the criterion at .05 alpha indicates that we want to be wrong no more than 5% of the time. Being wrong in this context means to see a significant relationship where none exists.

 

5% Error

Two points should be made: (a) 5% is a lot of error and (b) seeing things that don’t exist is not good. Five-percent of the population of the US is over 50 million people; that’s a lot of error. If elevators failed 5% of the time, no one would ride them. If OPEC trims production by 5%, they cut 1.5 million barrels a day. There are 230 million people use the internet, about 5% of the world’s population.

We use a relatively low standard of 5% because our numbers are fuzzy. Social science research is like watching a tennis game through blurry glasses which haven’t been washed in months. We have some understanding of what is going on-better than if we hadn’t attended the match-but no easy way to summarize the experience.

Second, seeing things that don’t exist is dangerous. In statistics, it is the equivalent of hallucination. We want to see the relationships that exist and not see additional ones that live only in our heads. Decisions which produce conclusions of relationship that don’t exist are called Type I errors.

If Type I error is statistical hallucination, Type II error is statistical blindness. It is NOT seeing relationships when they do exist. Not being able to see well is the pits (I can tell you from personal experience) but it’s not as bad as hallucinating. So we put most of our focus on limiting Type I error.

We pick an alpha level (how much Type I error we are willing to accept) and look up its respective critical value. If the F we calculate is smaller than the critical value, we assume the pattern we see is due to chance. And we continue to assume that it is caused by chance until it is so clear, so distinct, so accurate and it can’t be ignored. We only accept patterns that are significantly different from chance.

When the F we calculate is larger than the critical value, we are 95% sure that the pattern we see is not caused by chance. By setting the alpha level at .05, we have set the amount of Type I decision error at 5%.

 

 EXAMPLE

Group 1          Group 2             Group 3
     6                     6                        13
     4                     1                        10
     5                     3                          6
     8                     2                          7
     2                     8                          9

In order to calculate an Analysis of Variance for this data, we fill in the blanks for the Analysis of Variance’s summary table:

                      SS             df                ms 
Regression   ____       ____            ____ 
Error            ____        ____            ____ 
Total            ____        ____            ____

Degrees of Freedom 

Let’s start with the degrees of freedom. Like ANOR, an ANOVA Since this is a simple linear regression, we know that dfbetween = k-1, where k is the number of columns (kolumns) or groups (kroups?). Three columns minus 1 = 2.

We know that dfwithin is equal to N-k (the number of people minus the number of columns); so 15-3 = 12. The total degrees of freedom is equal to N-1; 15-1 = 14.

With this in mind, let’s update the summary table with what we know:

                      SS             df            ms 
Regression   ____           2            ____ 
Error            ____          12            ____ 
Total            ____          14            ____

Calculating SS

To calculate the Sum of Squares for this study, we begin with collecting some summary information. Find the sum of each group, the n of each group, X-squares of each group (square each score and add them up), and the SS for each group. Then, add each column across and put it in a Totals column. Like this:

                  X1        X2         X3
                  6           6          13
                  4           1          10
                  5           3            6
                  8           2            7
                  2           8            9

Sum          25         20          45          90
SumXsq  145       114        435        694
N                5           5           5           15
SS             20         34         30           84

 

Sum of Squares Within

SS within each group (20, 34 and 30, respectively) is the amount of dispersion WITHIN that group. So the sum of those SS, gives us Within SS (84 in this case). That is, SSwithin is the SSx1 + SSx2 + SSx3….

Sum of Squares Between

The SSbetween is a bit more challenging to calculate. Here is the formula for it:

 

Impressive, huh?

Let me explain. It won’t be so bad if we go slow. We start with the little groups (columns).

First, take the sum of the first column (it’s 25) and square it; 625.

     Do the same for each column.

         The square of 20 is 400.

         The square of 45 is 2025

Now add them together. So 625 + 400 + 2025 = 3050

Then divide 3050 by 5. We are dividing it by the number of scores in each group. NOT the number of columns; remember, it’s the number of scores. So 3050 divided by 5 gives us 610. 

Now square the total of all the raw scores (90) and divide it by N (the number of people in the study). Okay, 90 times itself is 8100. And 8100/15 = 540.

And subtract this from the subgroup numbers:

610 minus 540 = 70

The SSbetween equals 70.

  

Sum of Squares Total

To find the SStotal we use the totals column and plug those numbers into the regular Sum of Squares formula. That is, we ignore which group the scores come from and treat them as if they were all in one group. So, 694 minus (902) / 15. Notice that the last part of the formula has already been calculated with we did the SSbetween. What we come to, then, is 694 minus 540, which equals 154. The SStotal = 154.

Updating the summary table gives us:

                    SS       df          ms
Between    70.00      2       35.00
Within       84.00    12         7.00  
Total       154.00    14       11.00

 

Finding F

F is mean squares between (msbetween) divided by mean squares within (mswithin). In this case, F = 35 / 7 = 5.00

To test the significance of this F, we look up the critical value for the test at 2 and 12 degrees of freedom. Using the F Table, we find that the critical value at .05 alpha is 3.88. Since the value we calculated is larger than the one in the book, F is significant.

 

Interpretation

Since the F is significant, what do we do now?

Now, all of those t-tests we couldn’t do because we were afraid of Type I error are available for our calculating pleasure. So we do t-tests between:

AB
AC
BC

We might find that there is a significant difference between each group, such as this:

 

Or we might find that there is a not a significant difference between two of the groups but that there is a significant difference between them and the third group:

 

Also, which did best depends on whether the numbers are of money (you want the higher means) or errors (you want to lower means). Doing the t-tests between each combination of means will tell us which ones are significant, and which are likely to be due to chance.

Just think, if the F had not been significant, there would not be anything left to do. We would have stopped with the calculating of F and concluded that the differences we see are due to chance. How boring, huh? It’s a lot more fun to do lots of t-tests. Where’s my calculator?

 

 

NOW YOU CHOOSE
    Day9: 1-Way ANOVA
    Bit More About 1-Way ANOVA
    Even More About 1-Way ANOVA
    Calculate 1-Way ANOVA
    Practice Problems
    More Practice Problems
    Word Problems
        Sim1        Sim2         Sim3
        Sim4        Sim5         Sim6
        Sim7        Sim8         Sim9
    Vocabulary
    Formulas
    Quiz 9
    Summary