Chapter 26 Introduction to two-way ANOVA

The experimenter who believes that only one factor at a time should be varied is amply provided for by using a factorial experiment.

Box et al. (2005)

26.1 Introduction

One-way ANOVA allows us to determine whether there are significant differences between the effects of two or more treatments. The treatments we are interested in comparing are the different levels of a factor. These levels may represent quantitative variations of a general treatment (e.g. the effect of different concentrations of slug poison on slugs), or qualitatively different varieties of a class of treatments (e.g. the effect of different diets on weight loss). Fairly obviously, we are less likely to be interested in questions which involve comparing completely different sorts of treatment. For example, it is hard to see the value of an experiment comparing the movement of slugs where the three different treatments are: (i) half-strength Slugit (ii) moist wood substrate (iii) darkness. None of the treatments are comparable and so it will be very difficult to interpret differences between them.

Although the above experiment would not be very useful, we might well be interested in whether the moisture level of the substrate has an effect on movement rate, and similarly, whether movement is affected by the slug being in the light or dark. To address these questions we could obviously design two separate experiments—one where the treatments are wooden boards with two or more different moisture levels, and another where the treatments are ‘light’ or ‘dark’.

Although this is a perfectly valid procedure, it still leaves us lacking some information. With the moisture experiment, we have to decide whether to run it in the light or the dark, and with the light and dark experiment we have to decide how moist the wooden boards we use should be. If we decide to run our moisture level experiment in the dark, then we end up (we hope) knowing something about the effect of moisture on slug movement, but only in dark conditions—we can’t say whether the effect of moisture would have been different had the slugs been in the light (and obviously we can’t say anything about the effect of light and dark more generally since that is the subject of our other experiment).

One obvious solution might be to run two moisture experiments… one in the dark and one in the light, and similarly three light/dark experiments, one at each different moisture level. This is indeed what we want to do, but instead of running each combination separately, it is more powerful, and experimentally less problematic, to run all the combinations together.

If we used four slugs in each combination, we would require: 4 replicates \(\times\) 3 moisture levels \(\times\) 2 light levels \(=\) 24 slugs. We would end up with 4 measurements of movement rate in each combination of treatments (figures are cm per min), e.g.

		Moisture level
Light level	<5%	50%	100%
Dark	2, 3, 5, 0	3, 9, 5, 10	15, 8, 11, 12
Light	4, 2, 7, 1	10, 7, 4, 13	13, 17, 12, 9

An experiment of this sort, where measurements are made under each combination of several levels of two or more different kinds of experimental treatments, is called a fully factorial experiment. This type of experimental design gets its name from the fact it involves every combination of treatments among two or more factors. The example here is a two-factor experiment because it has two different kinds of treatment (illumination and moisture).

It should be straightforward to see that different factors can be combined in a single experiment, and that this seems to yield the maximum amount of information, but to get at that information we need to be able to analyse the data. Fortunately the principles of ANOVA that you have seen already can be extended to provide a powerful and elegant way of analysing data from factorial designs. With two different sets of treatment (as here) this approach is referred to as two-way ANOVA (also known as two-factor ANOVA).

A two-way ANOVA on data from the slug experiment would tell us whether slug movement was affected by (1) moisture and (2) illumination, and (3) whether the effect of illumination depends on moisture levels (and vice versa). So instead of just one result (as we get from a one-way ANOVA) there are now three to consider. The effect of moisture and of illumination are termed main effects and the effect of each moisture level / illumination combination is termed the interaction.

What are these?

The main effects are fairly obvious:

The moisture effect: …tells you whether there is a significant difference between the mean movement of slugs among the three moisture levels (i.e., the means of the data in each of the three columns in the table above, across both light levels).
The illumination effect: …tells you whether there is any difference between slug movement in the light and dark (i.e., the means of the data in each of the two rows in the table above, across all moisture levels).

The interaction is a bit more tricky:

The interaction between moisture and illumination: …tells you whether there are differences between slug movement rates which are due to specific combinations of different moisture and illumination levels, which cannot be accounted for just by combining the mean effects of moisture level and of illumination level (i.e. are there differences between the means of the data from each cell in the table, having taken account of the overall effects of moisture and illumination?). Another way of looking at the interaction is that it indicates whether slug movement responds differently to moisture depending on whether it is in the light or dark.

All this will probably make more sense when we have an example to work with, so we’ll carry out a two-way ANOVA and then come back to how the results should be interpreted.

Treatments

When writing about factorial experiments, the word ‘treatment’ tends to be used in two subtly different ways:

Some people enumerate different treatments at the level of combinations of factor levels. For example, if we were carrying out an experiment with two factors, each of which has two levels (‘A’ vs. ‘B’ and ‘X’ vs. ‘Y’), we would say that the experiment has four treatments.
Others delineate treatments at the level of individual factors, and then refer to ‘treatment combinations’ to distinguish unique experimental conditions. In our example we would say that each factor involves two treatments and overall, the experiment involves four treatment combinations.

Notice that ‘factor levels’ and ‘treatments’ are synonymous when using the second naming convention (this suggests the first definition is probably the more useful one). We will adopt this second convention in this course because it is so widely used.

26.2 Degrees of freedom, mean squares, and F-statistics

We are not going to step through the logic underpinning the calculations of the degrees of freedom, sum of squares, mean squares, and F-statistics. The logic is no different than that used in one-way ANOVA. It is a bit trickier to explain and visualise though. Ultimately, an F-statistic is calculated for each term, which is the ratio of the term’s mean square and the error mean square. A higher F-statistic is more likely to be significant, and the p-value is calculated by comparing the F-statistic to the theoretical F distribution.

26.3 Multiple comparison tests

Having established that there are significant differences, we might wish to go further and specify between which means these differences occur. With one-way ANOVA obviously there was only one set of means to compare with the multiple comparison tests. Now, however, there are three possible sets of means: the two main effects and the interaction.

26.4 Beyond two-way ANOVA

It is possible to have more complex designs using 3 or more factors (‘multi-way fully factorial’)—for example we could add to our experiment considered earlier by running our existing treatment combinations at each of three different temperatures—but as the experiment becomes more complex, so does the analysis and interpretation (and also the work involved in running it: adding three temperature treatments would mean we needed 72 slugs!).

A multi-way factorial design isn’t really a ‘different’ kind of design from the two-way case we have examined. The principle of the 2-way fully factorial design can be directly extended to multi-way fully factorial designs. In a three-way design (factors A, B, C) there are three main effects (A, B, C), three pairwise interactions (A x B, A x C, B x C) and one new kind of interaction: a three-way interaction (A x B x C). The challenge posed by such designs is that the results can be tricky to interpret (what is a three-way interaction?).

Analysis of Variance is a large and complex subject—we are only scratching the surface in this book. Most intermediate level biostatistics texts deal with the more involved designs of ANOVA. As with many aspects of statistics and experimental design there is much to be said for doing experiments and analyses you are confident you understand and can interpret, even if more complex forms of analysis are technically possible (providing of course the simpler approach is appropriate!). When contemplating a design that looks like it might require more than two factors, it is a good idea to talk to someone who knows about these things to ensure that is indeed necessary.