What to Do if Levenes Test Is Significant in Mixed Anova

---

An ANOVA is a statistical exam that is used to make up one's mind whether or not in that location is a statistically meaning divergence between the means of 3 or more independent groups.

The hypotheses used in an ANOVA are as follows:

The zilch hypothesis (H₀): µ₁ = µ_ii = µ₃ = … = µ₁₀₀₀(the ways are equal for each group)

The alternative hypothesis: (Ha): at to the lowest degree one of the ways is different from the others

If the p-value from the ANOVA is less than the significance level, we can reject the null hypothesis and conclude that we have sufficient evidence to say that at least one of the means of the groups is different from the others.

Notwithstanding, this doesn't tell uswhichgroups are different from each other. It simply tells us that not all of the group means are equal.

In lodge to find out exactly which groups are unlike from each other, nosotros must conduct a post hoc examination(too known as a multiple comparison exam), which will allow us to explore the deviation between multiple grouping means while too decision-making for the family-wise mistake charge per unit.

Technical Annotation: It'southward important to annotation that we only need to conduct a postal service hoc exam when the p-value for the ANOVA is statistically significant. If the p-value is not statistically significant, this indicates that the means for all of the groups are not different from each other, so at that place is no need to deport a mail hoc test to find out which groups are unlike from each other.

The Family-Wise Mistake Charge per unit

As mentioned earlier, mail service hoc tests allow usa to test for difference between multiple group means while likewise decision-making for the family-wise error rate.

In a hypothesis test, at that place is e'er a type I fault rate, which is defined by our significance level (alpha) and tells usa the probability of rejecting a nothing hypothesis that is really true. In other words, it'south the probability of getting a "fake positive", i.e. when we claim there is a statistically significant divergence among groups, just there actually isn't.

When we perform one hypothesis test, the blazon I mistake rate is equal to the significance level, which is ordinarily chosen to be 0.01, 0.05, or 0.x. However, when we behave multiple hypothesis tests at once, the probability of getting a false positive increases.

For example, imagine that we roll a xx-sided dice. The probability that the die lands on a "ane" is only v%. But if we roll two dice at once, the probability that one of the dice will land on a "ane" increases to 9.75%. If we whorl 5 dice at once, the probability increases to 22.vi%.

The more dice we roll, the college the probability that 1 of the die will state on a "1." Similarly, if we conduct several hypothesis tests at once using a significance level of .05, the probability that we get a imitation positive increases to beyond just 0.05.

Multiple Comparisons in ANOVA

When we conduct an ANOVA, at that place are often iii or more than groups that we are comparing to one another. Thus, when we conduct a mail service hoc test to explore the difference betwixt the group means, in that location are severalpairwisecomparisons we want to explore.

For example, suppose we have four groups: A, B, C, and D. This ways at that place are a total of six pairwise comparisons nosotros desire to look at with a post hoc test:

A – B (the departure betwixt the group A mean and the group B mean)
A – C
A – D
B – C
B – D
C – D

If nosotros accept more than four groups, the number of pairwise comparisons we volition want to look at volition only increase even more. The following tabular array illustrates how many pairwise comparisons are associated with each number of groups along with the family-wise fault rate:

Notice that the family-wise error rate increases rapidly every bit the number of groups (and consequently the number of pairwise comparisons) increases. In fact, in one case we attain six groups, the probability of us getting a fake positive is actually above 50%!

This means we would accept serious doubts about our results if we were to make this many pairwise comparisons, knowing that our family-wise fault rate was and then high.

Fortunately, postal service hoc tests provide u.s.a. with a way to make multiple comparisons between groups while controlling the family unit-wise fault charge per unit.

Case: One-Way ANOVA with Post Hoc Tests

The post-obit example illustrates how to perform a one-way ANOVA with postal service hoc tests.

Note: This instance uses the programming language R, just you don't need to know R to sympathise the results of the test or the big takeaways.

Kickoff, we'll create a dataset that contains 4 groups (A, B, C, D) with 20 observations per grouping:

                          #brand this instance reproducible                prepare.seed(1)                #load                tidyr                library to convert data from wide to long format              library(tidyr)              #create wide dataset              data <- data.frame(A = runif(20, two, 5),                    B = runif(20, iii, 5),                    C = runif(twenty, 3, 6),                    D = runif(twenty, 4, 6))              #catechumen to long dataset for ANOVA              data_long <- gather(data, key = "group", value = "amount", A, B, C, D)              #view first vi lines of dataset              head(data_long)  #  group   amount #1     A 2.796526 #ii     A three.116372 #3     A three.718560 #four     A 4.724623 #5     A 2.605046 #6     A 4.695169                      

Adjacent, we'll fit a 1-way ANOVA to the dataset:

                          #fit anova model              anova_model <- aov(amount ~ group, information = data_long)              #view summary of anova model              summary(anova_model)  #            Df Sum Sq Mean Sq F value   Pr(>F)     #group        iii  25.37   viii.458   17.66 8.53e-09 *** #Residuals   76  36.39   0.479                      

From the ANOVA tabular array output, we see that the F-statistic is 17.66 and the corresponding p-value is extremely pocket-sized.

This ways we accept sufficient evidence to reject the null hypothesis that all of the group means are equal. Next, we can use a mail hoc test to notice which group means are unlike from each other.

We volition walk through examples of the following postal service hoc tests:

Tukey's Test – useful when you desire to brand every possible pairwise comparison

Holm'southward Method – a slightly more conservative examination compared to Tukey's Test

Dunnett's Correction – useful when you want to compare every group mean to a command mean, and you're not interested in comparing the treatment means with one another.

Tukey's Exam

We can perform Tukey's Exam for multiple comparisons past using the congenital-in R roleTukeyHSD() as follows:

                          #perform Tukey's Exam for multiple comparisons              TukeyHSD(anova_model, conf.level=.95)   #  Tukey multiple comparisons of means #    95% family-wise confidence level # #Fit: aov(formula = amount ~ group, data = data_long) # #$group #         diff          lwr       upr     p adj #B-A 0.2822630 -0.292540425 0.8570664 0.5721402 #C-A 0.8561388  0.281335427 i.4309423 0.0011117 #D-A 1.4676027  0.892799258 2.0424061 0.0000000 #C-B 0.5738759 -0.000927561 1.1486793 0.0505270 #D-B 1.1853397  0.610536271 1.7601431 0.0000041 #D-C 0.6114638  0.036660419 one.1862672 0.0326371                      

Notice that we specified our confidence level to exist 95%, which means nosotros desire our family-wise fault charge per unit to be .05. R gives us two metrics to compare each pairwise divergence:

• Confidence interval for the mean difference (given by the values oflwrandupr)
• Adjusted p-value for the mean difference

Both the conviction interval and the p-value will pb to the same conclusion.

For case, the 95% conviction interval for the hateful difference betwixt group C and group A is (0.2813, 1.4309), and since this interval doesn't contain zip we know that the departure between these two group ways is statistically significant. In particular, we know that the deviation is positive, since the lower bound of the confidence interval is greater than nix.

Likewise, the p-value for the mean departure betwixt group C and group A is 0.0011, which is less than our significance level of 0.05, and so this also indicates that the difference betwixt these ii grouping means is statistically significant.

We tin can too visualize the 95% confidence intervals that effect from the Tukey Examination by using the plot() part in R:

            plot(TukeyHSD(anova_model, conf.level=.95))                      

If the interval contains zero, and so nosotros know that the difference in group ways is not statistically significant. In the example above, the differences for B-A and C-B are not statistically significant, but the differences for the other four pairwise comparisons are statistically meaning.

Holm's Method

Another post hoc exam we tin can perform is holm'south method. This is more often than not viewed as a more conservative test compared to Tukey'due south Exam.

We can utilize the post-obit code in R to perform holm'southward method for multiple pairwise comparisons:

                          #perform holm's method for multiple comparisons              pairwise.t.test(data_long$amount, data_long$grouping, p.arrange="holm")  #	Pairwise comparisons using t tests with pooled SD  # #data:  data_long$corporeality and data_long$grouping  # #  A       B       C       #B 0.20099 -       -       #C 0.00079 0.02108 -       #D ane.9e-08 3.4e-06 0.01974 # #P value adjustment method: holm                      

This examination provides a filigree of p-values for each pairwise comparison. For example, the p-value for the difference between the grouping A and grouping B mean is 0.20099.

If you compare the p-values of this examination with the p-values from Tukey's Test, you'll notice that each of the pairwise comparisons lead to the same conclusion, except for the departure between group C and D. The p-value for this difference was .0505 in Tukey's Exam compared to .02108 in Holm'south Method.

Thus, using Tukey's Exam we concluded that the difference between group C and grouping D was not statistically significant at the .05 significance level, but using Holm's Method we concluded that the difference betwixt grouping C and group Dwasstatistically significant.

In general, the p-values produced by Holm's Method tend to be lower than those produced by Tukey'south Test.

Dunnett's Correction

Even so another method we tin use for multiple comparisons is Dunett'due south Correction. We would use this approach when we desire to compare every group hateful to a control mean, and we're non interested in comparing the treatment means with one some other.

For instance, using the code below we compare the group means of B, C, and D all to that of group A. And then, we employ group A as our control group and we aren't interested in the differences betwixt groups B, C, and D.

                          #load multcomp library necessary for using Dunnett'southward Correction              library(multcomp)              #convert grouping variable to factor              data_long$group <- as.gene(data_long$group)              #fit anova model              anova_model <- aov(corporeality ~ grouping, data = data_long)              #perform comparisons              dunnet_comparison <- glht(anova_model, linfct = mcp(group = "Dunnett"))              #view summary of comparisons              summary(dunnet_comparison)  #Multiple Comparisons of Means: Dunnett Contrasts # #Fit: aov(formula = amount ~ group, data = data_long) # #Linear Hypotheses: #           Estimate Std. Fault t value Pr(>|t|)     #B - A == 0   0.2823     0.2188   1.290 0.432445     #C - A == 0   0.8561     0.2188   iii.912 0.000545 *** #D - A == 0   1.4676     0.2188   6.707  < 1e-04 ***          

From the p-values in the output nosotros can see the post-obit:

• The difference between the group B and group A mean is not statistically significant at a significance level of .05. The p-value for this test is 0.4324.
• The difference between the group C and group A mean is statistically significant at a significance level of .05. The p-value for this exam is 0.0005.
• The divergence betwixt the group D and grouping A mean is statistically significant at a significance level of .05. The p-value for this test is 0.00004.

Every bit we stated earlier, this approach treats group A every bit the "control" group and but compares every other group mean to that of group A. Observe that in that location are no tests performed for the differences between groups B, C, and D because we aren't interested in the differences betwixt those groups.

A Note on Postal service Hoc Tests & Statistical Power

Post hoc tests do a great task of controlling the family unit-wise fault rate, only the tradeoff is that they reduce the statistical power of the comparisons. This is because the only way to lower the family-wise mistake rate is to use a lower significance level for all of the individual comparisons.

For case, when we use Tukey's Test for six pairwise comparisons and nosotros want to maintain a family unit-wise error rate of .05, we must use a significance level of approximately 0.011 for each individual significance level. The more pairwise comparisons nosotros have, the lower the significance level we must apply for each private significance level.

The trouble with this is that lower significance levels represent to lower statistical power. This means that if a difference between group ways actually does be in the population, a written report with lower power is less likely to notice information technology.

One style to reduce the effects of this tradeoff is to simply reduce the number of pairwise comparisons we make. For instance, in the previous examples we performed six pairwise comparisons for the iv different groups. All the same, depending on the needs of your report, you may only exist interested in making a few comparisons.

Past making fewer comparisons, you lot don't accept to lower the statistical power as much.

It's of import to annotation that you should determinebeforeyou lot perform the ANOVA exactly which groups you lot want to brand comparisons between and which post hoc test y'all volition use to make these comparisons. Otherwise, if you but see which mail service hoc exam produces statistically pregnant results, that reduces the integrity of the written report.

Determination

In this post, we learned the following things:

• An ANOVA is used to determine whether or non there is a statistically significant divergence between the ways of 3 or more contained groups.
• If an ANOVA produces a p-value that is less than our significance level, we can use post hoc tests to find out which group ways differ from ane another.
• Post hoc tests allow united states to control the family unit-wise fault rate while performing multiple pairwise comparisons.
• The tradeoff of controlling the family unit-wise error rate is lower statistical power. We can reduce the effects of lower statistical power by making fewer pairwise comparisons.
• Y'all should determine beforehand which groups yous'd like to brand pairwise comparisons on and which post hoc test y'all volition use to do and so.

carpentersequal.blogspot.com

Source: https://www.statology.org/anova-post-hoc-tests/

Share this post