Kruskal-Wallis and Friedman test


Kruskal-Wallis is basically a non-parametric version of ANOVA. Thus, if you have the data which contain more than two groups to compare, and your data are ordinal or your data cannot assume the normality, Kruskal-Wallis is the way to go. Fortunately, in R, the way to run a Kruskal-Wallis test is similar to the way to run an ANOVA test.

There is also a non-parametric version of repeated-measure ANOVA, which is called Friedman test. If you are comparing the data against a between-subject factor, you can use a Kruskal-Wallis test. Otherwise, you need to use a Friedman test.

As you can see in the example below, Kruskal-Wallis and Friedman tests only support a one-way analysis. This means that you can compare the data only across one factor and unfortunately, you cannot easily extend these tests to two-way or mixed-design as we can do with ANOVA. This is one of the costs you have to pay when you have to use a non-parametric test. I will post non-parametric methods equivalent to two-way repeated-measure or mixed-design ANOVA once I figure them out.

Effect size

Unfortunately, there is no straight way to calculate the effect size of Kruskal Wallis test or Friedman test. Instead, you can calculate the effect size in the post-hoc test with Mann-Whitney or Wilcoxon test. The effect size of those tests are calculated as follows:


where N is the total number of the samples. Here is the standard value of r for small, medium, and large sizes.

small sizemedium sizelarge size

Kruskal-Wallis and Friedman tests give you a chi-squared. However, its degree of freedom is more than 1, and thus it is not straightforward to convert the chi-squared into the effect size. Thus, we calculate the effect size for the post-hoc comparison.

R code example (Kruskal-Wallis)

First, you need to prepare the data

> Value <- c(1,2,5,3,2,1,1,3,2,1,4,3,6,5,2,6,1,6,5,4, 
+ 9,6,7,7,5,1,8,9,6,5)
> Group <- factor(c(rep(1,10),rep(2,10),rep(3,10)))
> data <- data.frame(Group, Value)

Then, run a Kruskal-Wallis test. The format is pretty much the same as ANOVA.

> kruskal.test(Value ~ Group, data=data)

Now you get the result.

		Kruskal-Wallis rank sum test

data:  Value by Group 
Kruskal-Wallis chi-squared = 13.6754, df = 2, p-value = 0.001073

So, we have a significant effect of Group.

R code example (Friedman)

Let's use the same data as in the example of a Kruskal-Wallis test. But we need to format the data so that Group represents a within-subject factor.

> data2 <- cbind(data[data$Group==1,]$Value, data[data$Group==2,]$Value,
+ data[data$Group==3,]$Value)
> data2

	  [,1] [,2] [,3]
 [1,]    1    4    9
 [2,]    2    3    6
 [3,]    5    6    7
 [4,]    3    5    7
 [5,]    2    2    5
 [6,]    1    6    1
 [7,]    1    1    8
 [8,]    3    6    9
 [9,]    2    5    6
[10,]    1    4    5

The rows represent the values from each participant. Now, you run a Friedman"s test.

> friedman.test(data2)

		Friedman rank sum test

data:  data2 
Friedman chi-squared = 15.6216, df = 2, p-value = 0.0004053

You have a significant effect of Group.

Post-hoc test

Similar to ANOVA, you need to do a post-hoc test after Kruskal-Wallis and Friedman if you find a significant effect. As we do multiple t tests with Bonferroni or Holm correction, we can do Mann-Whitney or Wilcoxon tests with the same corrections. If you have unpaired (factorial, or non-repeated-measure) data, you can do pairwise comparison with a Mann-Whitney test. First, take a look at how to do pairwise comparison with a Mann-Whitney test with Bonferroni correction.

> pairwise.wilcox.test(Value, Group, p.adj="bonferroni", exact=F)

		Pairwise comparisons using Wilcoxon rank sum test 

data:  Value and Group 

  1      2     
2 0.0418 -     
3 0.0058 0.0791

P value adjustment method: bonferroni 

If you have any tied value in your data, you can only calculate an approximate p value. (You can try to calculate the exact p value by changing the option to "exact=T", but will get warnings if you have ties.) Unfortunately, it seems there is no way to get this around, but in many cases, this approximate p value is close enough to the exact p value, and will not cause a lot of troubles. But if you really want to be very precise, you can calculate the exact p value by using functions in coin package, and manually apply Bonferroni corrections to the p values you have gained by using p.adjust() function. For doing this, see Mann-Whitney page or Wilcoxon page.

Instead of Bonferroni correction, you can also use Holm correction.

> pairwise.wilcox.test(Value, Group, p.adj="holm", exact=F)

		Pairwise comparisons using Wilcoxon rank sum test 

data:  Value and Group 

  1      2     
2 0.0279 -     
3 0.0058 0.0279

P value adjustment method: holm 

If you have a within-factor (or repeated-measure data), you need to specify paired so that you will use a Wilcoxon test.

> pairwise.wilcox.test(Value, Group, p.adj="bonferroni", exact=F, paired=T)

		Pairwise comparisons using Wilcoxon rank sum test 

data:  Value and Group 

  1     2    
2 0.039 -    
3 0.026 0.174

P value adjustment method: bonferroni 

For calculating the effect size, you have to do a Mann-Whitney or Wilcoxon test to find the Z value (which is necessary for the calculation of the effect size). In this example, we are going to use a Mann-Whitney test. If you need to use a Wilcoxon test (i.e., you have a within-subject factor), see this page.

> library(coin)
> wilcox_test(Value ~ factor(Group), data=data[data$Group==1|data$Group==2,], distribution="exact")

		Exact Wilcoxon Mann-Whitney Rank Sum Test

data:  Value by factor(Group) (1, 2) 
Z = -2.4975, p-value = 0.01177
alternative hypothesis: true mu is not equal to 0 

Thus, the effect size is

> 2.4975 / sqrt(20)


Remember that you have to use the total sample size (10 * 2 = 20) to calculate the effect size r.

How to report

You can report the results of Kruskal Wallis or Friedman test as follows: A Kruskal Wallis test revealed a significant effect of Group on Value ((2)=13.7, p < 0.01). A post-hoc test using Mann-Whitney tests with Bonferroni correction showed the significant differences between Group A and B (p < 0.05, r = 0.56) and between Group A and C (p < 0.01, r = 0.70).
Could you maybe add the date you uploaded this information, then I can use this as a reference in my paper!
— Anonymous (2013-03-16 03:52:04)
I am glad to know that you are willing to cite my website. But I don't really recommend anyone citing a website (not just mine) because it is not archival in nature.

Kruskal-Wallis and Friedman tests are well explained in many of entry-level non-parametric statistical books. Just grab one of them in a library, and make sure that the content here is not wrong, and cite that book for your paper!
— KojiYatani (2013-03-18 18:40:40)
I would like to, but I'm using information on this website about the post hoc tests after investigated that the null hypethesis of the Friedman and Kruskal-Wallis test can be rejected and to find where exactly the differences can be found. I haven't found any books/official articles about post hoc tests, that's why I needed this website as a reference.
— Anonymous (2013-03-24 01:15:49)
Valid XHTML 1.0 Transitional :: Valid CSS :: Powered by WikkaWiki