User Tools

Site Tools


Chi-square, Fisher's exact, and McNemar's test

Chi-square test

A Chi-square test is a common test for nominal (categorical) data. One application of a Chi-square test is a test for independence. In this case, the null hypothesis is that the occurrence of the outcomes for the two groups is equal. For example, you have two user groups (e.g., male and female, or young and elderly). And you have nominal data for each group, for example, whether they use mobile devices or which OS they use. So, your data look like this. If your data of the two groups came from the same participants (i.e., the data were paired), you should use the McNemar's test.

Own device ADon't own device A

And now you are interested in figuring out whether the outcomes for the two groups were statistically equal. The assumption of Chi-square is that the samples are taken independently or are unpaired. If not, you need to use McNemar's test. And if you have only a small sample size, you should use the Fisher's exact test.

Effect size

The effect size of a Chi-square test can be described by phi or Cramer's V. If your data table is 2 x 2, you will calculate phi (k=2 in the equation below) and otherwise, Cramer's V (k>2 in the equation below) . But the calculation is pretty much the same and it is as follows:


where N is the total number of the samples, and k is the number of the rows or columns, whichever smaller, in your data table. And the chi-squared here is the value without any correction. Here are values which are considered small, medium and large sizes.

small sizemedium sizelarge size
Cramer's phi or V0.100.300.50

R code example

Let's use the examples above. First, prepare the data.

data <- matrix(c(25, 5, 15, 15), ncol=2, byrow=T) data2 <- matrix(c(16, 11, 3, 21, 8, 1), ncol=2, byrow=T)

And run a Chi-squared test.

chisq.test(data) Pearson's Chi-squared test with Yates' continuity correction data: data X-squared = 6.075, df = 1, p-value = 0.01371 chisq.test(data2) Pearson's Chi-squared test data: data2 X-squared = 2.1494, df = 2, p-value = 0.3414

So, the first example has a significant difference, which means the ownership of device A significantly differs between male and female users. The effect size of the first test can be calculated with vcd package:

library(vcd) assocstats(data) X^2 df P(> X^2) Likelihood Ratio 7.7592 1 0.0053440 Pearson 7.5000 1 0.0061699 Phi-Coefficient : 0.354 Contingency Coeff.: 0.333 Cramer's V : 0.354

For a 2×2 table, you can also calculate the odds ratio. The odds ratio is how the probability of the phenomena is affected by the dependent variable. This can be calculated as ad / bc.

Own device ADon't own device A
Malea = 25b =5
Femalec = 15d = 15
(25 * 15) / (5 * 15) 5

How to report

You can report the results of a Chi-square test like this:

Our Chi-square test with Yates' continuity correction revealed that the percentage of the ownership of device A significantly differed by gender ((1, N = 60) = 6.08, p < 0.01, = 0.35, the odds ratio is 5.0).

Fisher's exact test

You can instead use Fisher's exact test if your sample size is small. It is hard to say how many samples are small, but in general, it is better to use a Fisher's exact test than a Chi-square test when you have small than 10 in any cell of your data table (like the examples above).

R code example

Running a Fisher's exact test is pretty similar to Chi-square.

fisher.test(data) Fisher's Exact Test for Count Data data: data p-value = 0.0127 alternative hypothesis: true odds ratio is not equal to 1 95 percent confidence interval: 1.335859 20.757326 sample estimates: odds ratio 4.859427

How to report

How to report the results of a Fisher's exact test is pretty much the same as the Chi-square test. Unlike Chi-square test, you don't have any statistics like chi-squared. So, you just need to report the p value. Some people include the odd ratio with the confidence intervals.

McNemar's test

McNemar's test is basically a paired version of Chi-square test. Let's say you asked whether the participants liked the device before and after the experiment.

After experiment
Yes No
Before experimentYes62

Here, what you want to test is whether the number of the participants who liked the device were significantly changed between before and after the experiment.

Effect size

The effect size of the Fisher's exact test can be calculated in the same way as the one for the Chi-square test.

R code example

Running a McNemar's exact test is pretty similar to Chi-square.

data <- matrix(c(6, 2, 8, 4), ncol=2, byrow=T) mcnemar.test(data) McNemar's Chi-squared test with continuity correction data: data McNemar's chi-squared = 2.5, df = 1, p-value = 0.1138

Thus, we cannot reject the null hypothesis, and it means that the number of the participants who liked the device were not significantly changed between before and after the experiment. As you can see here, mcnemar.test() automatically makes correction for continuity. You can disable it with correct=F option, and the results will become the same with the function for Cochran Cochran's Q test.

McNemar's test and binomial test

In SPSS, the binomial distribution is used for McNemar's test. Thus, the results look different from those you can get in R. A binomial test is very similar to McNemar's test, but its null hypothesis is that the ratio of the two categories is equal to an expected distribution. In most cases, a binomial test is used for testing whether two categories are equally likely to occur.

Question 2 (post-treatment)
Yes No
Question 1 Yes a b
(pre-treatment) No c d

More precisely, you need to use a binomial test rather than McNemar's test if b+ c in the 2×2 table is small. However, in R, you can run McNemar's test with continuity correction, so it will cause a big problem because the results of a binmoal test and McNemar's test with continuity correction become similar.

If you want to do a binomial test like SPSS does, you need to use binomial function. And you need two numbers, which is the total count for the cases where the participants flipped the responses (i.e., b+ c. In the example we are using, 2 + 8 = 10), and the number of one of these two cases (i.e., 2 or 8).

After experiment
Yes No
Before experiment Yes 6 2
No 8 4
binom.test(2, 10, 0.5) Exact binomial test data: 2 and 10 number of successes = 2, number of trials = 10, p-value = 0.1094 alternative hypothesis: true probability of success is not equal to 0.5 95 percent confidence interval: 0.02521073 0.55609546 sample estimates: probability of success 0.2

In this case, the p value is pretty close regardless of the ways to do a McNemar's test.

How to report

How to report the results of a McNemar's test is pretty much the same as the Chi-square test.


Guest, 2015/02/16 02:13
Wow this really help thank you so much
Guest, 2015/03/07 00:10
# "chisq.test(data2)" gives a different results from the one shown above, with a warning message.

> data2 <- matrix(c(16, 11, 3, 21, 8, 1), ncol=2, byrow=T)

> chisq.test(data2)

Pearson's Chi-squared test

data: data2

X-squared = 19.465, df = 2, p-value = 5.932e-05

Warning message:

In chisq.test(data2) : 카이제곱 approximation은 정확하지 않을수도 있습니다 (roughly, chi-square approximation may not be correct.)
Guest, 2015/03/07 00:18
# continuation from the above
# I wonder if the following result of Fisher's exact test for 'data2' indicates it is significant
# unlike the chi-square test for the same data shown above.

> fisher.test(data2)

Fisher's Exact Test for Count Data

data: data2
p-value = 2.335e-05
alternative hypothesis: two.sided
Guest, 2015/04/09 17:53
Am I wrong to think that the number of columns assigned to data2 be 3, instead of 2?

data2 <- matrix(c(16, 11, 3, 21, 8, 1), ncol=3, byrow=T)
Guest, 2016/01/30 23:06
How P-value is computed, please
sildenafil viagra, 2016/09/14 18:21
Wow because this is great job! Congrats and keep it up. weferhervvbt
ASikebrecrealiBN, 2016/09/19 19:44
Yaboo! Don't worry about them, though: they will come out in the next process. Try to control your breathing, engage in foreplay, kissing, feeling your body and then delivered to the sex partner. One of the main causes will be premature ejaculation.
Godeagow, 2016/12/11 21:20
<a href=""></a>
[url -
"" - -

kefeMest, 2017/01/22 09:07
<a href=""></a> <a href=></a> <a href=></a>
Curtisgype, 2017/01/27 00:25

<a href=>Converse Noir Strass</a>
<a href=>Asics Cumulus 15</a>
<a href=>Nike Roshe One Ld 1000</a>
<a href=>Nike Roshe One Nere Zalando</a>
<a href=>Air Jordan 12 Retro The Master</a>
Thomassaw, 2017/01/28 14:51

<a href=>Adidas Gazelle Og Rouge Pas Cher</a>
<a href=>Saucony Shadow Nere E Bianche</a>
<a href=>Nike Air Huarache Noir Daim</a>
<a href=>Adidas Stan Smith Femme Or</a>
<a href=>Vans Grigie E Viola</a>
perfictgirls, 2017/01/31 05:43
egg insertion porngranniesluts;lang=en-jovencita-argentina-sabrosa-cojiendo
I mean, I'd told him I intended to marry Jen, so he knew she and I would fill a pile to stammer about.Of course, it was nothing and I select no offense, but I must say you seem to be highly well wordy for a enslaved.We are successful in that my parents were rich and left us a smallish fortune, so I prefer not sustain to work/ He is eighteen years senior and has always a apex or 2 about my sundress, jewellery, and beget-up..
mydirtyhobby, 2017/02/02 17:17
mi vesina desnudaoops i cumed
We hadn't needed a clubhouse for several years - we sensed we had outgrown the need for one - but I guess the privacy it frail to suggest would collected comply us well.She very likely did not even know.The next time queue moved, I permitted my gams to fumble her thighs a bit..
Richardmiff, 2017/02/16 10:53

<a href=>Ray Ban Wayfarer Sunglasses</a>
<a href=>Converse Trainers Sale Amazon</a>
<a href=>Nike Air Max 95 White And Grey</a>
<a href=>Max Nike Shoes</a>
<a href=>Nike Air Force 1 White Gum</a>
DouglasGYPE, 2017/02/17 03:30

<a href=>Air Max 95</a>
<a href=>Nike Air Force 1 High Outfit Black</a>
<a href=>Nike Air Shox Womens</a>
<a href=>Nike Trainers Womens Uk Sale</a>
<a href=>Nike Air Jordan 1 Grey And Black</a>
Qddieknill, 2017/02/17 16:22
Post by : July 07,2014 from : fabuleux louis vuitton lv277 working chronograph white ceramic case silver dial lady size montre lv 076268. Replique Montre Suisse ... <a href="">Pas Cher Louis Vuitton</a> Sac Louis Vuitton pas cher wql mfd <a href=>Sac Louis Vuitton pas cher</a>echarpe louis vuitton 2014 homme bonne qualite coton pas cher 213 noir,Echapres Louis Vuitton femmes Nouveau,Echapres Louis Vuitton Pas Cher art xon
Richardmiff, 2017/02/18 03:53

<a href=>Zapatos Mbt Sevilla Precios</a>
<a href=>Adidas Superstar Black</a>
<a href=>Reebok Blanco Con Azul</a>
<a href=>Nike Air Force 1 Mid White</a>
<a href=>Vans Toy Story Woody Comprar</a>
Richardmiff, 2017/02/19 21:55

<a href=>Adidas Yeezy Boost 350 Buy Online</a>
<a href=>Adidas Nmd Black Camo</a>
<a href=>Nike Air Max 90 Ultra Br Mens Shoe</a>
<a href=>Roshe Run Women Outfit Shorts</a>
<a href=>Nike Free Rn Flyknit With Jeans</a>
Richardmiff, 2017/02/20 08:59

<a href=>Adidas Stan Smith Aanbieding</a>
<a href=>New Balance Schoenen Hardlopen</a>
<a href=>Nike Free Damen Schwarz Grau</a>
<a href=>Saucony Grid 8000</a>
<a href=>Adidas Ultra Boost</a>
Curtisgype, 2017/02/23 12:36

<a href=>Nike Blazer Low Black</a>
<a href=>Zapatillas Adidas Energy Boost Mujer</a>
<a href=>Puma Creepers Fenty Rihanna</a>
<a href=ênis-timberland-gorge-c2-vermelho-440.php>Tênis Timberland Gorge C2 Vermelho</a>
<a href=>Nike Air Max 2017</a>
aMdFYRCdcB, 2017/03/03 05:29
<a href=>buy ambien</a> can you mix ambien and lunesta <a href=>buy ambien online</a> ambien and bilirubin <a href=>buy zolpidem</a> purchase zolpidem Las Vegas <a href=>buy ambien</a> ambien and gambling <a href=>buy ambien</a> can u mix trazodone and ambien <a href=>ambien 10 mg</a> ambien 10 mg orange <a href=>ambien generic</a> can you take ambien with soma <a href=>ambien for sale online</a> ambien online Sacramento <a href=>ambien generic</a> can you take ambien with az pack <a href=>ambien pills</a> buy ambien online Norfolk <a href=>buy generic tramadol no prescription</a> can you take tramadol and lisinopril <a href=>tramadol online</a> cloridrato de tramadol Г© morfina <a href=>order tramadol online</a> bruk av tramadol
Richardmiff, 2017/03/03 05:45

<a href=>Nike Kobe Shoes</a>
<a href=>Nike Basketball Shoes Ebay</a>
<a href=>New Balance Mt910gx2</a>
<a href=>Saucony Buty Do Biegania</a>
<a href=>Adidas Energy Boost Mujer</a>
qQmoTRTOcv, 2017/03/03 20:19
<a href=>buy ambien</a> ambien leg weakness <a href=>buy ambien online</a> ambien hormone therapy <a href=>generic ambien online</a> sleep aid better than ambien <a href=>buy ambien</a> can i take an ambien while pregnant <a href=>buy ambien</a> ambien side effect stories <a href=>buy ambien online</a> will ambien show up on drug test <a href=>buy ambien online</a> does lunesta work the same as ambien <a href=>buy zolpidem</a> can you mix pain pills and ambien <a href=>ambien no prescription</a> ambien and cold medication <a href=>buy ambien online</a> how long does it take for ambien to hit you <a href=>buy tramadolВ rx</a> tramadol vs nucynta <a href=>tramadol 100</a> tramadol worth <a href=>buy tramadolВ cod next day delivery</a> tramadol 50 mg stronger than vicodin
bHozBGtlAc, 2017/03/04 16:28
<a href=>order ambien</a> zolpidem buy Las Vegas <a href=>generic ambien buy</a> get ambien Eugene <a href=>ambien 10mg</a> buy zolpidem Arlington <a href=>buy ambien</a> man kills wife while on ambien <a href=>buy ambien</a> benzo withdrawal ambien <a href=>buying ambien online</a> ambien cr generic pictures <a href=>buy ambien online</a> order zolpidem Tacoma <a href=>buy ambien online</a> side effects of ambien for the elderly <a href=>buy ambien online</a> overdose of ambien symptoms <a href=>buy ambien online</a> can order ambien online <a href=>tramadol online</a> anyone taken tramadol during pregnancy <a href=>tramadol 50mg</a> tramadol/apap 37.5mg/325mg tabs <a href=>buy tramadolВ cheap</a> bijwerkingen tramadol honden
Guest, 2017/03/05 05:14;_ylu=X3oDMTByaWg0YW05BGNvbG8DYmYxBHBvcwM4BHZ0aWQDBHNlYwNzcg--/RV=2/RE=1483844986/RO=10/RU=
pNyPUNGtoc, 2017/03/05 06:37
<a href=>buy ambien</a> half life ambien 5mg <a href=>ambien for sale online</a> can you take 2 ambien at once <a href=>ambien cr</a> purchase ambien Rhode Island <a href=>generic zolpidem</a> ambien dopamine <a href=>buy ambien</a> how much does ambien cost on street <a href=>buy ambien online</a> skyrim ambient occlusion nvidia control panel <a href=>generic ambien</a> drug interactions phentermine ambien <a href=>ambien for sale no prescription</a> order zolpidem Chesapeake <a href=>ambien generic</a> ambien makes me see things <a href=>buy ambien online</a> what to do if you take too much ambien <a href=>buy tramadol online</a> apakah tramadol narkoba <a href=>buy tramadol online</a> wanneer tramadol innemen <a href=>buy tramadol</a> tramadol testicle pain
dNCABweaKR, 2017/03/05 23:10
<a href=>ambien online no prescription</a> ambien 1 mg <a href=>buy ambien online</a> is doxepin like ambien <a href=>ambien sleep medication</a> sleeping pill better than ambien <a href=>ambien medication</a> how many ambien does it take to trip <a href=>generic ambien</a> ambien spanish <a href=>discount ambien</a> get zolpidem Maine <a href=>discount ambien</a> danger with ambien <a href=>buy ambien online</a> what dosage does ambien come in <a href=>buy ambien online</a> can you take ambien with imitrex <a href=>buy ambien online</a> zolpidem online Cedar Rapids <a href=>tramadol no prescription</a> tramadol for subutex withdrawal <a href=>buy tramadol</a> does the drug tramadol cause weight gain <a href=>tramadol generic</a> make tramadol stronger
RxBojIkdfU, 2017/03/06 13:11
<a href=>order ambien online no prescription</a> can ambien be taken with prozac <a href=>buy ambien online</a> buy ambien Akron <a href=>buy generic ambien</a> ambien video games <a href=>order ambien</a> can i take a xanax and ambien together <a href=>zolpidem without prescription</a> how to deal with ambien withdrawal <a href=>ambien generic</a> can you take antacids with ambien <a href=>ambien generic</a> muscle cramps ambien <a href=>buy ambien online</a> does ambien cr show up drug test <a href=>buy ambien online</a> how do i get an ambien prescription <a href=>ambien without a rx</a> zolpidem buy Coral Springs <a href=>order tramadol</a> oxycontin and tramadol mixing <a href=>tramadol no prescription</a> adolonta capsulas tramadol <a href=>generic tramadol</a> tramadol 24 hours after adderall
LofhXsFGuR, 2017/03/07 15:46
<a href=>ambien 10mg</a> fda recall ambien <a href=>generic ambien</a> purchase ambien Tennessee <a href=>buy ambien</a> ambien dosage levels <a href=>ambien 10mg</a> ambien face cream <a href=>ambien price</a> ambien makes me feel better <a href=>buy ambien online</a> can ambien cause breast pain <a href=>ambien no prescription</a> ibuprofen 800 and ambien <a href=>buy ambien online</a> ambien cr and sleep apnea <a href=>ambien sale</a> does gabapentin interact with ambien <a href=>ambien no rx</a> taking ambien and staying awake <a href=>buy tramadolВ 100mg</a> tramadol for chronic pain in dogs <a href=>buy tramadol online</a> purchase tramadol Frisco <a href=>order tramadol online</a> does tramadol work for toothaches
WLSTHyteYf, 2017/03/08 23:35
<a href=>ambien 10mg</a> can you take ambien and diazepam <a href=>buy ambien online</a> how long before sleep take ambien <a href=>buy ambien</a> how do you trip on ambien <a href=>ambien without prescriptions</a> how many milligrams does it take to overdose on ambien <a href=>order ambien</a> get ambien Birmingham <a href=>buy ambien online</a> can i mix ambien and klonopin <a href=>buy ambien online</a> which is worse ambien or xanax <a href=>ambien sleep medication</a> fun things to do when your on ambien <a href=>zolpidem 10 mg</a> is it legal to order ambien online <a href=>order ambien online no prescription</a> how long does ambien stay in the body <a href=>buy tramadol onlineВ no prescription</a> calmador tramadol dosis <a href=>buy tramadol online</a> best site buy tramadol <a href=>order tramadol</a> methamphetamine and tramadol
LvrTjPclyo, 2017/03/09 11:55
<a href=>zolpidem 10 mg</a> cara mengobati penyakit ambien/wasir <a href=>buy ambien online</a> anything over the counter like ambien <a href=>buy ambien</a> can i mix melatonin with ambien <a href=>buy ambien online</a> ambien making me hyper <a href=>buy ambien</a> ambien and life expectancy <a href=>buy ambien online</a> is ambien bad to take every night <a href=>buy generic ambien</a> does ambien cause ringing in ears <a href=>buy ambien online</a> ambien withdrawal symptoms side effects <a href=>buy ambien online</a> benadryl ambien interaction <a href=>generic ambien</a> ambien pill white <a href=>buy tramadol online</a> ultram tramadol for depression <a href=>tramadol no prescription</a> side effects of stopping tramadol suddenly <a href=>buy tramadol online</a> chewing tramadol hcl
YbDbRIPkXR, 2017/03/10 00:35
<a href=>buying ambien online</a> is it ok to take ambien and advil pm <a href=>buy ambien online</a> what does an ambien cr look like <a href=>zolpidem 10 mg</a> zolpidem buy Visalia <a href=>ambien generic</a> ambien joint pain <a href=>buy ambien</a> sulfamethoxazole ambien <a href=>buy ambien online</a> ambien generic 10mg <a href=>ambien drug</a> can i take ambien twice a night <a href=>ambien sleep</a> purchase ambien Laredo <a href=>zolpidem online</a> sleeping medications similar to ambien <a href=>buy ambien</a> does ambien build up in your system <a href=>buy tramadol online</a> tramadol aine <a href=>buy tramadolВ tablets</a> posologie maximale tramadol <a href=>buy tramadol</a> how many tramadol get you high
oAPFmASCeE, 2017/03/10 15:37
<a href=>buy ambien</a> zolpidem Huntington Beach <a href=>zolpidem online</a> how many ambien will cause overdose <a href=>buy zolpidem</a> can you take adderall in the morning and ambien at night <a href=>ambien online</a> can you mix ambien and muscle relaxers <a href=>ambien generic</a> ambien buy Hollywood <a href=>ambien online no prescription</a> is it bad to mix suboxone and ambien <a href=>buy ambien online</a> is ambien gluten free <a href=>zolpidem drug</a> ambien out of body experiences <a href=>buy ambien online</a> buy zolpidem Fargo <a href=>ambien cr</a> what is the high from ambien like <a href=>buy tramadol online</a> medicamentos con ketorolaco y tramadol <a href=>order tramadol online</a> tramadol+paracetamol brand names <a href=>cheap tramadol</a> tramadol side effects in dogs uk
Curtisgype, 2017/03/13 07:51

<a href=>Puma Shoes White And Red</a>
<a href=>Scarpe Puma Immagini</a>
<a href=>Puma Sneakers Ribbon</a>
<a href=>Puma Basket Classic White Damen</a>
<a href=>Puma Lila</a>
Please enter your comment. You cannot remove your comments by yourself. So double-check before you submit.:
If you can't read the letters on the image, download this .wav file to get them read to you.
hcistats/chisquare.txt · Last modified: 2014/08/14 05:22 by Koji Yatani

Page Tools