User Tools

Site Tools


Power Analysis


NHST can tell you how likely randomly sampled data would be like your data or even more extreme than it given that the null hypothesis (e.g., there is no difference in the mean across the groups to compare) is true. As a standard threshold, we use 0.05, and we call this alpha. This means that if randomly sampled data can be like your data at lower than 5% change, you reject the null hypothesis (and claim that we observe a difference). Thus, if your p value is lower than alpha, we say that you have a significant result.

If a difference does not exist, but we reject the null hypothesis, it will be an error. We call this the Type I error or false positive. As you have seen in various NHSTs, your p value will never be zero (can be really really small). And your p value directly means the probability of this Type I error. The threshold alpha = 0.05 means that we simply determine that if the chance that we have a Type I error is less than 5%, we can kinda ignore it because it is unlikely to happen.

However, there is another kind of errors we may have in NHSTs as illustrated in the following table.

Reject the null hypothesisFail to reject the null hypothesis
The null hypothesis is true.Type I error (false positive)True negative
The null hypothesis is false.True positiveType II error (false negative)

Another possible error is that we fail to reject the null hypothesis although the null hypothesis is actually false (or we conclude that we do not have a difference although there is a difference). This is called a Type II error or false negative. Its probability is represented as beta. Thus, 1 - beta is the probably to have true negative, and it is called power. Power analysis is a way to how likely we have a Type II error.

To summarize in a bit more mathematical notation, alpha = p(Reject H0 | H0 is true). H0 is the null hypothesis. And p = p(Data | H0 is true). If p is smaller than alpha, we call the result significant. beta = p(fail to reject H0 | H0 is false), and power = p(reject H0 | H0 is false) = 1 - beta.

There are several pieces of information we need for power analysis.

  • Sample size,
  • Effect size (the one of the underlying population and not the observed one),
  • Alpha, and
  • Beta or Power.

We just need three of the above information. Power analysis then estimates the rest. Based on what you estimate, power analysis often has different names.

  • A priori power analysis: Estimating the sample size given the effect size, alpha and power,
  • Sensitivity analysis: Estimating the effect size given the sample size, alpha and power.
  • Criterion analysis: Estimating the alpha given the sample size, effect size, and power.
  • Post hoc power analysis: Estimating the power given the sample size, effect size, alpha.

Here, we focus on a priori power analysis because I think it is the most important application of power analysis.

Sample size estimation

Probably, the most important way to use power analysis is to estimate a sample size to get the desired alpha and power. As explained in the section above, as long as you have three parameters, you can calculate the rest parameter. Particularly, estimating the effect size can be useful when you design an experiment. Let's take a look at an example with t tests.

Let's say that we have two groups to compare and we do a within-subject design (so we are using a t paired test). We want to have a difference with alpha=0.05. It is kinda hard to set a proper effect size, but let's say 0.5 (regarded as a middle-size effect). The power is also kinda tricky to set. Here, we shall accept 30% false negative in cases where the null hypothesis fails to be rejected. So, the power is 0.7.

library(pwr) pwr.t.test(power=0.7, d=0.5, sig.level=0.05, type="paired") Paired t test power calculation n = 26.66474 d = 0.5 sig.level = 0.05 power = 0.7 alternative = two.sided NOTE: n is number of *pairs*

Thus, we need 27 samples for each group to meet the criteria we specified.

For an ANOVA test, we can use the pwr.anova.test() function. The parameters are similar except that k for the number of groups to compare, and f for the effect size. Usually this effect size is reported as Cohen's f^2 instead of f, but for power analysis, we need f. And not that this does not equal to the eta-squared or partial eta-squared. So be careful about it. For example, if we have three groups to compare, we assume that the effect size of the underlying populations is 1.0, we set the power to 0.7, we can estimate the sample size we need to get as follows.

library(pwr) pwr.anova.test(k=3, f=1.0, sig.level=0.05, power=0.7)

Another thing I should mention is retrospective power analysis. It means that we are trying to estimate the power (or the observed power) based on the observed effect size. This looks similar to post hoc power analysis, but the difference is in the effect size: retrospective power analysis uses the observed power, and post hoc power analysis uses the population effect size determined in a priori manner. This retrospective power analysis is often considered to be useful to determine if there is really no difference between the groups we are comparing.

However, retrospective power analysis is regarded as an inappropriate analysis for determining no existence of differences between the groups, and many researchers have suggested avoiding this type of analysis. A more detailed discussion is available in the following paper: Hoenig, J. N., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. American Statistician, 55, 19-24., so if you are interested, I would recommend you to read their paper.

Here is a quick summary of why retrospective power analysis is not appropriate. Power analysis assumes the effect size of the underlying population. It is known that the observed effect size (the one calculated from your data) generally is not a good approximation of the true effect size (the one we want to use for power analysis).

When you want to claim that there is no difference, you want to have a case in which your p value is high (so it is unlikely that you say there is a difference although there isn't) and your power is also high (so it is unlikely that you say there is not a difference although there is). However, the p value and the observed power (the power calculated based on the observed effect size) are correlated to each other: when the p value becomes large, the power tends to become low. Thus, you cannot really expect what you want to get from power analysis for claiming no difference.

So if you see this type of power analysis, be very careful.


QbzLL, 2021/01/11 03:32
Medication information leaflet. Generic Name. <a href="">can you get prednisone no prescription</a> in USA. Some about drug. Get here.
<a href=>Everything trends of medicament.</a> <a href=>Some information about medication.</a> <a href=>Actual about drug.</a> 9646e37
Holder, 2022/11/02 17:25
Janes, 2022/11/09 21:19
Rodolfo, 2022/11/13 20:43
Nicholas, 2022/11/16 00:27
Cody, 2022/11/27 18:39
Darleen, 2022/12/28 03:56
Anthony, 2023/01/30 07:18
<a target="_blank" href="">replica purses</a>, 2023/03/08 05:47
offer excellent fashion Replica Shoes Sunglasses, Clothes, Glasses Frames, Watches, Jewelry, Scarves, Belts Caps, Sneakers,Accessories, selection, and service on one website. offer the customer the best possible designer fashions for the least amount of money Shopping has never been so easy and hassle free
Rodolfo, 2023/03/16 10:14
YEEZY, 2023/04/06 03:20 Yeezy Adidas UK Air Jordan 4 Air Jordan 1 Jordan 1 Air Jordans Nike Air Jordans Jordans 1 Air Jordan 1 Nike Jordan 1 Nike Jordan Jordan 1s Jordan 1 Adidas Yeezy Adidas Yeezy Yeezy Adidas Yeezy Adidas Yeezy Yeezy Shoes Air Jordan 1 Air Jordan 1 Jordan 1 Jordan 1 Air Jordan 4 Air Jordans Air Jordan 1 Jordan 1 Air Jordan Shoes Air Jordan Air Jordans Air Jordan 1 Jordan 1 Jordan 1 Jordan 1 Jordan 1 Jordan 11 Jordan 1s Air Jordan 4 Jordan 4 Air Jordan 4 Jordan 4 Air Jordan 4 Jordan 4 Jordan Jordan 4 Military Black Jordan 4 Jordan 4s Jordan Retro 4 Jordan 11 Jordans 4 Jordan Shoes Jordans Shoes NFL Shop Official Online Store Nike Air Jordan Nike Air VaporMax Nike Jordan 1 Nike Jordans Nike Outlet Nike Outlet Store Nike Outlet Nike Outlet Store Online Shopping Nike Outlet Nike Shoes Nike UK Off White Yeezy Yeezy 350 Yeezy Yeezy 350 Yeezy Boost 350 Yeezy 350 V2 Yeezy 450 Yeezy 700 Adidas yeezy Yeezy Foam Runner Yeezy Yeezy Yeezy Shoes Yeezy Yeezy Slides Yeezy Slides Yeezy Supply Jordan Shoes Jordans Yeezy Slides Nike Shoes Nike Outlet Nike Outlet Ray-Ban Sunglasses Yeezy Shoes Moncler Jacket Moncler Outlet Moncler Outlet Moncler Jacket Moncler Moncler UK Moncler Jacket Moncler Moncler Outlet Yeezy Yeezy Shoes Yeezy Sneakers Yeezy Yeezy Shoes Yeezy 350 Pandora Jewelry Pandora Yeezy Yeezy Shoes Yeezy Sneakers Yeezy Yeezy Shoes Yeezy 350 Pandora Jewelry Pandora Pandora Jewelry Yeezy Ray Bans Ray Bans Sunglasses Ray Ban Outlet Pandora Jewelry Yeezy Slides Adidas Yeezy Yeezy Shoes Yeezy 350 Yeezy Yeezy 350
bestrvguide, 2023/05/31 08:27
From 4K UHD to OLED, we have a Smart TV model that will meet your needs and exceed your expectations.
jiuer7845, 2023/07/22 03:01 Yeezy Yeezys Jordan 1 Air Jordan 1 Nike Jordan 1 Jordan 1S Jordan 1 Jordan Shoes Jordan Shoes Nike UK Yeezy 450 Jordan Retro 4 Nike Outlet Store Online Shopping YEEZY Adidas Yeezy Official Website Yeezy 350 Air Jordan 4 Yeezy Foam Runner Yeezy Slides Ray Ban Glasses Adidas UK NFL Shop Official Online Store Yeezy Slides Adidas Yeezy Yeezy 350 Yeezy Yeezy 350 Yeezy Shoes Yeezys Off White Ray Ban Adidas Yeezy Yeezy Shoes Yeezy Shoes Yeezy 700 Adidas Yeezy Yeezy Nike Jordan 1 Jordan 1 Jordan 1 Jordan 1 Jordan UK Jordans Shoes Jordan Nike Yeezy Jordan 4 Nike Outlet YEEZY UK Adidas Yeezy Yeezy uk Jordan 4 Yeezys Supply Yeezy YEEZY Ray Bans Sunglasses Adidas NFL Shop Yeezys Adidas Yeezys Yeezy 350 V2 Yeezys Yeezy Boost 350 Yeezys Shoes Yeezy Ray Bans Yeezy Yeezys Adidas Yeezy Slides Yeezy Yeezy Cheap Yeezys Yeezy Supply Yeezy Shoes Yeezy Supply Yeezys Yeezy Shoes Yeezy Yeezy Pandora Jewelry

Tags:yeezy shoes, Yeezy, Adidas Yeezy
KevinVes, 2023/08/28 23:25
cialis tablets reviews <a href="">cialis going generic</a> commander cialis en ligne
Enter your comment:
If you can't read the letters on the image, download this .wav file to get them read to you.
hcistats/poweranalysis.txt · Last modified: 2014/03/29 02:20 by Koji Yatani

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki