Table of Contents
Generalized Linear Model (GLM)
Generalized Linear Model (GLM) is a more generic framework that supports statistical analysis with some sort of linear formation of a model. You can think that GLM is like an upper class of linear regression and logistic regression. These regressions are special cases of GLM. In GLM, your model should look like below.
where f is an function. In other words,
where is the inverse function of f. What this formula means: we first do linear calculation with the coefficients ( and independent variables (x). Then, we apply transformation to it by using the function , and the result is the predicted dependent variable. So, if , it is linear regression. If , it is logistic regression. By the way, is called the inverse-logit function, and the logit function is x / (1 - x). So far so good? :)
The nice thing about GLM is that we can handle other data distributions than the normal distribution by putting an appropriate function to as we do in logistic regression. With logistic regression, the data distribution of the dependent variable becomes the probability for binomial data. And there are some common data distributions and function used in GLM.
- Poisson model: The data distribution of the dependent variable is Poisson, and (this means that the transformation for the dependent variable is the exponential function). This model is used for count data which are non-negative integers.
- logistic-binomial model: This uses the inverse-logit function for the transformation () and uses the binomial distribution for the dependent variable. This model is used when the dependent variable represents the number of trials of interest out of the all trials (e.g., the count of the trials in which the participants successfully completed the tasks).
- Multinomial logit and probit model: This is an extended model of logistic and probit regressions for ordinal or categorical data with more than two options. This model uses the logit or probit function for transformation, and uses the multinomial distribution for the dependent variable.
There are more models used in GLM , but the the models above are probably useful for an analysis in the context of HCI research, so I am going to explain these models in this page.
One thing you should note that the methods explained in this page does not cover multilevel models (often referred as mixed-effect models). If I explain it with terms which are more familiar to HCI folks, you need to use slightly different procedures to accommodate repeated-measure factors. This will be explained in a separate page, but understanding GLM is definitely useful to understand multilevel models.
This model uses the Poisson distribution. It is well known that the Poisson distribution represents the probability of the number of events occurrence within a period of time. For instance, the number of car accidents in a city may follow the Poisson distribution.
R code example
Let's take a look at an example of the following hypothetical data. You have studied how many emails 20 smart phone users (Device A and Device B) sent to someone from their mobile devices during the period of the study. You have the average number of the emails per day for each participant, and the data are like this.
|Participant||Device||Unlimited Data Plan||Email counts|
In addition to the different devices, you also recorded whether each participant had an unlimited data plan, which you expected is likely to affect the number of emails sent from the mobile devices. So, now your hypothesis is that the number of emails sent from mobile devices can be influenced by the difference of the devices and whether the participants had an unlimited data plan. OK, it's time to do regression! First, prepare the data.
Please note that Device and DataPlan are treated as categorical data here. If your independent variable is continuous, you don't have to use factor(), of course. We the run Poisson regression.
So the deviance has been improved by about 6 points, which is not too bad. And the model we have is as follows.
where DeviceB = 1 if the participant used Device B, and otherwise 0, and DeviceY = 1 if the participant had an unlimited data plan, and otherwise 0. Now we look at the exponential part, which is
Let's take a deeper look at this model. First, we see the difference between Device A and Device B. If DeviceB = 1, u will have a exp(0.2187) = 1.24 times increment. So the difference of the devices may have an effect on encouraging people to send more emails; however, DeviceB is not statistically significant (p = 0.19), so we cannot strongly argue that there is such an effect, and we probably need more data.
How about the effect of the data plan? If DataPlanY = 1, u will have a exp(0.3923) = 1.48 times increment, which sounds something. Moreover, DataPlanY is statistically significant (p <0.05). Thus, we can think that an unlimited data plan has a positive effect on the number of emails sent from the mobile devices.
So, it seems that the data plan matters for the usage of emails on mobile devices, but there is a concern about this analysis, not in terms of the method itself, but in terms of the observation. Users who had unlimited data plans happened to use emails more often than users who don't have unlimited data plans. If this is the case, our analysis is not correct. If you have noticed this before, you are a very sharp reader :). Yes, it is a concern, and we have a solution for it. Let's say you also measure how many emails each participant sent from all types of devices they used (e.g., desktop machines, laptops, internet tablets, etc), which are shown as “Total email counts” in the table below.
|Participant||Device||Unlimited Data Plan||Email counts||Total email counts|
We use this “Total email count” as a baseline for each participant. This baseline is called offset in GLM. With the offset, the model becomes like this.
where j is the index of the data point (y_j means the dependent variable of j-th data point). This model allows us to have an adjustment (offset) for each data point. More intuitively, we are going to use the rate of the number of sent emails from the mobile devices over the total number of the sent emails. Thus, x_ji is the rate, and o_j is the total number of the sent emails. We can do this in the glm() function by using the offset option. But you have to calculate its logarithmic.
So, we still have a significant effect of the data plan.
It's not completely done even if you consider the offset. Another problem you may have is overdispersion. We have built a model with some (often very) strong assumptions: we only have two factors and the distribution of the dependent variable follows the Poisson distribution. Thus, the predicted values by our model might be good enough to describe the variance we observed in the data. If this is the case, the model is not appropriate enough to use. This is what we call overdispersion, and often happens in Poisson regression.
To see whether the model suffers from overdispersion, we can calculate the sum of square of the standardized residuals (i.e., values of the difference between the observed values and predicted values), and compare it to the chi-squared distribution.
Here is the procedure. You first need to calculate the predicted value.
Then, calculate the sum of square of z. If this is much larger than 1, it implies that you have overdispersion. We need two values: the sample size and the number of coefficients. In our case, they are 20 and 3, respectively.
We also can do a test with the chi-squared distribution.
If this value is below 0.05, the test indicates that you have the dispersion. In this example, we don't seem to have the dispersion, which is good news.
What if we have the overdispersion? We then need to adjust the standard error for each coefficient by multiplying them by the root of the sum of square of z. For example, supposed that our sum of square of z turned out 10 in our example. The coefficient for DataPlanY is 0.41366 with the standard error 0.17353. This means that the coefficient is likely [0.23983, 0.58689]. With the adjustment for the overdispersion, the standard error becomes 0.17353 * sqrt(10) = 0.54875. Thus, the adjusted coefficient is likely [-0.13509, 0.96241], which means that it can be zero (which means that the effect of DataPlanY can be zero). Fortunately, you don't have to do this manually, and you can re-fit the model by using quasipoisson as follows.
It also calculates the p value based on the adjustment for the overdispersion.
This is basically the logistic regression, but we are going to use the binomial distribution for the dependent variable. In this way, we can model the number of trials of interest out of the all trials, like a success rate or an error rate.
R code example
Let's say you have a game application in which the participants can hit monsters by a mouse click on them. In your experiment, you set different time limits (e.g., 10, 20, 30, 40, 50, and 60 sec), and measured the success rate (i.e., the number of the monsters the participants could clicked during the trial out of the pre-defiend number of the monsters). Now, what you want to do is to create a model of the success rate with the game time limit. Let's say your data look like this.
(Well, you will have much more data in reality, but I put 12 samples here to make the example simple.) Let's try to do logistic regression for this data. First, we prepare the data in R.
Note that we prepare both the numbers of “Success” and “Failure”, and make it into one variable, called “GameResult”. In this way, the logistic regression is going to use the binomial distribution. Now, we do the same thing with the glm() function.
You get the result.
You can do a similar analysis as described in the previous section. For instance, the model greatly improves the deviance, which indicates that this model is very powerful. Time is statistically significant on predicting the success rate.
This model also often suffers from the overdispersion, so you have to be careful about it. You can find more details on how to calculate the overdispersion in the previous section. You can use quasibinomial to adjust the model for overdispersion.
Poisson model or logistic-binomial model
The Poisson model is similar to the logistic-binomial model, particularly in a sense that both treat count data, and you may wonder how we should choose which model to use. Here is the rule of thumb.
- If the dependent variable does not have a upper limit, you should use the Poisson model.
- Otherwise, you should use the logistic-binomial model.
Multinomial logit model
The content in this page is largely what I learned from the following book. Data Analysis Using Regression and Multilevel/Hierarchical Models by Andrew Gelman and Jennifer Hill. It is the best book I have read about regression analysis (well, I haven't read that many books, but I'm sure this is an excellent book).
<a href=http://wellbutrin100.com/#>wellbutrin bupropion</a> - wellbutrin 150mg price
<a href=" https://lasixtop.com/# ">lasix side effects</a>
<a href=http://wellbutrin100.com/#>wellbutrin prescription cost</a> - wellbutrin xl 300mg
pet antibiotics without vet prescription
<a href=http://wellbutrin100.com/#>wellbutrin 150mg price</a> - purchase wellbutrin xl
buy suprax online http://worldrx100.com/ myambutol tablets
<a href=http://worldrx100.online/#>minocin tablets</a> biaxin price
buy myambutol online http://bio-catalyst.com/ cefadroxil capsules
<a href=http://bio-catalyst.com/#>buy vantin</a> floxin online
stromectol for sale http://worldrx100.online/ cefadroxil online
<a href=http://bio-catalyst.com/#>buy ceftin online</a> buy suprax
buy doxycycline online http://worldrx100.com/ buy floxin generic
<a href=http://bio-catalyst.com/#>cipro for sale</a> minocin capsules
buy biaxin http://bio-catalyst.com/ cefixime generic
<a href=http://worldrx100.online/#>fucidin online</a> buy minocycline online
order clindamycin http://worldrx100.online/ terramycin tablets
<a href=http://worldrx100.online/#>buy tetracycline</a> cefixime generic
panmycin tablets http://bio-catalyst.com/ myambutol price
<a href=http://bio-catalyst.com/#>order omnicef</a> biaxin online
fucidin price http://worldrx100.online/ buy bactrim online
<a href=http://worldrx100.com/#>panmycin generic</a> buy zyvox online
order vantin online http://bio-catalyst.com/ floxin capsules
<a href=https://allpillrx.com/#>augmentin price</a> buy suprax generic
minomycin online http://bio-catalyst.com/ cephalexin online
<a href=http://worldrx100.com/#>order ceftin online</a> buy ceftin
zithromax online http://bio-catalyst.com/ buy panmycin online
<a href=https://allpillrx.com/#>buy chloramphenicol</a> nitrofurantoin generic
buy clindamycin http://worldrx100.com/ buy vantin online
<a href=http://worldrx100.com/#>clindamycin online</a> erythromycin capsules
minocycline online http://worldrx100.online/ cefixime online
<a href=http://bio-catalyst.com/#>cefadroxil capsules</a> order tinidazole online
order minomycin online https://allpillrx.com/ buy ampicillin generic
<a href=http://worldrx100.com/#>terramycin tablets</a> chloramphenicol online
ceftin tablets https://allpillrx.com/ minocycline tablets
<a href=http://bio-catalyst.com/#>buy doxycycline generic</a> fucidin tablets
buy noroxin http://worldrx100.com/ buy amoxil
<a href=http://worldrx100.com/#>cleocin price</a> ketoconazole tablets
buy ketoconazole http://worldrx100.online/ generic ciplox
<a href=http://bio-catalyst.com/#>buy keftab online</a> buy cleocin online
order augmentin http://worldrx100.com/ floxin price
<a href=https://allpillrx.com/#>buy clindamycin generic</a> order bactrim
generic nitrofurantoin http://worldrx100.online/ generic biaxin
<a href=http://worldrx100.com/#>generic augmentin</a> order terramycin
ampicillin online https://allpillrx.com/ clindamycin generic
<a href=http://worldrx100.online/#>buy vantin online</a> doxycycline generic
buy nitrofurantoin generic http://bio-catalyst.com/ order cefixime online
<a href=https://allpillrx.com/#>order clindamycin online</a> suprax tablets
sumycin price http://bio-catalyst.com/ generic cleocin
<a href=http://worldrx100.com/#>suprax tablets</a> order zithromax
keflex generic http://worldrx100.online/ omnicef for sale
<a href=http://bio-catalyst.com/#>buy doxycycline generic</a> ampicillin for sale
ampicillin online http://bio-catalyst.com/ panmycin generic
<a href=http://bio-catalyst.com/#>buy trimox generic</a> generic chloramphenicol
buy fucidin http://worldrx100.online/ generic noroxin
<a href=https://allpillrx.com/#>order tetracycline</a> buy terramycin online
order tinidazole http://worldrx100.online/ buy fucidin
<a href=https://allpillrx.com/#>buy amoxil online</a> buy omnicef
buy nitrofurantoin http://worldrx100.com/ panmycin capsules
<a href=http://worldrx100.com/#>buy sumycin generic</a> buy zyvox
buy omnicef generic http://bio-catalyst.com/ erythromycin tablets
<a href=https://allpillrx.com/#>order bactrim</a> order cipro
amoxil generic http://bio-catalyst.com/ order ciplox
<a href=http://worldrx100.online/#>cleocin for sale</a> chloromycetin tablets
minocycline price http://bio-catalyst.com/ buy augmentin online
<a href=http://bio-catalyst.com/#>order suprax</a> doxycycline generic
buy ketoconazole http://worldrx100.online/ buy ciplox online
<a href=http://worldrx100.com/#>buy omnicef online</a> fucidin for sale
clindamycin tablets http://bio-catalyst.com/ terramycin capsules
<a href=http://worldrx100.com/#>chloramphenicol capsules</a> tinidazole tablets
trimox price http://worldrx100.com/ generic biaxin
<a href=http://worldrx100.online/#>buy myambutol generic</a> buy keflex
buy ceftin generic https://allpillrx.com/ vantin price
<a href=http://worldrx100.com/#>cephalexin online</a> nitrofurantoin online
buy cefixime generic http://bio-catalyst.com/ buy cefixime online
<a href=http://worldrx100.com/#>buy chloramphenicol</a> flagyl generic
generic cephalexin http://worldrx100.com/ chloramphenicol tablets
<a href=http://worldrx100.online/#>order ketoconazole online</a> ketoconazole capsules
ciplox for sale http://bio-catalyst.com/ cefixime tablets
<a href=http://worldrx100.online/#>generic cleocin</a> generic minocin
order cefadroxil https://allpillrx.com/ buy chloramphenicol generic
<a href=https://allpillrx.com/#>floxin for sale</a> chloromycetin online
buy ciplox online https://allpillrx.com/ chloramphenicol for sale
<a href=http://worldrx100.online/#>generic floxin</a> ampicillin online
buy zithromax online http://bio-catalyst.com/ buy minocin online
<a href=http://worldrx100.com/#>zithromax for sale</a> ciplox generic
trimox online https://allpillrx.com/ generic sumycin
<a href=http://bio-catalyst.com/#>tinidazole tablets</a> flagyl for sale
minocycline tablets http://bio-catalyst.com/ buy trimox generic
<a href=http://worldrx100.online/#>order ciplox</a> trimox for sale
keftab price http://worldrx100.online/ buy cefadroxil
<a href=https://allpillrx.com/#>doxycycline for sale</a> keflex for sale
buy minomycin online http://worldrx100.com/ cefadroxil capsules
<a href=https://allpillrx.com/#>order clindamycin online</a> buy panmycin online
buy cefixime online http://bio-catalyst.com/ order noroxin online
<a href=http://worldrx100.online/#>panmycin tablets</a> ciplox tablets
buy floxin generic http://worldrx100.online/ order sumycin
<a href=http://worldrx100.com/#>order fucidin</a> order tetracycline
tetracycline generic https://allpillrx.com/ buy ciplox online
<a href=http://worldrx100.online/#>keflex tablets</a> generic terramycin
buy terramycin online http://bio-catalyst.com/ ketoconazole tablets
<a href=http://worldrx100.online/#>order tinidazole online</a> terramycin generic
buy doxycycline generic https://allpillrx.com/ cefixime capsules
<a href=https://allpillrx.com/#>trimox price</a> order ceftin online
cephalexin capsules http://bio-catalyst.com/ buy cephalexin generic
<a href=http://bio-catalyst.com/#>cephalexin price</a> order chloromycetin
generic cefadroxil http://worldrx100.online/ chloromycetin generic
<a href=http://bio-catalyst.com/#>ampicillin online</a> order chloromycetin
buy ceftin online https://allpillrx.com/ generic amoxil
<a href=https://allpillrx.com/#>tinidazole tablets</a> keflex tablets
order zyvox https://allpillrx.com/ floxin capsules
<a href=http://bio-catalyst.com/#>buy terramycin generic</a> order ketoconazole
buy sumycin generic http://worldrx100.online/ order ceftin
<a href=http://worldrx100.online/#>keflex price</a> generic trimox
generic terramycin http://worldrx100.online/ buy terramycin
<a href=http://bio-catalyst.com/#>buy myambutol generic</a> cefadroxil price
order cefadroxil http://worldrx100.com/ buy ketoconazole generic
<a href=https://allpillrx.com/#>ciplox online</a> noroxin capsules
generic biaxin http://worldrx100.online/ generic minocin
<a href=http://worldrx100.com/#>buy roxithromycin generic</a> generic roxithromycin
order minomycin http://worldrx100.com/ sumycin price
<a href=http://bio-catalyst.com/#>trimox tablets</a> buy nitrofurantoin online
floxin price http://bio-catalyst.com/ order keftab online
<a href=http://worldrx100.com/#>order floxin</a> cephalexin generic
buy floxin generic https://allpillrx.com/ buy terramycin online
<a href=http://bio-catalyst.com/#>order zyvox online</a> ampicillin generic
fucidin generic https://allpillrx.com/ buy ampicillin
<a href=https://allpillrx.com/#>buy suprax online</a> buy chloramphenicol online
minocycline price http://bio-catalyst.com/ ciplox generic
<a href=http://bio-catalyst.com/#>ampicillin tablets</a> order keflex
noroxin capsules http://bio-catalyst.com/ buy cephalexin
<a href=http://worldrx100.com/#>buy ceftin</a> buy ceftin generic
order bactrim online http://worldrx100.online/ order clindamycin
<a href=https://allpillrx.com/#>ampicillin capsules</a> zithromax online
buy zyvox https://allpillrx.com/ buy fucidin generic
<a href=https://allpillrx.com/#>ciplox generic</a> buy zyvox
buy flagyl generic https://allpillrx.com/ vantin online
<a href=http://worldrx100.online/#>chloramphenicol generic</a> trimox for sale
erythromycin tablets http://bio-catalyst.com/ buy tetracycline generic
<a href=http://worldrx100.com/#>erythromycin online</a> minomycin online
minocycline capsules http://worldrx100.com/ generic fucidin
<a href=http://worldrx100.online/#>amoxil online</a> roxithromycin generic
buy bactrim online http://worldrx100.com/ cleocin capsules
<a href=http://worldrx100.online/#>ampicillin capsules</a> terramycin online
buy ceftin http://worldrx100.online/ keflex online
<a href=http://bio-catalyst.com/#>buy amoxil</a> buy clindamycin
order suprax online http://worldrx100.online/ ampicillin price
<a href=http://worldrx100.online/#>tinidazole tablets</a> vantin online
buy bactrim generic http://worldrx100.online/ buy trimox
<a href=http://bio-catalyst.com/#>buy cefadroxil generic</a> buy ampicillin generic