Two sample tests
We focus on testing the association or dependence of two variables observed on a given population. The question is: does the distribution of values of the first variable vary, given the values of the second? According to the nature (continuous, discrete, or binary) of the two variables, many tests can be used. We shall present only a few of them.
Continuous againt discrete
Specific tests have been developed when the discrete variable is binary. The first section focuses on them.
Continuous against binary
Consider a continuous variable, denoted by Cont (e.g. age), and a binary variable, denoted by Bin (e.g. gender). Descriptive statistics consists in computing the summaries of Cont by values of Bin, then display box plots of Cont against Bin:
by(Cont, Bin, summary)
boxplot(Cont~Bin)
If there is an association between Cont and Bin, the two summaries are different; of the two box plots, one is above the other, or one is larger than the other, etc.
Example In the titanic data set, study the link between the age and the gender.
R> titanic <- read.table("data/titanic.csv",header=TRUE,sep=";") R> G <- titanic[,"gender"] R> A <- titanic[,"age"] R> by(A,G,summary) G: F Min. 1st Qu. Median Mean 3rd Qu. Max. 0.1667 19.0000 27.0000 28.6871 38.0000 76.0000 ------------------------------------------------------------ G: M Min. 1st Qu. Median Mean 3rd Qu. Max. 0.3333 21.0000 28.0000 30.5852 39.0000 80.0000Code R :
R> boxplot(A~G)
Résultat : #r{codeimg}
The distributions can be represented through the plot of the empirical distribution function. Denote by X and Y the two subsamples of Cont, according to the values of Bin (e.g. X: age of women, Y: age of men). The ecdf are obtained with
plot(ecdf(X),col="red »)
lines(ecdf(Y),col="blue »)
Back to the example
Code R :R> Aw <- A[G=="F"] # ages of women R> Am <- A[G=="M"] # ages of men R> plot(ecdf(Aw),col="red") # ages of women R> lines(ecdf(Am),col="blue") # ages of men
Résultat : #r{codeimg}
The summaries and the plots are not sufficient to decide if the age means of men and women were different. To decide if the link between age and gender is significant, three tests can be applied.
In all three cases, the alternative is in two.sided (default), less, greater. The interpretation of less and greater is relative to the first variable X:
two.sided: the mean, variance, or ecdf ofXis not equal to that ofY,less: the mean, variance, or ecdf ofXis less that ofY,greater: the mean, variance, or ecdf ofXis greater than that ofY.
Recall that if the ecdf of X is less than that of Y, the values of X tend to be larger than those of `Y.
Example In the titanic data set, study the link between the age and the gender.
R> t.test(Aw,Am) Welch Two Sample t-test data: Aw and Am t = -2.0497, df = 798.36, p-value = 0.04072 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -3.71595773 -0.08036699 sample estimates: mean of x mean of y 28.68707 30.58523 R> t.test(A~G) Welch Two Sample t-test data: A by G t = -2.0497, df = 798.36, p-value = 0.04072 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -3.71595773 -0.08036699 sample estimates: mean in group F mean in group M 28.68707 30.58523The two instructions are equivalent.
The mean of women is less than that of men. To test if it is significant, we apply a one-sided test:
R> t.test(A~G, alternative="less") Welch Two Sample t-test data: A by G t = -2.0497, df = 798.36, p-value = 0.02036 alternative hypothesis: true difference in means is less than 0 95 percent confidence interval: -Inf -0.3731636 sample estimates: mean in group F mean in group M 28.68707 30.58523
We can also compare the two variances and the two ecdf through the Fisher's (variance) test and Kolmogorov-Smirnov test.
R> var.test(Aw,Am) F test to compare two variances data: Aw and Am F = 1.0419, num df = 387, denom df = 657, p-value = 0.6442 alternative hypothesis: true ratio of variances is not equal to 1 95 percent confidence interval: 0.8740042 1.2473141 sample estimates: ratio of variances 1.041945 R> ks.test(Aw,Am,alternative="greater") Two-sample Kolmogorov-Smirnov test data: Aw and Am D^+ = 0.084636, p-value = 0.03029 alternative hypothesis: the CDF of x lies above that of y
Continuous against discrete
This section considers the case of tha continuous variable and a discrete variable (not necessarily binary).
Consider a continuous variable, denoted by Cont (e.g. heart rate), and a discrete variable, denoted by Disc (e.g. type of disease). Here again, the first thing to do is to display summaries of Cont by values of Disc, and box plots of Cont against `Disc.
by(Cont,Disc,summary)
boxplot(Cont~Disc)
Example In the titanic data set, study the link between the age and the class.
R> titanic <- read.table("data/titanic.csv",header=TRUE,sep=";") R> P <- titanic[, "pclass"] R> A <- titanic[,"age"] R> by(A,P,summary) R> boxplot(A~P)
Résultat : #r{codeimg}
The discrete variable Disc divides the population in as many groups, as different values: if there are 3 types of disease, then there are 3 groups of patients, one per type of disease. If there is no dependence between Cont and Disc, the means of Cont within the different groups should be equal.
If there are only two groups, the variable is binary, and the tests of the previous section apply. In that case, the two-sided T-test will return exactly the same p-value as the one-way anova.
If the p-value is small, the conclusion is that there are significant differences between the means of the different groups. Those differences should be visible on the box plot. Differences between the means of two particular groups, or between the mean of one group and the mean in the rest of the population, can be tested using a one-sided T-test.
Without giving mathematical details, we shall now explain the expression analysis of variance. The global variance of Cont can be written as the sum of two variances: global variance = variance between groups + variance within groups
1. The variance between groups is the variance of the means of groups: the smaller the differences between means, the smaller the variance between groups.
2. The variance within groups is the mean of variances of groups. 2
If all group means are equal (null hypothesis of the test), then the variance between groups is null. The higher the ratio of the first variance to the second, the more significant the differences between the means. That ratio is returned by the function oneway.test, together with the corresponding p-value.
Back to the example In the titanic data set, study the link between the age and the class.
R> oneway.test(A~P) One-way analysis of means (not assuming equal variances) data: A and P F = 99.967, num df = 2.00, denom df = 546.72, p-value < 2.2e-16
The next step is to test the difference between two specific classes. This can be done with a T-test.
R> A1 <- A[P==1] # ages of first class R> A2 <- A[P==2] # ages of second class R> A3 <- A[P==3] # ages of third class passengers R> t.test(A1,c(A2,A3),alternative="greater") # first against others Welch Two Sample t-test data: A1 and c(A2, A3) t = 13.01, df = 454.27, p-value < 2.2e-16 alternative hypothesis: true difference in means is greater than 0 95 percent confidence interval: 11.12341 Inf sample estimates: mean of x mean of y 39.15992 26.42290 R> t.test(A1,A2,alternative="greater") # first against second Welch Two Sample t-test data: A1 and A2 t = 7.9947, df = 542.78, p-value = 3.932e-15 alternative hypothesis: true difference in means is greater than 0 95 percent confidence interval: 7.663748 Inf sample estimates: mean of x mean of y 39.15992 29.50670 R> t.test(A1,A3,alternative="greater") # first against third Welch Two Sample t-test data: A1 and A3 t = 14.129, df = 499.8, p-value < 2.2e-16 alternative hypothesis: true difference in means is greater than 0 95 percent confidence interval: 12.67057 Inf sample estimates: mean of x mean of y 39.15992 24.81637 R> t.test(A2,c(A1,A3)) # second against others Welch Two Sample t-test data: A2 and c(A1, A3) t = -0.50225, df = 475.13, p-value = 0.6157 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -2.450859 1.453016 sample estimates: mean of x mean of y 29.50670 30.00563 R> t.test(A2,A3,alternative="greater") # first against others Welch Two Sample t-test data: A2 and A3 t = 4.6948, df = 470.7, p-value = 1.753e-06 alternative hypothesis: true difference in means is greater than 0 95 percent confidence interval: 3.043793 Inf sample estimates: mean of x mean of y 29.50670 24.81637
Discrete against discrete
The conditional distributions of two discrete variables X and Y, can be visualized using contingency tables and barplots.If the two variables are independent (null hypothesis), then the conditional frequencies of one variable, should not depend on the values of the other. Visually, the bar heights should be equal on the bar plot.
Example In the titanic data set, let us compute the conditional frequencies of the survival given the passengers class:
R> S<-titanic[ ,"survived"] R> prop.table(table(S,P),1) P S 1 2 3 no 0.1663974 0.2358643 0.5977383 yes 0.4238876 0.2693208 0.3067916Code R :
R> barplot(prop.table(table(S,P),1),beside=TRUE)
Résultat : #r{codeimg}
Chi-squared test
The parameter of the chi- squared distribution is the product of the number of values in X minus one, times the number of values in Y minus one. For instance, if there are 3 values in X and 4 in Y, the parameter will be df = (3-1)*(4-1) = 6. The function chisq.test automatically calculates df.
Example Let us test the link between class and survival in titanic dataset.
R> chisq.test(table(S,P)) Pearson's Chi-squared test data: table(S, P) X-squared = 107.5, df = 2, p-value < 2.2e-16
Proportion test
In the specific case of two binary variables, one can also use a proportion test to decide whether the proportion of the population with a given value for X is the same, over the two values of Y.
As usual, the alternative is in:
two.sided: the proportion x1/n1 is not equal to the proportion x2/n2less: the proportion x1/n1 is less than the proportion x2/n2greater: the proportion x1/n1 is greater than the proportion x2/n2.
Example In the titanic data set, we can test the link between survival and gender with a proportion test:
R> SF<-S[G=="F"] R> SM<-S[G=="M"] R> x1<-sum(SF=="yes") R> n1<-length(SF) R> x2<-sum(SM=="yes") R> n2<-length(SM) R> prop.test(c(x1,x2),c(n1,n2)) 2-sample test for equality of proportions with continuity correction data: c(x1, x2) out of c(n1, n2) X-squared = 300.5, df = 1, p-value < 2.2e-16 alternative hypothesis: two.sided 95 percent confidence interval: 0.4924883 0.6023320 sample estimates: prop 1 prop 2 0.7525773 0.2051672
We can continue analysing the link between the two variables testing a one-sided alternative: Did women survive more than men ? Women data (x1, n1) are given first in the instruction, the alternative we want to test is therefore greater: H1 : x1/n1 ≥ x2/n2:
R> prop.test(c(x1,x2),c(n1,n2), alternative="greater") 2-sample test for equality of proportions with continuity correction data: c(x1, x2) out of c(n1, n2) X-squared = 300.5, df = 1, p-value < 2.2e-16 alternative hypothesis: greater 95 percent confidence interval: 0.5009889 1.0000000 sample estimates: prop 1 prop 2 0.7525773 0.2051672
Fisher's exact test
Another possibility exists for two binary variables, which is preferable: Fisher’s exact test. Explaining it will be the occasion to present the odds-ratio, which is widely used in the applications to medicine.
In the standard presentation, the two binary variables are understood as “exposure” (e.g. smoking or not) and “disease” (e.g. lung cancer or not). The contingency table contains the frequencies of 4 categories of patients:
| Diseased | Healthy | |
| exposed | ||
| unexposed |
The odds-ratio, a confidence interval, and the p-value are returned.
Example We can apply the Fisher test to the titanic data set:
R> table(G,S) S G no yes F 96 292 M 523 135 R> fisher.test(table(G,S)) Fisher's Exact Test for Count Data data: table(G, S) p-value < 2.2e-16 alternative hypothesis: true odds ratio is not equal to 1 95 percent confidence interval: 0.06225653 0.11556419 sample estimates: odds ratio 0.08513036
The odds-ratio (0.08) is less than 1. We can now test if being a women protected from death. The alternative is H1: the odds-ratio is less than 1:
R> fisher.test(table(S,G), alternative="less") Fisher's Exact Test for Count Data data: table(S, G) p-value < 2.2e-16 alternative hypothesis: true odds ratio is less than 1 95 percent confidence interval: 0.0000000 0.1103079 sample estimates: odds ratio 0.08513036
Continuous against continuous
The dependence between two continuous variables X and Y, observed over the same population, will be tested through their correlation. The first thing to do is a scatter plot of the two variables.
plot(X,Y)
If the correlation is positive, the two variables tend to vary in the same direction: if one increases, the other one does too. If the correlation is negative, the variables vary in opposite directions. The closer the correlation is to plus or minus 1, the closer the scatter plot to a straight line, and the stronger the association of the two variables.
The alternative is in:
two.sided: the correlation is not equal to 0 (XandYare correlated),less: the correlation is less than 0 (XandYare negatively correlated),greater: the correlation is greater than 0 (XandYare positively correlated).
Example 2 In the tauber data set, study the link between age, height and weight.
R> TA <- read.table("data/tauber.csv", header=TRUE,sep=";") R> head(TA) gender age height weight 1 F 68 113 20.0 2 M 74 116 17.5 3 M 69 120 23.0 4 M 72 121 25.0 5 M 73 114 17.0 6 M 65 115 20.0The correlations between each pair of variables are plotted with Code R :
R> A <- TA[,"age"] R> H <- TA[,"height"] R> W <- TA[,"weight"] R> AHW <- TA[,2:4] R> pairs(AHW)
Résultat : #r{codeimg}
The plot is equivalent too
R> plot(AHW)
or
R> plot(A,H) R> plot(A,W) R> plot(H,W)
Their correlation is obtained with
R> cor(AHW) age height weight age 1.0000000 0.3671279 0.2214119 height 0.3671279 1.0000000 0.7186317 weight 0.2214119 0.7186317 1.0000000
The correlation between age and height is smaller than that betweenheightandweight` but we can not directly conclude if they are significantly different from zero. There is not a unique threshold on the scale of the correlation to conclude if a correlation is different from zero or not. We need to apply a correlation test to decide.
R> cor.test(A,H,alternative="greater") Pearson's product-moment correlation data: A and H t = 21.214, df = 2889, p-value < 2.2e-16 alternative hypothesis: true correlation is greater than 0 95 percent confidence interval: 0.3403532 1.0000000 sample estimates: cor 0.3671279 R> cor.test(A,W,alternative="greater") Pearson's product-moment correlation data: A and W t = 12.204, df = 2889, p-value < 2.2e-16 alternative hypothesis: true correlation is greater than 0 95 percent confidence interval: 0.1921155 1.0000000 sample estimates: cor 0.2214119 R> cor.test(H,W,alternative="greater") Pearson's product-moment correlation data: H and W t = 55.546, df = 2889, p-value < 2.2e-16 alternative hypothesis: true correlation is greater than 0 95 percent confidence interval: 0.7035029 1.0000000 sample estimates: cor 0.7186317