VE401 Final Review

Fisher’s Hypothesis Test

Hypotheses and Testing

Example: I am a soft drink manufacturer. You are a seller and want to buy my products. I claim that my product has μ≥500ml. You don’t trust me and sampled 20 bottles from my batch.

Data=RandomVariate[NormalDistribution[499,2],20]

Mean[Data]

I claim that the samples are too “extreme” and you are picking bottles with small volumes by chance. Actually, more and more have larger volume. Would you believe me?

Steps:

Figure out the parameter θ and the null value (Subscript[θ, 0]) you are testing. (θ=μ,(Subscript[θ, 0])=500 in my example)

Set up a null hypothesis (Subscript[H, 0]). It can be one of the following: (μ≥500 in my example)

θ=(Subscript[θ, 0]).

θ≤(Subscript[θ, 0]).

θ≥(Subscript[θ, 0]).

Figure out the distribution of the estimator of θ and thus the test statistics. ((((OverBar[X])-μ)/(σ/(Sqrt[n]))) follows standard normal distribution. We take Z=(((OverBar[X])-μ)/(σ/(Sqrt[n]))) as test statistics)

Analyze data, calculate P-value to see if there is enough evidence of reject (Subscript[H, 0]).

Significance or P-value for One-Tailed Test

Intuition: How likely will we observe the data if what I claim is true? How “extreme” is the data? P=P[getting this or even more "extreme" observation|(Subscript[H, 0]) is true].

Mathematical representation: Denoting the sample statistics of (OverHat[θ]) as k, we have

({{({{P value=({{{, ({{P[(OverHat[θ])≤k|θ=(Subscript[θ, 0])], if null hypothesis is (Subscript[H, 0]):θ≥(Subscript[θ, 0]) }, {P[(OverHat[θ])≥k|θ=(Subscript[θ, 0])], if null hypothesis is (Subscript[H, 0]):θ≤(Subscript[θ, 0])}})}})}})}})

Example:

What is the P-value for my claim that μ≥500ml? Assume that we know true variance (σ^2)=4(ml^2) and sample statistics (OverBar[X])=499.115 ml.

For (Subscript[H, 0]):μ≥500, we calculate (suppose we already know that σ=2).

({{({{P[getting more "extreme" observation|(Subscript[H, 0]) is true]=P[(OverBar[X])≤ 499.11533291015076`|μ≥500, σ=2]}, {   <=P[(OverBar[X])≤ 499.11533291015076`|μ=500, σ=2]}, {=P[(((OverBar[X])-500)/(2/(Sqrt[20])))≤((499.11533291015076`-500)/(2/(Sqrt[20]))) ]
=P[Z≤-1.978]
=0.024}})}})

CDF[NormalDistribution[],((Mean[Data]-500)/(2/(Sqrt[20])))]

So, there is at most 2.4% chance to obtain such “extreme” data if my claim is true. Since it is small, so we reject (Subscript[H, 0]) at the 2.4% level of significance.

Significance or P-value for Two-Tailed Test

Mathematical representation: Denoting the sample statistics of (OverHat[θ]) as k, we have

({{P value=({{P[|(OverHat[θ])-(Subscript[θ, 0])|≥|k-(Subscript[θ, 0])| |θ=(Subscript[θ, 0])] , if null hypothesis is (Subscript[H, 0]):θ=(Subscript[θ, 0])}})}})

Example:

What is the P-value if my claim is μ=500ml? Assume that we know true variance (σ^2)=4(ml^2) and sample statistics (OverBar[X])=499.115 ml.

For (Subscript[H, 0]):μ≥500, we calculate (suppose we already know that σ=2).

({{({{P[getting more "extreme" observation|(Subscript[H, 0]) is true]=P[|(OverBar[X])-500|≥| 499.115-500||μ=500, σ=2]}, {   =2P[(OverBar[X])≤ 499.11533291015076`|μ=500, σ=1]}, {=0.048}})}})

So, there is at most 4.8% chance to obtain such “extreme” data if my claim is true. Since it is small, so we reject (Subscript[H, 0]) at the 4.8% level of significance.

Neyman-Pearson Decision Theory

Hypotheses and Testing

Steps:

Figure out the parameter θ and the null value (Subscript[θ, 0]) you are testing.

We set up two competing hypotheses (Subscript[H, 0]) and (Subscript[H, 1]). (e.g. (Subscript[H, 0]):μ≥500, (Subscript[H, 1]):μ≤499). They can be one of the following: for some δ>0,

(Subscript[H, 0]):θ=(Subscript[θ, 0]), (Subscript[H, 1]):|θ-(Subscript[θ, 0])|≥δ.

(Subscript[H, 0]):θ≤(Subscript[θ, 0]), (Subscript[H, 1]):θ≥(Subscript[θ, 0])+δ.

(Subscript[H, 0]):θ≥(Subscript[θ, 0]), (Subscript[H, 1]):θ≤(Subscript[θ, 0])-δ .

Decide α and β. Calculate the critical region and the sample size.

Analyze data. If the sample statistics lies in the interval of (Subscript[H, 1]), we accept (Subscript[H, 1]). Otherwise, we see the whether the sample statistics lies in the critical region. If yes, then reject (Subscript[H, 0]).

Type I and Type II Errors

Intuition: Type I error is the case when my claim is falsely rejected. Type II error is the case when my claim is falsely accepted.

({{ , , In fact, }, {, , (Subscript[H, 0]) true, (Subscript[H, 0]) false}, {Our choice, Accept (Subscript[H, 0]), Good, Type II error}, {, Reject (Subscript[H, 0]), Type I error, Good}})

α and Critical Region

Definition: α=P[Type I error]. The critical region is the interval such that if (Subscript[H, 0]) is true, the probability of statistics falling into the interval (letting us to reject (Subscript[H, 0])) is at most α. In other words, the complement of critical region is a 100(1-α)% confidence interval for the parameter.

Usage: If P value≤α, we reject (Subscript[H, 0]). If sample statistics lies in the critical region, we reject (Subscript[H, 0]).

Example:

We set α=0.05. Calculate the critical region for (Subscript[H, 0]):μ≥500ml. Assume that we know true variance (σ^2)=4(ml^2). According to the critical region, should we reject (Subscript[H, 0])?

We want Z=(((OverBar[X])-(Subscript[μ, 0]))/(σ/(Sqrt[n])))<-(Subscript[z, 0.05]) to reject (Subscript[H, 0]). The critical region is then (-∞,(Subscript[μ, 0])-(Subscript[z, 0.05])(σ/(Sqrt[n])))=(-∞,499.264). Since our data (OverBar[X])=499.115 lies in the critical region, we reject (Subscript[H, 0]).

(Subscript[z, α_]):=InverseCDF[NormalDistribution[],1-α];
500-((2(Subscript[z, 0.05]) )/(Sqrt[20]))

β and Sample Size

Definition: β=P[Type II error]=P[Sample statistics lies outside critical region|(Subscript[H, 1]) true].

Usage:

To calculate a good sample size. n≈((((((Subscript[z, α/2])+(Subscript[z, β])))^2)(σ^2))/(δ^2)).

To calculate the power of a test. power=1-β. A powerful test needs smaller sample size, and is more likely to reject (Subscript[H, 0]).

To plot the OC curve, or β vs. d curve, so that we find n more easily. Here

({{({{d=({{{, ({{((|μ-(Subscript[μ, 0])|)/σ), if (Subscript[H, 0]):μ=(Subscript[μ, 0])}, {((μ-(Subscript[μ, 0]))/σ), if (Subscript[H, 0]):μ≤(Subscript[μ, 0])}, {-((μ-(Subscript[μ, 0]))/σ), if (Subscript[H, 0]):μ≥(Subscript[μ, 0])}})}})}})}})

Examples:

What is the power of my test given that the true mean μ=499ml, true variance (σ^2)=4(ml^2) and the sample size n=20?

We have d=((|499-500|)/2)=0.5,n=20. According to the OC Curve, β≈0.3. So the power is approximately 0.7. So the test is not powerful enough.

If the true mean of my product has volume μ=499ml, and the variance is (σ^2)=4(ml^2), what sample size do we need to get a power of 0.8, if we set α=0.05?

We have d=((|499-500|)/2)=0.5. We will need about 30 samples to get a power of 0.8.

((((((Subscript[z, 0.05])+(Subscript[z, 0.2])))^2)4)/(((499-500))^2))

Plot the OC Curve of the test of my product, given α=0.05, sample size n=20 and the variance (σ^2)=4(ml^2).

In my case (Subscript[H, 0]):μ≥500, (Subscript[H, 1]):μ≤499, we need to calculate

({{({{({{P[fail to reject (Subscript[H, 0])]=P[Sample statistics lies outside critical region]}, {=P[(OverBar[X])>499.264|(((Subscript[μ, 0])-μ)/σ)=d]}, {<=P[(OverBar[X])>499.264|μ=500-2 d]}, {=P[(((OverBar[X])-(500-2d))/(2/(Sqrt[20])))>((499.264-(500-2d))/(2/(Sqrt[20])))]}, {=P[Z>((499.264-(500-2d))/(2/(Sqrt[20])))]}})}})}})

Plot[1-CDF[NormalDistribution[],((499.264-(500-2d))/(2/(Sqrt[20])))],{d,-.5,2},AxesLabel->{"d","P[fail to reject (Subscript[H, 0])]"}, PlotLabel->"OC Curve of Z test when n=20"]

When d=0, we can see that β=1-α=0.95.

How large should the true mean μ differ from (Subscript[μ, 0])=500ml so that (Subscript[H, 0]) is rejected 90% of the time (given (Subscript[H, 1]) is true)?

According to the OC Curve, at d≈0.65 we obtain β=0.1 or a power of 0.9. This means we need (((Subscript[μ, 0])-μ)/σ)=((500-μ)/2)>0.65=>μ<498.7.

Null Hypothesis Significance Tests (NHST)

Hypotheses and Testing

Steps:

Figure out the parameter θ and the null value (Subscript[θ, 0]) you are testing.

We set up two competing hypotheses (Subscript[H, 0]) and (Subscript[H, 1]). (e.g. (Subscript[H, 0]):μ≥500, (Subscript[H, 1]):μ≤499). They can be one of the following: for some δ>0,

(Subscript[H, 0]):θ=(Subscript[θ, 0]), (Subscript[H, 1]):θ≠(Subscript[θ, 0]).

(Subscript[H, 0]):θ≤(Subscript[θ, 0]), (Subscript[H, 1]):θ>(Subscript[θ, 0]).

(Subscript[H, 0]):θ≥(Subscript[θ, 0]), (Subscript[H, 1]):θ<(Subscript[θ, 0]).

Decide α.

Analyze data. Calculate the P-value and see if there is enough evidence to reject (Subscript[H, 0]).

Note: NHST is biased against failing to reject (Subscript[H, 0]) (too easy to reject (Subscript[H, 0])).

Single Sample Tests and Comparison of Parameter

Test Statistics

From now on, you will learn a lot of tests. For each test, you need to pay attention to:

What parameter are we testing?

What parameter do we know?

What is the distribution of test statistics?

Denoting the distribution of test statistics as 𝒟, and (Subscript[𝒹, α]) as the number such that (∫_(-∞)^(Subscript[𝒹, α]))(Subscript[f, 𝒟])(x)dx=1-α, we reject at significance level α,

(Subscript[H, 0]):θ=(Subscript[θ, 0]) if 𝒟>(Subscript[𝒹, α/2]) or 𝒟<(Subscript[𝒹, 1-α/2]). (Test statistics is too small or too big)

(Subscript[H, 0]):θ≤(Subscript[θ, 0]) if 𝒟>(Subscript[𝒹, α]). (Test statistics is too big)

(Subscript[H, 0]):θ≥(Subscript[θ, 0]) if 𝒟<(Subscript[𝒹, 1-α]). (Test statistics is too small)

Single Sample Tests for Mean (Z-Test and T-Test)

({{Testing 
Parameter, Distribution of (Subscript[X, i]), Variance  σ, Sample Size n, Test Statistics, OC Curve x-axis
(for two sided test)}, {μ, normal, known, any, Z=(((OverBar[X])-(Subscript[μ, 0]))/(σ/(Sqrt[n]))), d=((|μ-(Subscript[μ, 0])|)/σ)}, {, any, known, large, , }, {, any, unknown, large, Z=(((OverBar[X])-(Subscript[μ, 0]))/(S/(Sqrt[n]))), d=((|μ-(Subscript[μ, 0])|)/S)}, {, normal, unknown, small, (Subscript[T, n-1])=(((OverBar[X])-(Subscript[μ, 0]))/(S/(Sqrt[n]))), }})

Example:

If the true variance is unknown, what is the power of my T-test if the true mean is μ=499ml?

((500-499)/(Sqrt[Variance[Data]]))

The corresponding power is about 1-0.35=0.65.

Single Sample Tests for Variance (Chi-Squared Test)

({{Testing Parameter, Distribution of (Subscript[X, i]), Test Statistics, OC Curve x-axis
(for two sided test)}, {σ, Normal, (Subscript[χ, n-1]^2)=(((n-1)(S^2))/(Subscript[σ, 0]^2)), λ=(σ/(Subscript[σ, 0]))}})

Single Sample Tests for Median (Non-Parametric)å

Sign Test for the Median

({{Testing Parameter, Distribution of (Subscript[X, i]), Test Statistics}, {M, any, (Subscript[Binomial, n,0.5])=(Subscript[Q, + ])}})

where (SubPlus[Q]) is the number of data that lies above (Subscript[M, 0]). (Usually the data equal to (Subscript[M, 0]) is discarded).

Example:

Data

Test the hypothesis (Subscript[H, 0]):M≥500ml.

We can count that (SubPlus[Q])=7. P[(SubPlus[Q])≤7]=(1/(2^20))(∑_(i=0)^7)(({{20}, {i}}))=0.13. There is no enough evidence to reject (Subscript[H, 0]).

Wilcoxon Signed Rank Test

({{Testing Parameter, Distribution of (Subscript[X, i]), Test Statistics}, {M, symmetric, (SubPlus[W])}})

where (SubPlus[W]) is the sum of the positive ranks when ranking |(Subscript[X, i])-(Subscript[M, 0])|. When there is a tie, we take the average value. For example, if the first, second and third smallest |(Subscript[X, i])-(Subscript[M, 0])|  is a group of 3 ties, we rank them all as ((1+3)/2)=2.
For small n, you can refer to a table of critical values here: http://users.stat.ufl.edu/~winner/tables/wilcox_signrank.pdf. For n≥10, we can approximate the test statistics with E[W]=((n(n+1))/4), Var[W]=((n(n+1)(2n+1))/24). For each group of t ties, the variance is reduced by (((t^3)-t)/48).

Sort[Data-500]

Test the hypothesis (Subscript[H, 0]):M≥500ml.

We can count that (SubPlus[W])=1+4+5+8+10+13+18=59. According to the table, the critical value for α=0.05 is 60. Since 59<60 we are able to reject (Subscript[H, 0]) with P<0.05.

Needs["HypothesisTesting`"]
SignedRankTest[Data,500,"TestDataTable",AlternativeHypothesis->"Less"]

Estimation of Proportion

For a Bernoulli random variable (Subscript[X, i]), we can calculate (OverHat[p])=(OverBar[X])=(1/n)(∑_(i=1)^n)(Subscript[x, i]). The confidence interval for p would be

({{({{(OverHat[p])±(Subscript[z, α/2])(Sqrt[(OverHat[p])(1-(OverHat[p]))/n])}})}})

If we want to test that |p-(OverHat[p])|<d for some d>0 with 100(1-α)% confidence, we need the sample size to be at least

({{({{n=(((Subscript[z, α/2]^2)(OverHat[p])(1-(OverHat[p])))/(d^2))≤}})((Subscript[z, α/2]^2)/(4(d^2)))}})

Example:

Data

Give a 95% confidence interval of the proportion of good product (with volume ≥500ml) in my batch.

We have (OverHat[p])=7/20=0.35 and n=20. A confidence interval would be 0.35±0.21.

Single Sample Test of Proportion (Z-Test)

According to central limit theorem, (OverHat[p]) is approximately normally distributed with mean p and variance p(1-p)/n.

({{Testing Parameter, Distribution of (Subscript[X, i]), Sample Size, Test Statistics}, {p, Bernoulli, large, Z=(((OverHat[p])-(Subscript[p, 0]))/(Sqrt[(Subscript[p, 0])(1-(Subscript[p, 0]))/n]))}})

Comparing Two Proportions (Z-Test, Pooled Test)

({{Testing Parameter, Distribution of (Subscript[X, i]), Sample Size, Hypothesis (Subscript[H, 0]), Test Statistics}, {(Subscript[p, 1])-(Subscript[p, 2]), Bernoulli, large, |(Subscript[p, 1])-(Subscript[p, 2])|≥d
for some d>0, Z=(((Subscript[(OverHat[p]), 1])-(Subscript[(OverHat[p]), 2])-(Subscript[((Subscript[p, 1])-(Subscript[p, 2])), 0]))/(Sqrt[(((Subscript[(OverHat[p]), 1])(1-(Subscript[(OverHat[p]), 1])))/(Subscript[n, 1]))+(((Subscript[(OverHat[p]), 2])(1-(Subscript[(OverHat[p]), 2])))/(Subscript[n, 2]))]))}, {, , , (Subscript[p, 1])=(Subscript[p, 2]), Z=(((Subscript[(OverHat[p]), 1])-(Subscript[(OverHat[p]), 2]))/(Sqrt[(OverHat[p])(1-(OverHat[p]))((1/(Subscript[n, 1]))+(1/(Subscript[n, 2])))]))
where (OverHat[p])=(((Subscript[n, 1])(Subscript[(OverHat[p]), 1])+(Subscript[n, 2])(Subscript[(OverHat[p]), 2]))/((Subscript[n, 1])+(Subscript[n, 2])))}})

F Distribution

Purpose: To describe the distribution of the ratio of two scaled chi-squared random variables.

Mathematical representation: (Subscript[F, (Subscript[γ, 1]),(Subscript[γ, 2])])=(((Subscript[χ, (Subscript[γ, 1])]^2)/(Subscript[γ, 1]))/((Subscript[χ, (Subscript[γ, 2])]^2)/(Subscript[γ, 2])))is said to follow an F-distribution with (Subscript[γ, 1]) and (Subscript[γ, 2]) degrees of freedom.

Properties:

P[(Subscript[F, (Subscript[γ, 1]),(Subscript[γ, 2])])<x]=1-P[(Subscript[F, (Subscript[γ, 2]),(Subscript[γ, 1])])<(1/x)].

(Subscript[f, 1-α,(Subscript[γ, 1]),(Subscript[γ, 2])])=(1/(Subscript[f, α,(Subscript[γ, 2]),(Subscript[γ, 1])])).

P[(1/x)<F<x]=P[F<x]-P[F<(1/x)]=P[F<x]-

CDF[FRatioDistribution[4,5],1]

Comparing Two Variance (F-Test)

({{Testing Parameter, Distribution of (Subscript[X, i]), Test Statistics, Hypothesis (Subscript[H, 0]), OC Curve x-axis
(for(Subscript[n, 1])=(Subscript[n, 2])=n)}, {((Subscript[σ, 1])/(Subscript[σ, 2])), Normal, (Subscript[F, (Subscript[n, 1])-1,(Subscript[n, 2])-1])=((Subscript[S, 1]^2)/(Subscript[S, 2]^2)), ((Subscript[σ, 1])/(Subscript[σ, 2]))≥1
((Subscript[σ, 1])/(Subscript[σ, 2]))≤1
((Subscript[σ, 1])/(Subscript[σ, 2]))=1, λ=((Subscript[σ, 1])/(Subscript[σ, 2]))}})

Example:

Plot the OC Curve For F-test for sample sizes (Subscript[n, 1]), (Subscript[n, 2]).

({{({{P[fail to reject (Subscript[H, 0])]=P[Sample statistics lies outside critical region]}, {=P[(Subscript[f, 0.975,(Subscript[n, 1])-1,(Subscript[n, 2])-1])<((Subscript[S, 1]^2)/(Subscript[S, 2]^2))<(Subscript[f, 0.025,(Subscript[n, 1])-1,(Subscript[n, 2])-1])|λ=((Subscript[σ, 1])/(Subscript[σ, 2]))]}, {=P[((1/(Subscript[σ, 1]^2))/(1/(Subscript[σ, 2]^2)))(Subscript[f, 0.975,(Subscript[n, 1])-1,(Subscript[n, 2])-1])<(((Subscript[S, 1]^2)/(Subscript[σ, 1]^2))/((Subscript[S, 2]^2)/(Subscript[σ, 2]^2)))<((1/(Subscript[σ, 1]^2))/(1/(Subscript[σ, 2]^2)))(Subscript[f, 0.025,(Subscript[n, 1])-1,(Subscript[n, 2])-1])|λ=((Subscript[σ, 1])/(Subscript[σ, 2]))]}, {=P[(1/(λ^2))(Subscript[f, 0.975,(Subscript[n, 1])-1,(Subscript[n, 2])-1])<(((Subscript[S, 1]^2)/(Subscript[σ, 1]^2))/((Subscript[S, 2]^2)/(Subscript[σ, 2]^2)))<(1/(λ^2))(Subscript[f, 0.025,(Subscript[n, 1])-1,(Subscript[n, 2])-1])|λ=((Subscript[σ, 1])/(Subscript[σ, 2]))]}, {=P[(1/(λ^2))(Subscript[f, 0.975,(Subscript[n, 1])-1,(Subscript[n, 2])-1])<(Subscript[F, (Subscript[n, 1])-1,(Subscript[n, 2])-1])<(1/(λ^2))(Subscript[f, 0.025,(Subscript[n, 1])-1,(Subscript[n, 2])-1])]}})}})

One nice property for this β=f(λ) curve is f(λ)=f(1/λ).

(Subscript[n, 1])=16;(Subscript[n, 2])=16;
f[λ_]:=CDF[FRatioDistribution[(Subscript[n, 1])-1,(Subscript[n, 2])-1],(1/(λ^2))InverseCDF[FRatioDistribution[(Subscript[n, 1])-1,(Subscript[n, 2])-1],0.975]]-CDF[FRatioDistribution[(Subscript[n, 1])-1,(Subscript[n, 2])-1],(1/(λ^2))InverseCDF[FRatioDistribution[(Subscript[n, 1])-1,(Subscript[n, 2])-1],0.025]];
Plot[f[λ],{λ,0,5},PlotLabel->"OC Curve of F test when (Subscript[n, 1])="<>ToString[(Subscript[n, 1])]<>",(Subscript[n, 2])="<>ToString[(Subscript[n, 2])],AxesLabel->{"λ","P[Accept (Subscript[H, 0])]"}]

Comparing Two Means (Z-test, Pooled/Paired T-test)

Example: I am a soft drink manufacturer and I have two machines that produce bottles of soft drink. I would like to know if there is a difference in the products of the two machines. I sampled 10 bottles from each of the two machines.

Data1=RandomVariate[NormalDistribution[499,2],10]
Data2=RandomVariate[NormalDistribution[500,2],10]

Mean[Data1]
Mean[Data2]

(Sqrt[Variance[Data1]])
(Sqrt[Variance[Data2]])

I would like to test (Subscript[H, 0]):(Subscript[μ, 1])≥(Subscript[μ, 2]), and (Subscript[H, 1]):(Subscript[μ, 1])≤(Subscript[μ, 2])-1.

({{Testing 
Parameter, Distribution 
of the two RVs, Sample size
(Subscript[n, 1]),(Subscript[n, 2]), Variance
(Subscript[σ, 1]^2), (Subscript[σ, 2]^2), Test Statistics, OC curve x-axis
(for two-tailed test)}, {(Subscript[μ, 1])-(Subscript[μ, 2]), Normal, any, known, Z=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[σ, 1]^2)/(Subscript[n, 1])+(Subscript[σ, 2]^2)/(Subscript[n, 2])])), d=((|(Subscript[μ, 1])-(Subscript[μ, 2])|)/(Sqrt[(Subscript[σ, 1]^2)+(Subscript[σ, 2]^2)])) where
n=(((Subscript[σ, 1]^2)+(Subscript[σ, 2]^2))/((Subscript[σ, 1]^2)/(Subscript[n, 1])+(Subscript[σ, 2]^2)/(Subscript[n, 2])))}, {, , large, unknown, Z=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[S, 1]^2)/(Subscript[n, 1])+(Subscript[S, 2]^2)/(Subscript[n, 2])])), }, {, , small, unknown with
σ=(Subscript[σ, 1])=(Subscript[σ, 2]), (Subscript[T, (Subscript[n, 1])+(Subscript[n, 2])-2])=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[S, p]^2)(1/(Subscript[n, 1])+1/(Subscript[n, 2]))]))
where (Subscript[S, p]^2)=((((Subscript[n, 1])-1)(Subscript[S, 1]^2)+((Subscript[n, 2])-1)(Subscript[S, 2]^2))/((Subscript[n, 1])+(Subscript[n, 2])-2)), In case of n=(Subscript[n, 1])=(Subscript[n, 2]), 
d=((|(Subscript[μ, 1])-(Subscript[μ, 2])|)/(2σ))
where we use modified 
sample size (SuperStar[n])=2n-1,
and σ can be estimated.}, {, , small, unknown, (Subscript[T, γ])=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[S, 1]^2)/(Subscript[n, 1])+(Subscript[S, 2]^2)/(Subscript[n, 2])])) where
γ=(((((Subscript[S, 1]^2)/(Subscript[n, 1])+(Subscript[S, 2]^2)/(Subscript[n, 2])))^2)/((((((Subscript[S, 1]^2)/(Subscript[n, 1])))^2)/((Subscript[n, 1])-1))+(((((Subscript[S, 2]^2)/(Subscript[n, 2])))^2)/((Subscript[n, 2])-1)))) (we take
the floor), No simple OC Curves}})

When variances are unknown, current recommendations are to always use Welch’s test.

Suppose I know that (Subscript[σ, 1])=(Subscript[σ, 2])=2, which test would you use? What is the test statistics?

The Z-test with known variance. The test statistics is Z=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[σ, 1]^2)/(Subscript[n, 1])+(Subscript[σ, 2]^2)/(Subscript[n, 2])]))=((498.813-499.471)/(Sqrt[8/10]))=-0.73. This gives us a P-value of 0.23. This is large, so we failed to reject (Subscript[H, 0]).

CDF[NormalDistribution[],((Mean[Data1]-Mean[Data2])/(Sqrt[8/10]))]

Suppose we don’t know about the true variance, but suppose we know they are equal. which test would you use? What is the test statistics?

The T-test with equal variance. We have pooled variance (Subscript[S, p]^2)=((((Subscript[n, 1])-1)(Subscript[S, 1]^2)+((Subscript[n, 2])-1)(Subscript[S, 2]^2))/((Subscript[n, 1])+(Subscript[n, 2])-2))=((9(((1.64))^2)+9(((21.28))^2))/18)=2.15, so the test statistics is (Subscript[T, 10+10-2])=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[S, p]^2)(1/(Subscript[n, 1])+1/(Subscript[n, 2]))]))=((498.813-499.471)/(Sqrt[2.15×2/19]))=-1.0. This gives us a P-value of 0.16. So there is no evidence to reject (Subscript[H, 0]).

CDF[StudentTDistribution[18],((Mean[Data1]-Mean[Data2])/(Sqrt[((Variance[Data1]+Variance[Data2])/2) (2/10)]))]

(Sqrt[Variance[Data1]])
(Sqrt[Variance[Data2]])

Suppose we know that (Subscript[μ, 1])=499,(Subscript[μ, 2])=500. What is the power of the T-test for unknown variance if α=0.05?

We can use the estimated variance (Subscript[S, p]^2)=2.15, and calculate d=(((Subscript[μ, 2])-(Subscript[μ, 1]))/(2σ))≈(1/(2(Sqrt[2.15])))=0.34. Choosing (SuperStar[n])=2n-1=19≈20, we get a β of approximately 0.56, so the power is approximately 0.44.

Suppose we don’t know anything about the true variance. which test would you use? What is the test statistics?

Welch’s T-test with unknown variance. We have the degrees of freedom γ=⌊(((((Subscript[S, 1]^2)/(Subscript[n, 1])+(Subscript[S, 2]^2)/(Subscript[n, 2])))^2)/((((((Subscript[S, 1]^2)/(Subscript[n, 1])))^2)/((Subscript[n, 1])-1))+(((((Subscript[S, 2]^2)/(Subscript[n, 2])))^2)/((Subscript[n, 2])-1))))⌋=16. (Subscript[T, 16])=(((Subscript[(OverBar[X]), 1])-(Subscript[(OverBar[X]), 2])-(Subscript[((Subscript[μ, 1])-(Subscript[μ, 2])), 0]))/(Sqrt[(Subscript[S, p]^2)(1/(Subscript[n, 1])+1/(Subscript[n, 2]))]))=-1.0. This gives us again a P-value of 0.17. So there is no evidence to reject (Subscript[H, 0]).

Floor[((((Variance[Data1]/10+Variance[Data2]/10))^2)/(((((Variance[Data1]/10))^2)/9)+((((Variance[Data2]/10))^2)/9)))]

Comparing Locations (Non-parametric)

({{Testing Parameter, Distribution of X,Y, Sample Size, Hypothesis (Subscript[H, 0]), Test Statistics}, {P[X>Y], continuous, m samples of X
n samples of Y
m≤n., P[X>Y]=1/2
P[X>Y]≥1/2
P[X>Y]≤1/2, (Subscript[W, m])}})

Test for Mean of Difference of two RVs (Paired T - test)

({{Testing Parameter, Distribution of (X,Y), Sample Size, Test Statistics}, {(Subscript[μ, D])
where D=X-Y, joint bivariate
normal distribution, n=(Subscript[n, X])=(Subscript[n, Y]), (Subscript[T, n-1])=(((OverBar[D])-(Subscript[((Subscript[μ, D])), 0]))/(Sqrt[(Subscript[S, D]^2)/n]))}, {(Subscript[M, D])
where  D=X-Y, independent, same distribution
but different location (D symmetric), any, (SubPlus[W]) (Wilcoxon
signed rank statistics)}})

Comparison Between Pooled and Paired

({{Testing Parameter, Distribution of (X,Y), Test Statistics}, {(Subscript[μ, X])-(Subscript[μ, Y]), joint bivariate
normal distribution
, (Subscript[T, 2n-2])=(((OverBar[X])-(OverBar[Y]))/(Sqrt[2(Subscript[S, p]^2)/n]))    where (Subscript[S, p]^2)=(((Subscript[S, 1]^2)+(Subscript[S, 2]^2))/2)}, {, , (Subscript[T, n-1])=(((OverBar[D])-(Subscript[((Subscript[μ, D])), 0]))/(Sqrt[(Subscript[S, D]^2)/n]))}})
When the test statistics is the same,pooledT-test is more powerful because of more degrees of freedom.

Single Sample Test for Correlation Coefficient (Z - test)

({{Testing Parameter, Sample Size, Test Statistics}, {ρ, large,  Z=((Sqrt[n-3])/2) (ln(((1+R )/(1-R)))-ln(((1+(Subscript[ρ, 0]))/(1-(Subscript[ρ, 0])))))=(Sqrt[n-3])(artanh(R)-artanh((Subscript[ρ, 0])))}})

The 100(1-α)% confidence interval for ρ is tanh(artanh(R)±((Subscript[z, α/2])/(Sqrt[n-3])))

Linear Regression

If you want to perform simple linear regression, use the code below:

X= {1,1,3.3,3.3,4,4,4,5.6,5.6,5.6,6,6,6.5,6.5}
Y={1.6,1.8,1.8,2.7,2.6,2.6,2.2,3.5,2.8,2.1,3.4,3.2,3.4,100}

n= Length[X];
X2 =(X)^2;
Y2 = (Y)^2;
XY = X.Y;
Sxx=Total[X2]-n*(Mean[X]^2);
Syy=Total[Y2]-n*(Mean[Y]^2);
Sxy =Total[XY] -n*Mean[X]*Mean[Y];
b1 = Sxy/Sxx ;
b0 = Mean[Y]-b1*Mean[X];
SSE = Syy-b1*Sxy;
R2 = Sxy^2 /(Sxx*Syy);
S =Sqrt[ SSE/(n-2)];

Or just use the function MMA provides:

data={{0,1},{1,0},{3,2},{5,4}};

lm=LinearModelFit[data,x,x]

Normal[lm]

ListPlot[lm["FitResiduals"], Filling->Axis]

Lack of Fit: Error Sum of Squares pure error and lack of fit

Taking example from slide 666 - 667

SSEpe=6.453;
Syy=66.6437;
Sxy=154;
b1=0.051;

SSE=Syy−b1*Sxy

SSElf=SSE−SSEpe

Here F k−2;n−k, k-2=3 and n-k=10

F=((SSElf/(3))/(SSEpe/(10)))

InverseCDF[FRatioDistribution[3,10],0.95]

Based on the F3;10 distribution, we can reject H0 with P < 0.05 (f0.05;3;10 = 3.708). There is evidence that a linear regression model is not appropriate

Confidence and Prediction Intervals

X={100,120,140,160,180};
Y={45,54,66,74,85};
n= Length[X];
X2 =(X)^2;
Y2 = (Y)^2;
XY = X.Y;
Sxx=Total[X2]-n*(Mean[X]^2);
Syy=Total[Y2]-n*(Mean[Y]^2);
Sxy =Total[XY] -n*Mean[X]*Mean[Y];
b1 = Sxy/Sxx ;
b0 = Mean[Y]-b1*Mean[X];
SSE = Syy-b1*Sxy;
R2 = Sxy^2 /(Sxx*Syy);
S =Sqrt[ SSE/(n-2)];
Data = Transpose[{X,Y}];
model = LinearModelFit[Data,x,x];
alpha =0.05;
t=InverseCDF[StudentTDistribution[n-2],alpha/2];
conf = model["MeanPredictionBands",ConfidenceLevel->1-alpha]/.x->130 ;
pred= model["SinglePredictionBands",ConfidenceLevel->1-alpha]/.x->130 ;
Print["Band for b1: ",b1+t*(S/(Sqrt[Sxx]))," to " ,b1-(S/(Sqrt[Sxx]))]
Print["Band for b0: ",b0+t*((S*Sqrt[Total[X2]])/(Sqrt[n*Sxx]))," to ",b0-t*((S*Sqrt[Total[X2]])/(Sqrt[n*Sxx]))]

Multiple Linear Regression

X=(({{0.5, 5}, {1, 4}, {1.5, 3}, {2, 2}, {2.5, 1}}));
y=(({{-0.51}, {-2.09}, {-6.03}, {-9.28}, {-17.12}}));
b = Inverse[Transpose[X].X].Transpose[X].y

F-Test for Significance of Regression

m = LinearModelFit[{X,y}];
m
p = 2;
n=5;
FStatistics = ((n-p-1)/p) ((m["RSquared"])/(1-m["RSquared"]))

Confidence Interval

m["ParameterConfidenceIntervalTable",ConfidenceLevel->.90]

m["ParameterConfidenceIntervalTable",ConfidenceLevel->.95]

T-Test for the Model Parameters

See slide 731