Parallel designs: Don’t use the (conventional) t-test! [Power / Sample Size]
Hi Sereng,
Since in this post (based on the t-test assuming equal variances) I could reproduce your results:
According to the FDA’s guidance (Section IV.B.1.d.):
Though you had equally sized groups, variances were not equal.
This calls for the Welch-test with Satterthwaite’s approximation2 of the degrees of freedom:3,4 $$\eqalign{\nu&\approx\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{s_1^4}{n_1^2\,(n_1-1)} + \frac{s_2^4}{n_2^2\,(n_2-1)}}\\
&\approx\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{1}{n_1-1}\left(\frac{s_1^2}{n_1}\right)^2 + \frac{1}{n_2-1}\left(\frac{s_2^2}{n_2}\right)^2}}
$$ For good reasons in ,
@Divyen: If the confidence interval based on my derivation does not match the reported one, it is evident that the Welch-test was used. In such a case calculating the \(\small{MSE}\) is not that trivial. Maybe I will try it later.
❝ […] the reference drug Cmax had almost twice the CV of the Test drug.
❝ Parallel Group Design
❝ Two Groups (n=70/group)
❝ Ratio (90% CI): 109.00 (87.00-135.00)
Since in this post (based on the t-test assuming equal variances) I could reproduce your results:
According to the FDA’s guidance (Section IV.B.1.d.):
For parallel designs, the confidence interval for the difference of means in the log scale can be computed using the total between-subject variance.1 […] equal variances should not be assumed.
(my emphasis)
Though you had equally sized groups, variances were not equal.
This calls for the Welch-test with Satterthwaite’s approximation2 of the degrees of freedom:3,4 $$\eqalign{\nu&\approx\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{s_1^4}{n_1^2\,(n_1-1)} + \frac{s_2^4}{n_2^2\,(n_2-1)}}\\
&\approx\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{1}{n_1-1}\left(\frac{s_1^2}{n_1}\right)^2 + \frac{1}{n_2-1}\left(\frac{s_2^2}{n_2}\right)^2}}
$$ For good reasons in ,
SAS
, and other software packages it is the default.- Using a pre-test (F-test, Levene’s test, Bartlett’s test, Brown–Forsythe test) – as recommended in the past – is bad practice because it will inflate the Type I Error.5
- If \({s_{1}}^{2}={s_{2}}^{2}\;\wedge\;n_1=n_2\), the formula given above reduces to the simple \(\nu=n_1+n_2-2\) anyhow.
- In all other cases the Welch-test is conservative, which is a desirable property.
SPSS
both the conventional t-test and the Welch-test are performed. Always use the second row of the table of results.@Divyen: If the confidence interval based on my derivation does not match the reported one, it is evident that the Welch-test was used. In such a case calculating the \(\small{MSE}\) is not that trivial. Maybe I will try it later.
- Misleading terminology. There is no ‘total between-subject variance’. In a parallel design only the total variance – which is pooled from the between- and within-subject variances – is accessible.
- Satterthwaite FE. An Approximate Distribution of Estimates of Variance Components. Biom Bull. 1946; 2(6): 110–4. doi:10.2307/3002019.
- Both formulas are given in the literature. They are equivalent.
- Allwood M. The Satterthwaite Formula for Degrees of Freedom in the Two-Sample t-Test. College Board. 2008. Open access.
- Zimmermann DW. A note on preliminary tests of equality of variances. Br J Math Stat Psychol. 2004; 57(1): 173–81. doi:10.1348/000711004849222.
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- Back Calculating Sample Size Sereng 2022-05-12 17:40 [Power / Sample Size]
- Back Calculating Sample Size ElMaestro 2022-05-12 22:29
- Back Calculating Sample Size Helmut 2022-05-12 23:10
- Back Calculating Sample Size ElMaestro 2022-05-13 01:38
- Back Calculating Sample Size Sereng 2022-05-18 05:32
- PowerTOST: Total sample size Helmut 2022-05-18 08:55
- Back Calculating Sample Size dshah 2022-05-13 11:54
- Back Calculating Sample Size Helmut 2022-05-12 23:10
- Parallel designs: Don’t use the (conventional) t-test!Helmut 2022-05-17 14:17
- Back Calculating Sample Size ElMaestro 2022-05-12 22:29