## Parallel designs: Don’t use the (conventional) t-test! [Power / Sample Size]

Hi Sereng,

❝ […] the reference drug Cmax had almost twice the CV of the Test drug.

❝ Parallel Group Design

❝ Two Groups (n=70/group)

❝ Ratio (90% CI): 109.00 (87.00-135.00)

Since in this post (based on the t-test assuming equal variances) I could reproduce your results:
According to the FDA’s guidance (Section IV.B.1.d.):

For parallel designs, the confidence interval for the difference of means in the log scale can be computed using the total between-subject variance.1 […] equal variances should not be assumed.
(my emphasis)

Though you had equally sized groups, variances were not equal.
This calls for the Welch-test with Satter­th­waite’s approximation2 of the degrees of freedom:3,4 \eqalign{\nu&\approx\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{s_1^4}{n_1^2\,(n_1-1)} + \frac{s_2^4}{n_2^2\,(n_2-1)}}\\ &\approx\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{1}{n_1-1}\left(\frac{s_1^2}{n_1}\right)^2 + \frac{1}{n_2-1}\left(\frac{s_2^2}{n_2}\right)^2}} For good reasons in , SAS, and other software packages it is the default.
• Using a pre-test (F-test, Levene’s test, Bartlett’s test, Brown–Forsythe test) – as recommended in the past – is bad practice because it will inflate the Type I Error.5
• If $${s_{1}}^{2}={s_{2}}^{2}\;\wedge\;n_1=n_2$$, the formula given above reduces to the simple $$\nu=n_1+n_2-2$$ anyhow.
• In all other cases the Welch-test is conservative, which is a desirable property.
In SPSS both the conventional t-test and the Welch-test are performed. Always use the second row of the table of results.

@Divyen: If the confidence interval based on my derivation does not match the reported one, it is evident that the Welch-test was used. In such a case calculating the $$\small{MSE}$$ is not that trivial. Maybe I will try it later.

1. Misleading terminology. There is no ‘total between-subject variance’. In a parallel design only the total vari­ance – which is pooled from the between- and within-subject variances – is accessible.
2. Satterthwaite FE. An Approximate Distribution of Estimates of Variance Components. Biom Bull. 1946; 2(6): 110–4. doi:10.2307/3002019.
3. Both formulas are given in the literature. They are equivalent.
4. Allwood M. The Satterthwaite Formula for Degrees of Freedom in the Two-Sample t-Test. College Board. 2008. Open access.
5. Zimmermann DW. A note on preliminary tests of equality of variances. Br J Math Stat Psychol. 2004; 57(1): 173–81. doi:10.1348/000711004849222.

Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes