Imph
★    

2022-06-20 18:16
(647 d 03:35 ago)

Posting: # 23069
Views: 1,909
 

 Unequal variances in parallel design [Regulatives / Guidelines]

Hello,

I would like to know if in the parallel design, we should not assume the equality of the variances.

If the equality of variances is not assumed, how do the statistical model, the statistical analysis and the formula for the confidence interval become?

Best regards.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2022-06-20 18:47
(647 d 03:04 ago)

@ Imph
Posting: # 23070
Views: 1,743
 

 Welch’s t-test

Hi Imph,

❝ I would like to know if in the parallel design, we should not assume the equality of the variances.


Correct. Specifically stated by the FDA

For parallel designs […] equal variances should not be assumed.

and in between the lines by the EMA

The precise model to be used for the analysis should be pre-specified in the protocol. The statistical analysis should take into account sources of variation that can be reasonably assumed to have an effect on the response variable.


❝ […] how do the statistical model, the statistical analysis and the formula for the confidence interval become?


Perform Welch’s t-test, i.e., with Satterthwaite’s approximation of the degrees of freedom. $$\nu=\frac{\big{(}\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\big{)}^2}{\frac{s_1^4}{n_1^2(n_1-1)}+\frac{s_2^4}{n_2^2(n_2-1)}}$$where \(\small{n_1,n_2}\) are the number of subjects under treatments T and R, respectively. \(\small{s_1,s_2}\) are the standard deviations of treatment arms. Not by any chance it is the default in [image] and SAS, whereas SPSS calculates always the conventional t-test (assuming equal variances) and the Welch-test. In the output table, use the second row.
Calculate the confidence interval as usual with the t-value for given \(\small{\alpha,\,\nu}\).

It’s also easy in Phoenix WinNonlin and PKanalix. Consult the respective manual. You should check whether your setup is correct.*



Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Imph
★    

2022-06-23 15:02
(644 d 06:49 ago)

@ Helmut
Posting: # 23074
Views: 1,666
 

 Welch’s t-test

Hello,

Thank you for your valuable response.

Is it possible to conduct an analysis of variance according to the following table without assuming equality of variances?

[image]

Does Welch's correction only concern the calculation of confidence intervals?

Thank's a lot.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2022-06-23 16:19
(644 d 05:33 ago)

@ Imph
Posting: # 23075
Views: 1,643
 

 Welch’s t-test

Hi Imph,

❝ Thank you for your valuable response.


Welcome.

❝ Is it possible to conduct an analysis of variance according to the following table without assuming equality of variances?


You could but it would be wrong. Since you must not assume equal variances and in your table the residual degrees of freedom n1+n2–2 are the ones of the conventional t-test.

❝ Does Welch's correction only concern the calculation of confidence intervals?


You obtain also the p-value of the treatment difference (which is, however, not relevant in bioequivalence).

Cross-validated (‼) example codes are given as supplementary information to the publication mentioned in my last post:The first one should work also in Excel.

There is no excuse to use a flawed method.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Imph
★    

2022-06-26 12:06
(641 d 09:46 ago)

@ Helmut
Posting: # 23089
Views: 1,580
 

 Welch’s t-test

Hello,

How can we analyze the treatment effect if we cannot use the analysis of variance?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2022-06-26 14:12
(641 d 07:40 ago)

@ Imph
Posting: # 23090
Views: 1,570
 

 Read the paper, use the code(s)!

Hi Imph,

❝ How can we analyze the treatment effect if we cannot use the analysis of variance?


Why didn’t you bother reading the paper given there and trying one of the codes given there?
Again: Why are you interested in the – irrelevant – treatment effect (a p-value)? Statistical significance  clinical relevance. BE is solely based on the latter.

All formulas for log-transformed data. Testing for a statistically significant difference $$H_0:\mu_\text{T}-\mu_\text{R}=0\;vs\;H_1:\mu_\text{T}-\mu_\text{R}\neq0\tag{1}$$ was aban­doned more than thirty years ago by the Two One-Sided Tests [1] $$\eqalign{
H_{01}:\mu_\text{T}-\mu_\text{R}\leq\theta_1&vs\;H_{11}:\mu_\text{T}-\mu_\text{R}>\theta_1\\
&\text{and}\\
H_{02}:\mu_\text{T}-\mu_\text{R}\geq\theta_2&vs\;H_{12}:\mu_\text{T}-\mu_\text{R}<\theta_2}\tag{2}$$ or the operationally identical – and preferred in guidelines – confidence interval inclusion approach $$H_0:\mu_\text{T}-\mu_\text{R}\ni\left\{\theta_1,\,\theta_2\right\}\;vs\;H_1:\theta_1<\mu_\text{T}-\mu_\text{R}<\theta_2\tag{3}$$ The limits \(\small{\left\{\theta_1,\,\theta_2\right\}}\) are based on the clinically not relevant difference \(\small{\Delta}\). In other words, nobody is interested in a p-value. See also this article.

An [image]-script to evaluate the reference data set #6 [2] (unequal groups sizes and unequal variances):

data   <- read.csv("https://bebac.at/downloads/p6.csv", sep = ",", dec = ".")
alpha  <- 0.05
level  <- 1 - 2 * alpha
eq     <- FALSE # use Welch’s test; TRUE for the t-test
T      <- data[data$treatment == "T", ]
R      <- data[data$treatment == "R", ]
# By default a two-sided confidence interval
t      <- t.test(log(T$response), log(R$response), var.equal = eq, conf.level = level)
# The rest of the script is only cosmetics
mean.T <- mean(log(T$response))
var.T  <- var(log(T$response))
mean.R <- mean(log(R$response))
var.R  <- var(log(R$response))
PE     <- exp(as.numeric(t$estimate[1] - t$estimate[2]))
CI     <- exp(as.numeric(t$conf.int))
ifelse (eq, txt <- "\n  (assuming equal variances)",
            txt <- "\n  (adjusting for unequal group sizes and/or variances)")
cat(paste0("Descriptive statistics\nTest group\n  Observations  : ", nrow(T),
    sprintf("\n  Mean          : %+.7f (log-scale)", mean.T),
    sprintf("\n  Variance      :  %.7f (log-scale)", var.T),
    sprintf("\n  Geometric mean:  %.5f", exp(mean.T)),
    sprintf("\n  Geometric CV  : %.2f%%", 100 * sqrt(exp(var.T) - 1)),
    "\nReference group\n  Observations  : ", nrow(R),
    sprintf("\n  Mean          : %+.7f (log-scale)", mean.R),
    sprintf("\n  Variance      :  %.7f (log-scale)", var.R),
    sprintf("\n  Geometric mean:  %.5f", exp(mean.R)),
    sprintf("\n  Geometric CV  : %.2f%%\n\n ", 100 * sqrt(exp(var.R) - 1))),
    trimws(t$method), "based on log-transformed data", txt,
    "\n  alternative hypothesis: true difference in means is not equal to 0",
    "\n  standard error of the difference =", signif(t$stderr, 8),
    sprintf("%s %5g, df = %g, p = %5g", "\n  t =",
    t$statistic, t$parameter, t$p.value),
    sprintf("\n%s %6.2f%%", "\nPoint estimate         :", 100 * PE),
    sprintf("%s%g%% %s %6.2f%% %s %6.2f%%%s", "\n", 100 * level,
    "confidence interval:", 100 * CI[1], "–", 100 * CI[2], "\n")))

Gives:

Descriptive statistics
Test group
  Observations  : 24
  Mean          : -0.0477730 (log-scale)
  Variance      :  0.0611544 (log-scale)
  Geometric mean:  0.95335
  Geometric CV  : 25.11%
Reference group
  Observations  : 26
  Mean          : -0.0785241 (log-scale)
  Variance      :  0.0578189 (log-scale)
  Geometric mean:  0.92448
  Geometric CV  : 24.40%

  Welch Two Sample t-test based on log-transformed data
  (adjusting for unequal group sizes and/or variances)
  alternative hypothesis: true difference in means is not equal to 0
  standard error of the difference = 0.06907899
  t = 0.445158, df = 47.429, p = 0.658231

Point estimate         : 103.12%
90% confidence interval:  91.84% – 115.79%


By setting eq <- TRUE in the script you will get at the end (only for comparison):

  Two Sample t-test based on log-transformed data
  (assuming equal variances)
  alternative hypothesis: true difference in means is not equal to 0
  standard error of the difference = 0.06899995
  t = 0.445668, df = 48, p = 0.657841

Point estimate         : 103.12%
90% confidence interval:  91.85% – 115.78%


Results are similar because there is no large difference in group sizes and variances are similar as well. How­ever, the conventional t-test is liberal (anticonservative, inflated patient’s risk) because its confidence interval is always narrower than the one of Welch’s test.

If you have extremely different sample sizes and variances, the outcome may differ substantially.
The reference data set #7 [3] is actually a complete failure with a 90% CI of 97.38–138.5% (Welch’s test: 201.164 degrees of freedom, standard error of the difference 0.106596) but almost passes BE with 106.86–126.2% (t-test: 1198 df, SE 0.0505922).
Why is the CI obtained by the t-test narrower than by Welch’s test though the point estimates are equal? We have more degrees of freedom (≈ 6×) and the standard error of the difference is smaller (≈ ½).


  1. Schuirmann DJ. A comparison of the Two One-Sided Tests Procedure and the Power Approach for Assessing the Equivalence of Average Bioavailability. J Pharmacokin Biopharm. 1987; 15(6): 657–80. doi:10.1007/BF01068419.
  2. Bolton S, Bon C. Statistical consideration: alternate designs and approaches for bioequivalence assessments. In: Kanfer I, Shargel L, editors. Generic Drug Product Development. Bioequivalence Issues. New York: Informa Healthcare; 2008. p. 123–141.
  3. \(\small{T:n=\text{1,000},\,\widehat{x}_{\text{geom}}=1.20583,\,\widehat{CV}_{\text{geom}}=\phantom{2}25.15\%}\)
    \(\small{R:n=\phantom{1,}200,\,\widehat{x}_{\text{geom}}=1.03825,\,\widehat{CV}_{\text{geom}}=293.02\%}\)

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
87 visitors (0 registered, 87 guests [including 13 identified bots]).
Forum time: 20:52 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5