Strong Type I Error control + CI: Not yet… [Two-Stage / GS Designs]

posted by Helmut Homepage – Vienna, Austria, 2023-12-22 13:01 (179 d 03:09 ago) – Posting: # 23801
Views: 4,645

Hi Alex,

❝ I am wondering whether the following publication made it to your attention: …


❝ I was the opinion that only simulation-based methods (Fuglsang 2014) are currently available/acceptable for parallel group designs due to the complexity of contructing repeated confidence intervals allowing for unequal variances between groups (a requirement for parallel designs according to FDA guidelines).

Right. I had serious problems convincing European regulators even with extensive simulations (unequal variances and/or group sizes due to dropouts). Note that the FDA is fine with simulation-based methods (5th GBHI workshop*).

❝ However, in the publication repeated confidence intervals were constructed using the Fisher combination test assuming equal variances (3 parallel treatment arms were analysed using ANOVA). What do you think about it?

Equal variances are a rather strong assumption, right? Very – very! – unlikely in practice. The t-test is sensitive (i.e., liberal) to unequal variances and – to a minor extent – to unequal group sizes. Not by any chance the Welch-test is the default in R and SAS.
Was in the paper an ANOVA (with all arms) used? A pooled variance is just crap. Follow the “Two-at-a-Time” approach, i.e., two analyses with pairwise comparisons. That’s recommended in the latest guidelines (FDA, EMA, ICH M13A).

❝ As there seems to be no solution currently avaiable for adaptive parallel group designs that analytically controls the type-I-error using the confidence interval inclusion approach and allows for unequal variances, …


❝ … wouldn't it be acceptable to use a hypothesis test like in (Maurer 2016) only?

Unlikely, though that’s a Radio Yerewan question.

❝ In that case, we cannot construct confidence intervals consistent with the hypothesis test but the decision for BE=Y/N can be answered, right?

When we had a poster about this stuff (doi:10.1186/1745-6215-16-S2-P218), Franz said “it’s doable in principle”. Well roared, lion. It’s on the todo-list of Benjamin Lang (main author of the inverse normal method in the package Power2Stage). Difficult…

❝ I know that FDA guidelines state that it needs to be done by the confidence interval but is this less preferable than using Potvin's algorithm (not strictly controlling type-I-error)?

I don’t think that any agency will accept a study without a CI.

BTW, the EMA ❤️ a stage-term in the final analysis. Calls for an ANOVA, right? That’s like deciding between Skylla (ANOVA ignoring unequal variances to make regulators happy) and Charybdis (Welch-test given regulators headaches). Michael Tomashevskiy suggested some code a while ago but it’s not implemented in the function power.tsd.p() yet.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Complete thread:

UA Flag
 Admin contact
23,057 posts in 4,840 threads, 1,641 registered users;
79 visitors (0 registered, 79 guests [including 9 identified bots]).
Forum time: 17:11 CEST (Europe/Vienna)

You should treat as many patients as possible with the new drugs
while they still have the power to heal.    Armand Trousseau

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz