Alex
☆    

Austria,
2023-12-22 11:08
(72 d 16:27 ago)

Posting: # 23800
Views: 1,022
 

 Parallel Group Adaptive Designs [Two-Stage / GS Designs]

Hi all,

I am wondering whether the following publication made it to your attention: Wynne et al. (2022): A randomized, adaptive design, doubleblind, 3-arm, parallel study assessing the pharmacokinetics and safety of AVT02, a high-concentration (100 mg/mL) Adalimumab biosimilar, in healthy adult subjects (ALVOPAD FIRST). I would love to get your opinion on it.

I was the opinion that only simulation-based methods (Fuglsang 2014 - Sequential Bioequivalence Approaches for Parallel Designs) are currently available/acceptable for parallel group designs due to the complexity of contructing repeated confidence intervals allowing for unequal variances between groups (a requirement for parallel designs according to FDA guidelines). However, in the publication repeated confidence intervals were constructed using the Fisher combination test assuming equal variances (3 parallel treatment arms were analysed using ANOVA). What do you think about it?

I also have another question (likely a stupid one but it is in my head since some time). As there seems to be no solution currently avaiable for adaptive parallel group designs that analytically controls the type-I-error using the confidence interval inclusion approach and allows for unequal variances, wouldn't it be acceptable to use a hypothesis test like in (Maurer 2016 - Controlling the type I error rate in two-stage sequential adaptive designs when testing for average bioequivalence) only? In that case, we cannot construct confidence intervals consistent with the hypothesis test but the decision for BE=Y/N can be answered, right? I know that FDA guidelines state that it needs to be done by the confidence interval but is this less preferable than using Potvin's algorithm (not strictly controlling type-I-error)?


Thanks in advance, any opinion is highly recommended!
Alex
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2023-12-22 13:01
(72 d 14:34 ago)

@ Alex
Posting: # 23801
Views: 890
 

 Strong Type I Error control + CI: Not yet…

Hi Alex,

❝ I am wondering whether the following publication made it to your attention: …


No.

❝ I was the opinion that only simulation-based methods (Fuglsang 2014) are currently available/acceptable for parallel group designs due to the complexity of contructing repeated confidence intervals allowing for unequal variances between groups (a requirement for parallel designs according to FDA guidelines).


Right. I had serious problems convincing European regulators even with extensive simulations (unequal variances and/or group sizes due to dropouts). Note that the FDA is fine with simulation-based methods (5th GBHI workshop*).

❝ However, in the publication repeated confidence intervals were constructed using the Fisher combination test assuming equal variances (3 parallel treatment arms were analysed using ANOVA). What do you think about it?


Equal variances are a rather strong assumption, right? Very – very! – unlikely in practice. The t-test is sensitive (i.e., liberal) to unequal variances and – to a minor extent – to unequal group sizes. Not by any chance the Welch-test is the default in R and SAS.
Was in the paper an ANOVA (with all arms) used? A pooled variance is just crap. Follow the “Two-at-a-Time” approach, i.e., two analyses with pairwise comparisons. That’s recommended in the latest guidelines (FDA, EMA, ICH M13A).

❝ As there seems to be no solution currently avaiable for adaptive parallel group designs that analytically controls the type-I-error using the confidence interval inclusion approach and allows for unequal variances, …


Right.

❝ … wouldn't it be acceptable to use a hypothesis test like in (Maurer 2016) only?


Unlikely, though that’s a Radio Yerewan question.

❝ In that case, we cannot construct confidence intervals consistent with the hypothesis test but the decision for BE=Y/N can be answered, right?


When we had a poster about this stuff (doi:10.1186/1745-6215-16-S2-P218), Franz said “it’s doable in principle”. Well roared, lion. It’s on the todo-list of Benjamin Lang (main author of the inverse normal method in the package Power2Stage). Difficult…

❝ I know that FDA guidelines state that it needs to be done by the confidence interval but is this less preferable than using Potvin's algorithm (not strictly controlling type-I-error)?


I don’t think that any agency will accept a study without a CI.

BTW, the EMA ❤️ a stage-term in the final analysis. Calls for an ANOVA, right? That’s like deciding between Skylla (ANOVA ignoring unequal variances to make regulators happy) and Charybdis (Welch-test given regulators headaches). Michael Tomashevskiy suggested some code a while ago but it’s not implemented in the function power.tsd.p() yet.


  • Mehta M, Schug B, Blume HH, Beuerle G, Jiang W, Koenig J, Paixão P, Tampal N, Tsang Y-C, Walstab J, Wedemeyer R, Welink J. The Global Bioequivalence Harmonisation Initiative (GBHI): Report of the fifth international EUFEPS/AAPS conference. Europ. J. Pharm. Sci. 15 August 2023. doi:10.1016/j.ejps.2023.106566. [image] Open access PDF.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Alex
☆    

Austria,
2024-01-08 17:22
(55 d 10:14 ago)

@ Helmut
Posting: # 23822
Views: 637
 

 Strong Type I Error control + CI: Not yet…

Hi Helmut,

thanks for confirming my understanding.

Yes, as far as I understood, the paper uses ANOVA, which caused confusion at my side and was the reason for my post. I thought that I may have missed something. But I totally agree with you, the “Two-at-a-Time” approach with unequal variances would have been the better choice (but again leading to the problem of contructing RCIs for the inverse normal combination test and the need to use Potvin's algorithms).

Hope to see you again in person soon.
Alex
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-09 10:48
(54 d 16:48 ago)

@ Alex
Posting: # 23825
Views: 597
 

 Strong Type I Error control + CI: Not yet…

Hi Alex,

❝ Yes, as far as I understood, the paper uses ANOVA,…

In the meantime I got it. Correct.

❝ […] the “Two-at-a-Time” approach with unequal variances would have been the better choice (but again leading to the problem of contructing RCIs for the inverse normal combination test and the need to use Potvin's algorithms).

Yep.

❝ Hope to see you again in person soon.

Let’s go for it!

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-05 12:53
(58 d 14:43 ago)

@ Alex
Posting: # 23819
Views: 698
 

 Parallel Group Adaptive Designs

I am thinking of using two stage approach for a parallel BE study (in patients) is there an accepted procedure for sample size calculations? at stage 1 and stage 2?

Any instances where FDA accepted a two stage design methodology for calculating samples size for stage 2 after reviewing stage 1 results?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-05 13:40
(58 d 13:56 ago)

@ Achievwin
Posting: # 23820
Views: 693
 

 Simulations highly recommended

Hi Achievwin,

❝ I am thinking of using two stage approach for a parallel BE study (in patients) is there an accepted procedure for sample size calculations? at stage 1 and stage 2?

In general I recommend to perform the first stage in a sample size which gives you reasonably high power to pass already. The second stage is then only a kind of ‘safety net’ if you fail in the first.

❝ Any instances where FDA accepted a two stage design methodology for calculating samples size for stage 2 after reviewing stage 1 results?

It was outlined in a presentation »Uses of Adaptive Design Approach for Generic Bioequivalence Study Submitted in FDA« by Xiaojian Jiang (Deputy Director Division of Bioequivalence II, Office of Bioequivalence, Office of Generic Drugs) at the 5th GBHI (Amsterdam. 28 Sep 2022).

The relevant points:
  • FDA will accept appropriately designed BE studies that are scientifically justified
    • Published literature in peer-reviewed journals in which the applicant’s proposed approach is validated
    • Simulation results meeting desired criteria (e.g., the Type I error probability of the proposed approach is controlled at the nominal level of 0.05 for a BE test
  • The decision to use or not use adaptive design is at the applicant’s discretion
  • Applicants are encouraged to interact with the agency early to discuss their proposed adaptve designs
  • 14 ANDAs submitted to the FDA using two-stage adaptive design approach January 1996 to December 2020
    • Adaptive design has been used for both cross-over study and parallel study
    • Majority of the studies (10 ANDAs) were found acceptable
(my emphases)
If you give me your current email address, I will send you the presentation.

Practically for parallel designs simulations are mandatory because:
  1. The number of subjects in the T- and R-groups (in both stages) will be different due to dropouts.
  2. Variances likely will be different; hence the Welch-Satterthwaite test instead if the common t-test (which is liberal in these cases) has to be used.
  3. Fine for the FDA because no specific statistical model is specified by the agency.
Simulations can be performed in the [image]-package Power2Stage, function power.tsd.p().
Start with a reasonably narrow grid of n1 / CV1 combinations to find a – preliminary – adjusted α which controls the Type I Error. Repeat with some scenarios based on the worst case expected dropout rate (same for T and R, all under T – none under R and vice versa), different CVs (CVT < CVR, CVT > CVR). If the Type I Error is still controlled in all scenarios, fine. If not (likely), adjust more.

One of mine. αadj 0.0274, validated for n1 124–250, homoscedastic CV 50% (our best guess), heteroscedastic (variance ratios 1:4 to 4:1). Maximum empirical Type I Error 0.04987:

[image]

32 pages report with justification, methods, all results, scripts to reproduce them. Not accepted by the EMA because “we don’t like (‼) simulations”…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-08 17:55
(55 d 09:40 ago)

@ Helmut
Posting: # 23823
Views: 618
 

 Simulations highly recommended

Thank you for your nice reply.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-09 10:31
(54 d 17:04 ago)

@ Achievwin
Posting: # 23824
Views: 590
 

 Simulations mandatory

Hi Achievwin,

❝ Thank you for your nice reply.

Welcome.

Just to be clear: Some authors (right now I’m reviewing a manuscript submitted to Stat Med…) believe (‼) that Pocock’s \(\small{\color{Blue}{\alpha_\text{adj}=0.0294}}\) (94.12% CI) is ‘uni­versally’ valid. This is a gross misunderstanding and not even wrong. It was derived for superiority testing in a GSD (fixed sample size N) with one interim at – exactly – N/2 and known variances. The fact that it controlled the Type I Error in Pot­vin’s Method B (TSD in a 2×2×2 design) was a mere lucky punch.
In my example I needed \(\small{\color{Red}{\alpha_\text{adj}=0.0274}}\) (94.52% CI) to control the Type I Error.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-09 22:11
(54 d 05:24 ago)

@ Helmut
Posting: # 23826
Views: 556
 

 Simulations mandatory

Hi Helmut:

❝ Just to be clear: Some authors (right now I’m reviewing a manuscript submitted to Stat Med…)"


I and everyone absolutely interested to know essence of that article you are reviewing.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-09 23:10
(54 d 04:26 ago)

@ Achievwin
Posting: # 23827
Views: 566
 

 Reviews are confidential

Hi Achievwin,

❝ I and everyone absolutely interested to know essence of that article you are reviewing.

Sorry, as long as the review process is ongoing I cannot say anything about it.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-19 16:11
(44 d 11:25 ago)

@ Helmut
Posting: # 23833
Views: 433
 

 Reviews are confidential

❝ Sorry, as long as the review process is ongoing I cannot say anything about it.


understood and respect that
UA Flag
Activity
 Admin contact
22,919 posts in 4,807 threads, 1,643 registered users;
28 visitors (0 registered, 28 guests [including 7 identified bots]).
Forum time: 03:36 CET (Europe/Vienna)

The analysis of variance is not a mathematical theorem,
but rather a convenient method of arranging the arithmetic.    R.A. Fisher

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5