Alex
☆    

Austria,
2023-12-22 11:08
(179 d 05:16 ago)

Posting: # 23800
Views: 5,072
 

 Parallel Group Adaptive Designs [Two-Stage / GS Designs]

Hi all,

I am wondering whether the following publication made it to your attention: Wynne et al. (2022): A randomized, adaptive design, doubleblind, 3-arm, parallel study assessing the pharmacokinetics and safety of AVT02, a high-concentration (100 mg/mL) Adalimumab biosimilar, in healthy adult subjects (ALVOPAD FIRST). I would love to get your opinion on it.

I was the opinion that only simulation-based methods (Fuglsang 2014 - Sequential Bioequivalence Approaches for Parallel Designs) are currently available/acceptable for parallel group designs due to the complexity of contructing repeated confidence intervals allowing for unequal variances between groups (a requirement for parallel designs according to FDA guidelines). However, in the publication repeated confidence intervals were constructed using the Fisher combination test assuming equal variances (3 parallel treatment arms were analysed using ANOVA). What do you think about it?

I also have another question (likely a stupid one but it is in my head since some time). As there seems to be no solution currently avaiable for adaptive parallel group designs that analytically controls the type-I-error using the confidence interval inclusion approach and allows for unequal variances, wouldn't it be acceptable to use a hypothesis test like in (Maurer 2016 - Controlling the type I error rate in two-stage sequential adaptive designs when testing for average bioequivalence) only? In that case, we cannot construct confidence intervals consistent with the hypothesis test but the decision for BE=Y/N can be answered, right? I know that FDA guidelines state that it needs to be done by the confidence interval but is this less preferable than using Potvin's algorithm (not strictly controlling type-I-error)?


Thanks in advance, any opinion is highly recommended!
Alex
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2023-12-22 13:01
(179 d 03:23 ago)

@ Alex
Posting: # 23801
Views: 4,647
 

 Strong Type I Error control + CI: Not yet…

Hi Alex,

❝ I am wondering whether the following publication made it to your attention: …


No.

❝ I was the opinion that only simulation-based methods (Fuglsang 2014) are currently available/acceptable for parallel group designs due to the complexity of contructing repeated confidence intervals allowing for unequal variances between groups (a requirement for parallel designs according to FDA guidelines).


Right. I had serious problems convincing European regulators even with extensive simulations (unequal variances and/or group sizes due to dropouts). Note that the FDA is fine with simulation-based methods (5th GBHI workshop*).

❝ However, in the publication repeated confidence intervals were constructed using the Fisher combination test assuming equal variances (3 parallel treatment arms were analysed using ANOVA). What do you think about it?


Equal variances are a rather strong assumption, right? Very – very! – unlikely in practice. The t-test is sensitive (i.e., liberal) to unequal variances and – to a minor extent – to unequal group sizes. Not by any chance the Welch-test is the default in R and SAS.
Was in the paper an ANOVA (with all arms) used? A pooled variance is just crap. Follow the “Two-at-a-Time” approach, i.e., two analyses with pairwise comparisons. That’s recommended in the latest guidelines (FDA, EMA, ICH M13A).

❝ As there seems to be no solution currently avaiable for adaptive parallel group designs that analytically controls the type-I-error using the confidence interval inclusion approach and allows for unequal variances, …


Right.

❝ … wouldn't it be acceptable to use a hypothesis test like in (Maurer 2016) only?


Unlikely, though that’s a Radio Yerewan question.

❝ In that case, we cannot construct confidence intervals consistent with the hypothesis test but the decision for BE=Y/N can be answered, right?


When we had a poster about this stuff (doi:10.1186/1745-6215-16-S2-P218), Franz said “it’s doable in principle”. Well roared, lion. It’s on the todo-list of Benjamin Lang (main author of the inverse normal method in the package Power2Stage). Difficult…

❝ I know that FDA guidelines state that it needs to be done by the confidence interval but is this less preferable than using Potvin's algorithm (not strictly controlling type-I-error)?


I don’t think that any agency will accept a study without a CI.

BTW, the EMA ❤️ a stage-term in the final analysis. Calls for an ANOVA, right? That’s like deciding between Skylla (ANOVA ignoring unequal variances to make regulators happy) and Charybdis (Welch-test given regulators headaches). Michael Tomashevskiy suggested some code a while ago but it’s not implemented in the function power.tsd.p() yet.


  • Mehta M, Schug B, Blume HH, Beuerle G, Jiang W, Koenig J, Paixão P, Tampal N, Tsang Y-C, Walstab J, Wedemeyer R, Welink J. The Global Bioequivalence Harmonisation Initiative (GBHI): Report of the fifth international EUFEPS/AAPS conference. Europ. J. Pharm. Sci. 15 August 2023. doi:10.1016/j.ejps.2023.106566. [image] Open access PDF.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Alex
☆    

Austria,
2024-01-08 17:22
(161 d 23:02 ago)

@ Helmut
Posting: # 23822
Views: 4,416
 

 Strong Type I Error control + CI: Not yet…

Hi Helmut,

thanks for confirming my understanding.

Yes, as far as I understood, the paper uses ANOVA, which caused confusion at my side and was the reason for my post. I thought that I may have missed something. But I totally agree with you, the “Two-at-a-Time” approach with unequal variances would have been the better choice (but again leading to the problem of contructing RCIs for the inverse normal combination test and the need to use Potvin's algorithms).

Hope to see you again in person soon.
Alex
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-09 10:48
(161 d 05:36 ago)

@ Alex
Posting: # 23825
Views: 4,381
 

 Strong Type I Error control + CI: Not yet…

Hi Alex,

❝ Yes, as far as I understood, the paper uses ANOVA,…

In the meantime I got it. Correct.

❝ […] the “Two-at-a-Time” approach with unequal variances would have been the better choice (but again leading to the problem of contructing RCIs for the inverse normal combination test and the need to use Potvin's algorithms).

Yep.

❝ Hope to see you again in person soon.

Let’s go for it!

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
roman_max
☆    

Russia,
2024-04-04 17:18
(75 d 00:05 ago)

@ Helmut
Posting: # 23938
Views: 3,539
 

 Strong Type I Error control + CI: Not yet…

Hi Helmut,
hope you are doing fine and my post here fits to the subject.

I`m wondering how to show a power achieved after completion of the first stage in parallel design using Power2Stage functions, according to Potvin C algo?

When I do it for crossover study using interim.tsd.in function I can extract the value of Power Stage 1 component, but how about parallel study?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-04-04 22:16
(74 d 19:08 ago)

@ roman_max
Posting: # 23940
Views: 3,520
 

 Parallel design: Only simulation-based methods so far

Hi roman_max,

❝ hope you are doing fine and my post here fits to the subject.

Yes to both questions.

❝ I`m wondering how to show a power achieved after completion of the first stage in parallel design using Power2Stage functions, according to Potvin C algo?

Power in the interim is only a means of the decision tree (pass or fail in stage 1, continue to stage 2). Simply use the function power.TOST() with the respective alpha (0.0294 for Method B, 0.05 for Method C). Examples with the [image]-script at the end:

n1 <- c(50, 49) # 110 planned, 11 dropouts (different in the groups)
CV <- 0.45      # observed
PE <- 0.93      # observed (worse than the 0.95 assumed)
TSD.par(method = "B", CV, n1, PE)
Parallel design
Method B (‘Type 1’ TSD)
adj   = 0.0294 (both stages)
observed values in stage 1:
   CV = 0.45
   PE = 0.93
   n1 = 50 and 49 / per group (total 99)
interim power with adj 0.0294
not BE in stage 1, 94.12% CI: 78.85% - 109.69%
interim power = 0.4292, initiate stage 2 with 93 subjects


TSD.par(method = "C", CV, n1, PE)
Parallel design
Method C (‘Type 2’ TSD)
alpha = 0.05 (conditional on power)
adj   = 0.0294 (conditional)
observed values in stage 1:
   CV = 0.45
   PE = 0.93
   n1 = 50 and 49 / per group (total 99)
interim power with alpha 0.05
interim power = 0.5648
not BE in stage 1, 94.12% CI: 78.85% - 109.69%
initiate stage 2 with 93 subjects


n1 <- c(90, 89)
CV <- 0.45
PE <- 0.90
TSD.par(method = "B", CV, n1, PE)
Parallel design
Method B (‘Type 1’ TSD)
adj   = 0.0294 (both stages)
observed values in stage 1:
   CV = 0.45
   PE = 0.9
   n1 = 90 and 89 / per group (total 179)
interim power with adj 0.0294
not BE in stage 1, 94.12% CI: 79.66% - 101.69%
interim power = 0.7714, initiate stage 2 with 13 subjects


TSD.par(method = "C", CV, n1, PE)
Parallel design
Method C (‘Type 2’ TSD)
alpha = 0.05 (conditional on power)
adj   = 0.0294 (conditional)
observed values in stage 1:
   CV = 0.45
   PE = 0.9
   n1 = 90 and 89 / per group (total 179)
interim power with alpha 0.05
interim power = 0.8421
BE in stage 1, 90.00% CI: 80.94% - 100.08%

In the first example both methods perform equally. Not BE in stage 1 with the 94.12% CI, and therefore, same sample size in stage 2.
The second example shows why Method C can be more powerful than Method B. Whereas in Method B we fail with the 94.12% CI and therefore, have to initiate stage 2, in Method C we pass already in stage 1 with the 90% CI.

❝ When I do it for crossover study using interim.tsd.in function I can extract the value of Power Stage 1 component, but how about parallel study?

See the script below. However, no exact method is published for a parallel design yet and therefore, we don’t have it in Power2Stage.


TSD.par <- function(method = "B", CV, n1, PE) {
  library(PowerTOST)
  library(Power2Stage)
  # values of Fuglsang’s paper; don’t change!
  alpha  <- 0.05   # for Method C in stage 1
  adj    <- 0.0294 # adjusted alpha (conditionally in Method C, both stages in B)
  # Note: theta0 and target are fixed
  #       CIs in % rounded to two deciaml places acc. to GLs

  theta0 <- 0.95
  target <- 0.80
  info <- paste("\nParallel design")
  if (method == "B") {
    info <- paste(info, "\nMethod B (‘Type 1’ TSD)",
                  "\nadj   =", adj, "(both stages)")
  } else {
    info <- paste(info, "\nMethod C (‘Type 2’ TSD)",
                  "\nalpha =", alpha, "(conditional on power)",
                  "\nadj   =", adj, "(conditional)")
  }
  info <- paste(info, "\nobserved values in stage 1:",
                "\n   CV =", CV,
                "\n   PE =", PE,
                "\n   n1 =", paste(n1, collapse = " and "), "/ per group",
                sprintf("(total %.0f)", sum(n1)),
                "\ninterim power with")
  ifelse (method == "B",
    info <- paste(info, "adj", adj, "\n"),
    info <- paste(info, "alpha", alpha))
  cat(info)
  # the sample size of stage 2 is the same for both methods
  n2 <- sampleN2.TOST(alpha = adj, CV = CV, n1 = sum(n1), theta0 = theta0,
                      targetpower = target, design = "parallel")[["Sample size"]]
  prt.CI <- function(alpha, CI, lf = TRUE) {
    tmp <- sprintf("%.2f%% CI: %.2f%% - %.2f%%",
                   100 * (1 - 2 * alpha), CI[["lower"]], CI[["upper"]])
    if (lf) tmp <- paste0(tmp, "\n")
    return(tmp)
  }
  if (method == "B") {
    # we check for BE with adjusted alpha first
    CI.B <- round(100 * CI.BE(alpha = adj, pe = PE, CV = CV,
                              n = n1, design = "parallel"), 2)
    if (CI.B[["lower"]] >= 80 & CI.B[["upper"]] <= 125) {
      cat("BE in stage 1,", prt.CI(adj, CI.B, TRUE))
    } else {                # failed
      pwrB <- power.TOST(alpha = adj, CV = CV, theta0 = theta0,
                         design = "parallel", n = n1)
      if (pwrB >= target) { # stop because sufficient power
        cat("not BE in stage 1,", prt.CI(adj, CI.B, FALSE),
            "\nstop, because interim power =", signif(pwrB, 4), "\n")
      } else {              # low power → stage 2
        cat("not BE in stage 1,", prt.CI(adj, CI.B, FALSE),
            paste0("\ninterim power = ", signif(pwrB, 4),
                   ", initiate stage 2 with ", n2, " subjects\n"))
      }
    }
  } else {
    # we check power with unadjusted alpha first
    pwrC <- power.TOST(alpha = alpha, CV = CV, theta0 = theta0,
                       design = "parallel", n = n1)
    cat("\ninterim power =", signif(pwrC, 4))
    if (pwrC >= target) {
      # check for BE with unadjusted alpha first
      CI.C1 <- round(100 * CI.BE(alpha = alpha, pe = PE, CV = CV,
                                 n = n1, design = "parallel"), 2)
      if (CI.C1[["lower"]] >= 80 & CI.C1[["upper"]] <= 125) { # pass
        cat("\nBE in stage 1,", prt.CI(alpha, CI.C1, TRUE))
      } else {                                                # fail
        cat("\nnot BE in stage 1,", prt.CI(alpha, CI.C1, TRUE))
      }
    } else {
      # sufficient power, check for BE with adjusted alpha
      CI.C2 <- round(100 * CI.BE(alpha = adj, pe = PE, CV = CV,
                                 n = n1, design = "parallel"), 2)
      if (CI.C2[["lower"]] >= 80 & CI.C2[["upper"]] <= 125) { # pass
        cat("\nBE in stage 1,", prt.CI(adj, CI.C2, TRUE))
      } else {                                                # fail → stage 2
        cat("\nnot BE in stage 1,", prt.CI(adj, CI.C2, FALSE),
            "\ninitiate stage 2 with", n2, "subjects\n")
      }
    }
  }
}


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
roman_max
☆    

Russia,
2024-04-15 23:46
(63 d 17:38 ago)

@ Helmut
Posting: # 23954
Views: 3,080
 

 Parallel design: Only simulation-based methods so far

Hi Helmut,

sorry for my late reply and thanks a lot for very useful information and examples. I succesfully reproduced the code provided and implemented it to my study. Hope the Ministry will be happy as well :)
Achievwin
★    

US,
2024-01-05 12:53
(165 d 03:31 ago)

@ Alex
Posting: # 23819
Views: 4,502
 

 Parallel Group Adaptive Designs

I am thinking of using two stage approach for a parallel BE study (in patients) is there an accepted procedure for sample size calculations? at stage 1 and stage 2?

Any instances where FDA accepted a two stage design methodology for calculating samples size for stage 2 after reviewing stage 1 results?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-05 13:40
(165 d 02:44 ago)

@ Achievwin
Posting: # 23820
Views: 4,478
 

 Simulations highly recommended

Hi Achievwin,

❝ I am thinking of using two stage approach for a parallel BE study (in patients) is there an accepted procedure for sample size calculations? at stage 1 and stage 2?

In general I recommend to perform the first stage in a sample size which gives you reasonably high power to pass already. The second stage is then only a kind of ‘safety net’ if you fail in the first.

❝ Any instances where FDA accepted a two stage design methodology for calculating samples size for stage 2 after reviewing stage 1 results?

It was outlined in a presentation »Uses of Adaptive Design Approach for Generic Bioequivalence Study Submitted in FDA« by Xiaojian Jiang (Deputy Director Division of Bioequivalence II, Office of Bioequivalence, Office of Generic Drugs) at the 5th GBHI (Amsterdam. 28 Sep 2022).

The relevant points:
  • FDA will accept appropriately designed BE studies that are scientifically justified
    • Published literature in peer-reviewed journals in which the applicant’s proposed approach is validated
    • Simulation results meeting desired criteria (e.g., the Type I error probability of the proposed approach is controlled at the nominal level of 0.05 for a BE test
  • The decision to use or not use adaptive design is at the applicant’s discretion
  • Applicants are encouraged to interact with the agency early to discuss their proposed adaptve designs
  • 14 ANDAs submitted to the FDA using two-stage adaptive design approach January 1996 to December 2020
    • Adaptive design has been used for both cross-over study and parallel study
    • Majority of the studies (10 ANDAs) were found acceptable
(my emphases)
If you give me your current email address, I will send you the presentation.

Practically for parallel designs simulations are mandatory because:
  1. The number of subjects in the T- and R-groups (in both stages) will be different due to dropouts.
  2. Variances likely will be different; hence the Welch-Satterthwaite test instead if the common t-test (which is liberal in these cases) has to be used.
  3. Fine for the FDA because no specific statistical model is specified by the agency.
Simulations can be performed in the [image]-package Power2Stage, function power.tsd.p().
Start with a reasonably narrow grid of n1 / CV1 combinations to find a – preliminary – adjusted α which controls the Type I Error. Repeat with some scenarios based on the worst case expected dropout rate (same for T and R, all under T – none under R and vice versa), different CVs (CVT < CVR, CVT > CVR). If the Type I Error is still controlled in all scenarios, fine. If not (likely), adjust more.

One of mine. αadj 0.0274, validated for n1 124–250, homoscedastic CV 50% (our best guess), heteroscedastic (variance ratios 1:4 to 4:1). Maximum empirical Type I Error 0.04987:

[image]

32 pages report with justification, methods, all results, scripts to reproduce them. Not accepted by the EMA because “we don’t like (‼) simulations”…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-08 17:55
(161 d 22:29 ago)

@ Helmut
Posting: # 23823
Views: 4,386
 

 Simulations highly recommended

Thank you for your nice reply.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-09 10:31
(161 d 05:53 ago)

@ Achievwin
Posting: # 23824
Views: 4,374
 

 Simulations mandatory

Hi Achievwin,

❝ Thank you for your nice reply.

Welcome.

Just to be clear: Some authors (right now I’m reviewing a manuscript submitted to Stat Med…) believe (‼) that Pocock’s \(\small{\color{Blue}{\alpha_\text{adj}=0.0294}}\) (94.12% CI) is ‘uni­versally’ valid – even for reference-scaling of HVDs. This is a gross misunderstanding and not even wrong. It was derived for superiority testing in a GSD (fixed sample size N) with one interim at – exactly – N/2 and known variances. The fact that it controlled the Type I Error in Pot­vin’s Method B (TSD in a 2×2×2 design) was a mere lucky punch.
In my example I needed \(\small{\color{Red}{\alpha_\text{adj}=0.0274}}\) (94.52% CI) to control the Type I Error.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-09 22:11
(160 d 18:12 ago)

@ Helmut
Posting: # 23826
Views: 4,298
 

 Simulations mandatory

Hi Helmut:

❝ Just to be clear: Some authors (right now I’m reviewing a manuscript submitted to Stat Med…)"


I and everyone absolutely interested to know essence of that article you are reviewing.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-01-09 23:10
(160 d 17:14 ago)

@ Achievwin
Posting: # 23827
Views: 4,333
 

 Reviews are confidential

Hi Achievwin,

❝ I and everyone absolutely interested to know essence of that article you are reviewing.

Sorry, as long as the review process is ongoing I cannot say anything about it.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
★    

US,
2024-01-19 16:11
(151 d 00:13 ago)

@ Helmut
Posting: # 23833
Views: 4,221
 

 Reviews are confidential

❝ Sorry, as long as the review process is ongoing I cannot say anything about it.


understood and respect that
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-04-04 22:22
(74 d 19:02 ago)

@ Achievwin
Posting: # 23941
Views: 3,503
 

 Not accepted and for a reason…

Hi Achievwin,

the manuscript was not accepted on March 12, 2024. The authors failed to demonstrate that the Type I Error is controlled (see this article why).

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
mittyri
★★  

Russia,
2024-04-05 22:33
(73 d 18:51 ago)

@ Helmut
Posting: # 23942
Views: 3,486
 

 Quod licet Iovi

Hi Helmut,

❝ the manuscript was not accepted on March 12, 2024. The authors failed to demonstrate that the Type I Error is controlled (see this article why).


Didn't they mention this as a magic bullet argument?;-)

Kind regards,
Mittyri
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-04-05 23:50
(73 d 17:34 ago)

@ mittyri
Posting: # 23943
Views: 3,424
 

 Non licet bovi

Hi mittyri,

❝ Didn't they mention this as a magic bullet argument?;-)

That’s crap and the [image]-package is terrible. I tried it. See this rather lengthy thread.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
mittyri
★★  

Russia,
2024-04-06 00:21
(73 d 17:03 ago)

@ Helmut
Posting: # 23944
Views: 3,469
 

 Non licet bovi

Hi Helmut,

❝ That’s crap and the [image]-package is terrible. I tried it. See this rather lengthy thread.

yes, I know, I enjoyed your comprehensive analysis.
But as being said:
A spoken word takes its flight.
Anyone can point to that paper published or point to the conclusion on FDA site - would be difficult to fight against.
I suspect that the probability to have an erratum letter for that paper is near 0.

Kind regards,
Mittyri
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2024-04-06 09:21
(73 d 08:03 ago)

@ mittyri
Posting: # 23945
Views: 3,400
 

 Quod licet bove?

Hi mittyri,

❝ I suspect that the probability to have an erratum letter for that paper is near 0.

That’s an interesting idea. Don’t tempt me! :-D

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
23,057 posts in 4,840 threads, 1,641 registered users;
41 visitors (0 registered, 41 guests [including 6 identified bots]).
Forum time: 17:24 CEST (Europe/Vienna)

You should treat as many patients as possible with the new drugs
while they still have the power to heal.    Armand Trousseau

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5