Bioequivalence and Bioavailability Forum

To whom it may concern!

Since Two-Stage Designs were mentioned in various guidelines (EMA, FDA,…) sponsors want to combine them with options for HVDs/HVDPs (EMA: ABEL, FDA: RSABE). I do understand that this a tempting idea. What I have heard at recent conferences gives me the impression that sponsors already performed such studies. I don’t know whether any one of them was accepted. I know of one leading to a deficiency letter.

I will summarize the caveats.

• Reference-scaling might lead to an inflation of the type I error (TIE) – caused by modifying the Null-Hy­po­thesis “in the face of the data”. The method is intrinsically sequential:
  • Estimate the intra-subject variability of the reference.
    
  • Eventually widen the acceptance-range (EMA) or assess the linearized criterion (FDA).
• Chow and Liu¹ state:
  […] the equivalence limits for the scaled ABE involve the unknown intra-subject variability. As a result, the equivalence limits become random variables and are not fixed constants and the vari­ability introduced by scaling will have an impact on the type I error rate. Therefore, the attempt to use the scaled ABE for re­so­lution of high variable drug products will face the similar issues and challenges that the individual and population bioequivalence encountered in the 1990s […]. These issues must be satisfactorily addressed before the scaled ABE can be implemented into regulatory framework.
  
• Inflation of the TIE was already reported by Endrényi and Tóthfalusi² and recently demonstrated by Won­ne­mann et al.³ for the EMA’s method by mans of simulations.
  
• So far Two-Stage Sequential methods are only published for 2×2×2 crossovers and studies in a two-group parallel design. See the review⁴ for references. With one exception (“Type 2” TSDs showing BE in the first stage with at least the target power) all of them require an adjustment of the nominal α in order to maintain the overall TIE at ≤0.05.
  Though attempts have been made to adjust α based on the first stage’s data^5,6 regulatory accept­ance is prob­lem­atic, since – at least for the EMA – the adjusted level has to be pre-specified in the protocol.
  
• Until somebody comes up with a clever algo and publishes it, the combination of TSDs with refe­rence-scaling remains an open issue. It boils down to a Radio Yerevan’s answer: “In principle yes, but you have to demonstrate that the patient’s risk is maintained.”
  If one wants to pre-specify an adjusted α (GL…) one has to (empirically) find one which keeps the TIE for all possible combinations of stage 1 sample sizes and CVs. Currently I use a matrix with ~1,000 combos (n₁ and CV with a step size of 2). For every grid-point one has to simulate 10⁶ BE-studies for the TIE (slow con­ver­gence) and 10⁵ for power. This requires 1.1×10⁹ simulations.
  Contrary to 2×2×2 and parallel designs – where power can be directly calculated – for scaling we need simu­la­tions since the limits are not fixed, we have a restriction on the GMR, and an upper cap at 50% for EMA. The convergence is fine, but we still need 10⁵ simulations within every simulated study. This leads to ~10¹⁴ (110,000,000,000,000‼) simulations overall.
  To give you an idea:
  
  run0 <- 1e7; run  <- 1:run0
ptm  <- proc.time() # time the loop's overhead
for(j in seq_along(run)) { }
t0   <- as.numeric(1000*(proc.time()[3]-ptm[3])/run0)
run1 <- 1e4; run  <- 1:run1
ptm  <- proc.time()
for(j in seq_along(run)) {
  power.TOST(CV=0.4, theta0=0.9, design="2x2x2", n=134, method="nct") }
t1   <- as.numeric(1000*(proc.time()[3]-ptm[3])/run1-t0)
run2 <- 1e3; run  <- 1:run2
ptm  <- proc.time() # ABEL (EMA)
for(j in seq_along(run)) {
  power.scABEL(CV=0.4, theta0=0.9, design="2x2x4", n=30, nsims=1e5) }
t2   <- as.numeric(1000*(proc.time()[3]-ptm[3])/run2-t0)
run3 <- 1e3; run  <- 1:run3
ptm  <- proc.time() # RSABE (FDA)
for(j in seq_along(run)) {
  power.RSABE(CV=0.4, theta0=0.9, design="2x2x4", n=24, nsims=1e5) }
t3   <- as.numeric(1000*(proc.time()[3]-ptm[3])/run3-t0)
cat("Runtimes of PowerTOST's functions:\n",
sprintf("%s %5.2f %s", "2x2x2 ABE        :", t1, "ms\n"),
sprintf("%s %5.2f %s (%1.0fx slower)%s", "2x2x4 ABEL  (EMA):", t2, "ms", t2/t1, "\n"),
sprintf("%s %5.2f %s (%1.0fx slower)%s", "2x2x4 RSABE (FDA):", t3, "ms", t3/t1, "\n"))
  
  Gives on my machine:
  
  Runtimes of PowerTOST's functions:
 2x2x2 ABE        :  2.22 ms
 2x2x4 ABEL  (EMA): 82.82 ms (37x slower)
 2x2x4 RSABE (FDA): 80.45 ms (36x slower)
  
  Detlew did a great job. The scaled power-functions are not by a factor of 10⁵ slower – only ~40times. If you have a lot of time, go ahead and become famous.
  Furthermore, the intra-subject variabilities of test and reference must not be identical. Keep that in mind.

---

Edit: Easier with the package microbenchmark:

library(microbenchmark)
library(PowerTOST)
microbenchmark(
  power.TOST(CV=0.4,   theta0=0.9, design="2x2x2", n=134, method="nct"),
  power.scABEL(CV=0.4, theta0=0.9, design="2x2x4", n= 30, nsims=1e5),
  power.RSABE(CV=0.4,  theta0=0.9, design="2x2x4", n= 24, nsims=1e5))

On my machine (slow because I have 8 R-sessions running; CPU-load >95%):

Unit: milliseconds
          expr       min         lq      mean     median         uq       max neval
power.TOST()    3.703643   4.336665   5.82738   4.784337   5.475469  67.81569   100
power.scABEL() 84.496413  91.930565 120.51301  97.457502 156.742944 176.92144   100
power.RSABE()  91.391727 103.207520 139.51786 155.408528 167.997411 205.35759   100

---

References:
1. Chow S-C, Liu J-p. Design and Analysis of Bioavailability and Bioequivalence Studies. Boca Raton: Chapman & Hall/CRC; 3^rd ed 2009: p. 598.
   
2. Endrényi L, Tóthfalusi L. Regulatory conditions for the determination of bioequivalence of highly variable drugs. J Pharm Pharm Sci. 2009;12:138–49.  free resource.
   
3. Wonnemann M, Frömke C, Koch A. Inflation of the Type I Error: Investigations on Regulatory Recommendations for Bioequivalence of Highly Variable Drugs. Pharm Res. 2015;32(1):135–43. doi:10.1007/s11095-014-1450-z.
   
4. Schütz H. Two-stage designs in bioequivalence trials. Eur J Clin Pharmacol. 2015;71(3):271-81. doi:10.1007/s00228-015-1806-2.
   
5. Fuglsang A. Controlling type I errors for two-stage bioequivalence study designs. Clin Res Reg Aff. 2011;28(4):100–5. doi:10.3109/10601333.2011.631547.
   
6. Kieser M, Rauch G. Two-stage designs for cross-over bioequivalence trials. Stat Med. 2015;34(16):2403–16. doi:10.1002/sim.6487.