Bioequivalence and Bioavailability Forum

❝ I understand your frustration.

❝ Here's a good basis for answering, just remember:

❝

❝ 1. They do not know all the stuff that you know.

❝ 2. You cannot educate them.

❝ The rest is easy.

Assumed CV    : 25.00%
Assumed PE    : 95.00%
Target power  : 90.00%
Sample size   : 38
Achieved power: 90.89%
Dosed         : 44 (anticipated dropout-rate of 10%)
  100,000 simulated 2×2×2 studies
            CV: 13.87 –  40.66% (geom. mean 24.62%)
            PE: 83.65 – 107.59% (geom. mean 94.99%)
            n : 28 – 37 (median 34)

passed BE (90% CI within 80.00 – 125.00%): 98.61%
percentages of passing studies
  with ‘post hoc’ power of <50%:  0.03%
                     ≥50 – <60%:  3.11%
                     ≥60 – <70%:  7.99%
                     ≥70 – <80%: 16.74%
                     ≥80 – <90%: 30.51%
                           >90%: 41.63%
  100% not within CI           :  5.76% (∆ stat. significant)

• The a posteriori power of the study does not need to be calculated.

library(PowerTOST)
balance <- function(n, sequences) {
  # round up to get balanced sequences for potentially unbalanced case
  return (as.integer(sequences * (n %/% sequences + as.logical(n %% sequences))))
}
adjust.dropouts <- function(n, do.rate) {
  # to be dosed subjects which should result in n eligible subjects based on the
  # anticipated droput-rate
  return (as.integer(balance(n / (1 - do.rate), sequences = 2)))
}
set.seed(123456)
nsims   <- 1e5L # number of simulations
target  <- 0.90 # target power
CV      <- 25   # assumed CV
PE      <- 95   # assumed PE
do.rate <- 0.1  # anticipated dropout-rate
CV.do   <- 0.25 # assumed CV of the dropout-rate
tmp     <- sampleN.TOST(CV = CV/100, theta0 = PE/100, targetpower = target,
                        details = FALSE, print = FALSE)
n.des   <- tmp[["Sample size"]]
if (n.des >= 12) {
  power <- tmp[["Achieved power"]]
} else { # acc. to GL
  n.des <- 12
  power <- power.TOST(CV = CV/100, theta0 = PE/100, n = n.des)
}
n.adj   <- adjust.dropouts(n = n.des, do.rate = do.rate)
res     <- data.frame(CV = rep(NA, nsims), n = NA, PE = NA, lower = NA,
                      upper = NA, BE = FALSE, power = NA, signif = FALSE)
post    <- data.frame(sim = 1:nsims, pwr.50minus = FALSE, pwr.60 = FALSE,
                      pwr.70 = FALSE, pwr.80 = FALSE, pwr.90 = FALSE,
                      pwr.90plus = FALSE)
pb      <- txtProgressBar(0, 1, 0, char = "\u2588", width = NA, style = 3)
for (j in 1:nsims) {
  do           <- rlnorm(1, meanlog = log(do.rate) - 0.5*CV2mse(CV.do),
                            sdlog = sqrt(CV2mse(CV.do)))
  res$n[j]     <- as.integer(round(n.des * (1 - do)))
  res$CV[j]    <- 100*mse2CV(CV2mse(CV/100) *
                      rchisq(1, df = res$n[j] - 2)/(res$n[j] - 2))
  res$PE[j]    <- 100*exp(rnorm(1, mean = log(PE/100),
                          sd = sqrt(0.5 / res$n[j]) * sqrt(CV2mse(CV/100))))
  res[j, 4:5]  <- round(100*CI.BE(CV = res$CV[j]/100, pe = res$PE[j]/100,
                                  n = res$n[j]), 2)
  res$power[j] <- suppressMessages(
                    signif(power.TOST(CV = res$CV[j]/100, theta0 = res$PE[j]/100,
                                      n = res$n[j]), 5))
  if (res$lower[j] >= 80 & res$upper[j] <= 125) { # only the ones which pass
    res$BE[j] <- TRUE
    if (res$power[j] < 0.5) post$pwr.50minus[j] <- TRUE
    if (res$power[j] >= 0.5 & res$power[j] < 0.6) post$pwr.60[j] <- TRUE
    if (res$power[j] >= 0.6 & res$power[j] < 0.7) post$pwr.70[j] <- TRUE
    if (res$power[j] >= 0.7 & res$power[j] < 0.8) post$pwr.80[j] <- TRUE
    if (res$power[j] >= 0.8 & res$power[j] < 0.9) post$pwr.90[j] <- TRUE
    if (res$power[j] >= 0.9) post$pwr.90plus[j] <- TRUE
    if (res$lower[j] > 100 | res$upper[j] < 100) res$signif[j] <- TRUE
  }
  setTxtProgressBar(pb, j/nsims)
}
close(pb)
passed <- sum(res$BE)
cat("\nAssumed CV    :", sprintf("%.2f%%", CV),
    "\nAssumed PE    :", sprintf("%.2f%%", PE),
    "\nTarget power  :", sprintf("%.2f%%", 100*target),
    "\nSample size   :", n.des,
    "\nAchieved power:", sprintf("%.2f%%", 100*power),
    "\nDosed         :", n.adj,
    sprintf("(anticipated dropout-rate of %g%%)", 100*do.rate),
    "\n ", formatC(nsims, format = "d", big.mark = ","),
    "simulated 2\u00D72\u00D72 studies",
    "\n            CV:",
    sprintf("%5.2f \u2013 %6.2f%%", min(res$CV), max(res$CV)),
    sprintf("(geom. mean %.2f%%)", exp(mean(log(res$CV)))),
    "\n            PE:",
    sprintf("%5.2f \u2013 %6.2f%%", min(res$PE), max(res$PE)),
    sprintf("(geom. mean %.2f%%)", exp(mean(log(res$PE)))),
    "\n            n :", min(res$n), "\u2013", max(res$n),
    sprintf("(median %g)", median(res$n)),
    "\n\npassed BE (90% CI within 80.00 \u2013 125.00%):",
    sprintf("%5.2f%%", 100*passed/nsims),
    "\npercentages of passing studies",
    "\n  with \u2018post hoc\u2019 power of <50%:",
    sprintf("%5.2f%%", 100*sum(post$pwr.50minus)/passed),
    "\n                     \u226550 \u2013 <60%:",
    sprintf("%5.2f%%", 100*sum(post$pwr.60)/passed),
    "\n                     \u226560 \u2013 <70%:",
    sprintf("%5.2f%%", 100*sum(post$pwr.70)/passed),
    "\n                     \u226570 \u2013 <80%:",
    sprintf("%5.2f%%", 100*sum(post$pwr.80)/passed),
    "\n                     \u226580 \u2013 <90%:",
    sprintf("%5.2f%%", 100*sum(post$pwr.90)/passed),
    "\n                           >90%:",
    sprintf("%5.2f%%", 100*sum(post$pwr.90plus)/passed),
    "\n  100% not within CI           :",
    sprintf("%5.2f%%", 100*sum(res$signif)/passed),
    "(\u2206 stat. significant)\n\n")

❝ The statistically significant formulation effect for AUC_(0-t) is quite common. The sample size is usually estimated with intra-subject CV of C_max (as it is usually higher than of AUC_(0-t)).

❝ To continue with your example:

❝ Assumed intra-subject CV of C_max = 25% -> with assumed PE of 95% for target power 90%: n = 38

❝ Assumed intra-subject CV of AUC_(0-t) e.g. 10%

❝ When assumed parameters used for sample size estimation will be theoretically observed in the study, i.e. the observed GMR will be 95% and observed intra-subect CV will be 10%, we will get 90% CI equal to 91.40-98.74% and statistically significant formulation effect (at a 10% significance level).

❝ If the 90% CI does not contain 100%, the p value will always be <0.1 but how it could further exacerbated the uncertainty about fulfillment of the bioequivalence criteria? (Question for regulators.)

❝ ❝

  100% not within CI           :  5.76% (∆ stat. significant)

❝ I just want to point that it's observed more often for AUC_(0-t) than for C_max and it would be interesting to know the percentage of that also for AUC_(0-t) with lower variability. ;)

Assumed CV (Cmax)    : 25.00%
Assumed CVs (AUC)    : 25.00%, 20.00%, 15.00%, 10.00%
Assumed PE           : 95.00%
Target power         : 90.00%
Sample size          : 38 (based on Cmax)
Achieved power (Cmax): 90.89%
Achieved powers (AUC): 90.89%, 98.05%, 99.95%, 100.00%
Dosed                : 44 (anticipated dropout-rate of 10%)
  100,000 simulated 2×2×2 studies
  n: 28 – 37 (median 34)
  Cmax (25.00%)
    CV       : 13.87 –  40.66% (geom. mean  24.62%)
    PE       : 83.65 – 107.59% (geom. mean  94.99%)
    passed BE: 98.61% (‘empiric power’)
    passing studies with ‘post hoc’ power of <50%:  0.03%
                                       ≥50 – <60%:  3.11%
                                       ≥60 – <70%:  7.99%
                                       ≥70 – <80%: 16.74%
                                       ≥80 – <90%: 30.51%
                                             >90%: 41.63%
    100% not within CI (∆ stat. significant)     :  5.76%
   AUC (25.00%)
    CV       :  12.22 –  41.01% (geom. mean  24.60%)
    PE       :  82.68 – 107.20% (geom. mean  95.00%)
    passed BE:  98.63% (‘empiric power’)
    passing studies with ‘post hoc’ power of <50%:   0.03%
                                       ≥50 – <60%:   3.10%
                                       ≥60 – <70%:   7.74%
                                       ≥70 – <80%:  16.78%
                                       ≥80 – <90%:  30.30%
                                             >90%:  42.05%
    100% not within CI (∆ stat. significant)     :   5.82%
   AUC (20.00%)
    CV       :   9.77 –  31.70% (geom. mean 19.68%)
    PE       :  84.80 – 105.86% (geom. mean 95.00%)
    passed BE:  99.98% (‘empiric power’)
    passing studies with ‘post hoc’ power of <50%:   0.00%
                                       ≥50 – <60%:   0.13%
                                       ≥60 – <70%:   0.62%
                                       ≥70 – <80%:   2.87%
                                       ≥80 – <90%:  12.47%
                                             >90%:  83.92%
    100% not within CI (∆ stat. significant)     :  13.06%
   AUC (15.00%)
    CV       :   7.24 –  24.83% (geom. mean  14.77%)
    PE       :  88.31 – 102.73% (geom. mean  95.01%)
    passed BE: 100.00% (‘empiric power’)
    passing studies with ‘post hoc’ power of <50%:   0.00%
                                       ≥50 – <60%:   0.00%
                                       ≥60 – <70%:   0.00%
                                       ≥70 – <80%:   0.01%
                                       ≥80 – <90%:   0.23%
                                             >90%:  99.75%
    100% not within CI (∆ stat. significant)     :  31.35%
   AUC (10.00%)
    CV       :   4.77 –  15.32% (geom. mean   9.85%)
    PE       :  90.25 – 100.03% (geom. mean  95.00%)
    passed BE: 100.00% (‘empiric power’)
    passing studies with ‘post hoc’ power of <50%:   0.00%
                                       ≥50 – <60%:   0.00%
                                       ≥60 – <70%:   0.00%
                                       ≥70 – <80%:   0.00%
                                       ≥80 – <90%:   0.00%
                                             >90%: 100.00%
    100% not within CI (∆ stat. significant)     :  79.43%

❝ ❝

2. Lack of a posteriori data on the power...

❝ The only good thing on this point is that the regulators believe that test and reference IMPs are bioequivalent. (As they deal with power.)

❝ Posteriori power is never ending story. The question on the power should be raised to the protocol, if it is addressed to the report, it is just a suggestion for future projects.

❝ Additionaly You can remind them what the low power means: With lower power, the type II error (sponsor's risk) is higher - it means that the bioequivalent preparations could be assessed wrongly as not bioequivalent with higher probability. The regulators should sleep well with higher type II error (unless the study was designed for e.g. 50% power from the start - but such protocols should be rejected).

❝

❝ I wish the regulators would be interested also in the type I error - …

❝ … it would mean that regulators do not believe that test and reference IMPs are bioequivalent. The probability of approving non-bioequivalent test product should be up to 5%.

❝ I noticed several studies which suprised me more than this deficiency letter…

1. 1. Pilot:
      Not particularly nice PE though BE seems to be possible in a pivotal study.
      
   2. Pivotal:
      CV similar to the pilot study but the PE moved further away from 100% than in the pilot. Study failed and repeating in a larger sample size (like in your example) was considered futile. Product reformulated and →
2. 1. Pilot:
      CV similar to the others, PE promising this time.
      
   2. Pivotal:
      Passed BE with flying colors.

❝ PT Frustration - not related to vaccine

1. Homework (not for an initiated like you but interested readers): Why pass in the simulations more studies than what we expect for the sample size, or – in other words – why is the ‘empiric power’ higher than planned?
   
2. Oh dear, Bonferroni misused post mortem!
   Nonsense cause the entire α was already spent.