Bioequivalence and Bioavailability Forum

Hi Pharma88,

❝ 1. What to conclude if we got the result where study is failed in lower side i.e. below 80%...

❝ 2. What to conclude if we got the result where study is failed in upper side i.e. above 125%...

See this post and scroll down to formula (7.1). The text below (esp. 2. and footnote h.) will answer your questions.

❝ 3. What to conclude where study's post hoc power is less then 80 or 85%?

Nothing useful. Post hoc (a.k.a. a posteriori, retrospective) power is completely irrelevant for the BE assessment. Stop estimating it. See also the vignette of the package PowerTOST.

❝ What are the main factors associated with apart from human or instrument error?

You planned the study based on assumptions (T/R-ratio, variability, dropout-rate) and for a desired (target) power. If at least one of the assumptions is not fulfilled (see the post mentioned above), you may loose power and the chance of failing increases. Even if all assumptions turn out to be exactly realized in study, the chance of failing is \(\small{\beta=1-\pi}\), where \(\small{\beta}\) is the probability of the Type II Error (producer’s risk) and \(\small{\pi}\) the desired power.
In other words, if you plan studies for 80% power, one out of five will fail by pure chance. That’s life.

Simple example in :

library(PowerTOST)
set.seed(123456)
CV      <- 0.25 # assumed CV
theta0  <- 0.95 # assumed T/R-ratio
do.rate <- 0.10 # anticipated dropout rate (10%)
# defaults: targetpower 0.80, design = "2x2"
studies <- 20
N       <- sampleN.TOST(CV = CV, theta0 = theta0, details = FALSE,
                        print = FALSE)[["Sample size"]]
res     <- data.frame(study = c("as planned", 1:studies),
                      CV = c(CV, rnorm(mean = CV, n = studies, sd = 0.05)),
                      theta0 = c(theta0, rnorm(mean = theta0, n = studies, sd = 0.05)),
                      n = c(N, round(runif(n = studies, min = N*(1-do.rate), max = N))),
                      CL.lower = NA, CL.upper = NA, BE = "fail", assessment = NA,
                      power = NA)
for (j in 1:nrow(res)) {
  res[j, 5:6]  <- round(100*CI.BE(CV = res$CV[j], pe = res$theta0[j],
                                  n = res$n[j]), 2)
  if (res$CL.lower[j] >= 80 & res$CL.upper[j] <= 125) {
    res$BE[j]         <- "pass"
    res$assessment[j] <- "equivalent"
  } else {
    if (res$CL.lower[j] > 125 | res$CL.upper[j] < 80) {
      res$assessment[j] <- "inequivalent"
    } else {
      res$assessment[j] <- "indecisive"
    }
  }
  res$power[j] <- suppressMessages(
                     power.TOST(CV = res$CV[j], theta0 = res$theta0[j],
                                n = res$n[j]))
}
res[, c(2:3, 9)] <- signif(res[, c(2:3, 9)], 4)
txt <- paste(sprintf("%1.0f%%", 100*length(which(res$BE == "fail"))/studies),
             "of actual studies failed\n")
cat(txt); print(res, row.names = FALSE)

Gives

25% of actual studies failed
      study     CV theta0  n CL.lower CL.upper   BE assessment  power
 as planned 0.2500 0.9500 28    84.91   106.28 pass equivalent 0.8074
          1 0.2917 0.9123 26    79.66   104.48 fail indecisive 0.4729
          2 0.2362 1.0130 25    90.47   113.39 pass equivalent 0.8923
          3 0.2322 0.9519 28    85.75   105.68 pass equivalent 0.8647
          4 0.2544 0.9595 28    85.60   107.55 pass equivalent 0.8274
          5 0.3626 0.9731 27    82.64   114.59 pass equivalent 0.4513
          6 0.2917 0.9286 26    81.09   106.35 pass equivalent 0.5496
          7 0.3156 0.9508 26    82.15   110.05 pass equivalent 0.5534
          8 0.3751 0.9852 27    83.23   116.63 pass equivalent 0.4154
          9 0.3084 0.9986 27    86.80   114.88 pass equivalent 0.6819
         10 0.2287 0.9190 26    82.56   102.29 pass equivalent 0.6927
         11 0.2002 0.9072 26    82.58    99.67 pass equivalent 0.7181
         12 0.1943 0.9535 26    87.02   104.47 pass equivalent 0.9386
         13 0.2472 0.8977 26    79.97   100.77 fail indecisive 0.5040
         14 0.3087 0.8126 26    70.42    93.76 fail indecisive 0.0712
         15 0.3027 0.8935 26    77.64   102.83 fail indecisive 0.3582
         16 0.2529 0.9069 25    80.38   102.33 pass equivalent 0.5300
         17 0.2132 1.0280 28    93.38   113.17 pass equivalent 0.9547
         18 0.2965 1.0010 27    87.44   114.54 pass equivalent 0.7285
         19 0.3334 1.0020 26    85.91   116.91 pass equivalent 0.5542
         20 0.2780 0.8942 26    78.56   101.78 fail indecisive 0.4108

Little bit cheating because neither the T/R-ratio nor the CV follow a normal distribution. But you get the idea. Some of the studies pass even with low post hoc power. An example is #5, where the “worse” CV is counteracted by a “better” T/R-ratio. The post hoc power of just ~45% is not relevant. It was a lucky punch.
When you increase the number of simulated studies you will sooner or later end up with 20% failing.

Hi Pharma88,

in order to avoid surprises I recommend to perform a sensitivity analysis before designing the study (see also the vignette Power Analysis of PowerTOST).
In order to assess the impact of deviations from assumptions on power try this:

library(PowerTOST)
CV     <- 0.25  # assumed CV
theta0 <- 0.95  # assumed T/R-ratio
target <- 0.80  # target (desired) power
design <- "2x2" # any one given in known.designs()
# default BE limits: theta1 = 0.80, theta2 = 1.25
x      <- pa.ABE(CV = CV, theta0 = theta0,
                 targetpower = target, design = design)
plot(x, pct = FALSE, ratiolabel = "theta0")

Power depends on the CV, the T/R-ratio, and the final sample size of eligible subjects (dosed – dropouts). In each of the panels two parameters are kept constant and the third varied. We see that power is most sensitive to deviations of the T/R-ratio. Much less so to the CV followed by dropouts.

However, this is not the end of the story since potential deviations occur simultaneously. That’s a four-dimensional problem (power depends on theta0, CV, and n). A quick & dirty -script at end.

The lower right quadrants of each panel show “nice” combinations (T/R-ratio > assumed and CV < assumed). Higher power than desired, great. NB sometimes you will see a statistically significant treatment effect. No worries about that (not clinically relevant).
The other combinations are tricky. Since power is most sensitive to the T/R-ratio, it would need a substantially lower CV to compensate for a worse T/R-ratio. Have a look at the 0.80 contour lines in the lower left quadrant of the first panel (no dropouts). Say, the T/R-ratio is just 0.92. Then with any CV > 0.2069 power will be below our target.
On the other hand, “better” T/R-ratios allow for higher CVs. That’s shown in the upper right quadrants. However, if the CV gets too large, even a T/R-ratio of 1 gives not the target power.
In the upper left quadrants are the worst case combinations (T/R-ratio < assumed and CV > assumed). It might still be possible to show BE though with a lower chance (power < 0.80).
Like in the Power Analysis above we see that dropouts don’t hurt that much.
Note that – since power curves are symmetrical in log-scale – you get the same power for \(\small{\theta_0}\) and \(\small{1/\theta_0}\).
With

sensitivity(CV.tgt = CV.tgt, do.rate = do.rate, theta0.lo = 0.8)

you see that power equals the nominal \(\small{\alpha}\) at the lower limit of the acceptance range.

But again, this should be be done before the study.
If you demonstrated BE with a post hoc power < target, all is good (answering the 3^rd question of your post). If post hoc power is substantially lower than desired, you should reconsider your assumptions in designing the next study.

There is simple intuition behind results like these: If my car made it to the top of the hill, then it is powerful enough to climb that hill; if it didn’t, then it obviously isn’t powerful enough. Retrospective power is an obvious answer to a rather uninteresting question. A more meaningful question is to ask whether the car is powerful enough to climb a particular hill never climbed before; or whether a different car can climb that new hill. Such questions are prospective, not retrospective.
— Russell V. Lenth, Two Sample-Size Practices that I Don’t Recommend.

---

library(PowerTOST)
power.TOST.vectorized <- function(CV, theta0, ...) {
  # Supportive function, since only theta0 in power.TOST()
  # can be vectorized (not CV).
  # Returns a matrix of power with rows CV and columns theta0.
  power <- matrix(ncol = length(CV), nrow = length(theta0),
                  dimnames = list(c(paste0("theta0.", 1:length(theta0))),
                                  c(paste0("CV.", 1:length(CV)))))
  for (j in seq_along(CV)) {
    power[, j] <- suppressMessages( # for unbalanced cases
                    power.TOST(CV = CV[j], theta0 = theta0, ...))
  }
  return(power)
}
sensitivity <- function(alpha = 0.05, CV.tgt, CV.lo, CV.hi, theta0.tgt = 0.95,
                        theta0.lo, do.rate, target = 0.80, design = "2x2",
                        theta1, theta2, mesh = 25) {
  # alpha = 0.5 for assessing only the PE (Health Canada: Cmax)
  if (alpha <= 0 | alpha > 0.5)
    stop("alpha ", alpha, " does not make sense.")
  if (missing(CV.tgt))
    stop("CV.tgt must be given.")
  if (missing(CV.lo))
    CV.lo <- CV.tgt * 0.8
  if (missing(CV.hi))
    CV.hi <- CV.tgt * 1.2
  if (theta0.tgt >= 1)
    stop("theta0.tgt >=1 not implemented.")
  if (missing(theta1) & missing(theta2))
    theta1 <- 0.8
  if (!missing(theta1) & missing(theta2))
    theta2 <- 1/theta1
  if (missing(theta1) & !missing(theta2))
    theta1 <- 1/theta2
  if (missing(theta0.lo))
    theta0.lo <- theta0.tgt * 0.95
  if (theta0.lo < theta1) {
    message("theta0.lo ", theta0.lo, "< theta1 does not make sense. ",
            "Changed to theta1.")
    theta0.lo <- theta1
  }
  theta0.hi <- 1
  if (missing(do.rate))
    stop("do.rate must be given.")
  if (do.rate < 0)
    stop("do.rate", do.rate, " does not make sense.")
  if (target <= 0.5)
    stop("Target ", target, " does not make sense. Toss a coin instead.")
  if (target >= 1)
    stop("Target ", target, " does not make sense.")
  d.no <- PowerTOST:::.design.no(design)
  if (is.na(d.no))
    stop("design '", design, "' unknown.")
  if (mesh <= 10) {
    message("Too wide mesh is imprecise. Increased to 25.")
    mesh <- 25
  }
  CV      <- seq(CV.lo, CV.hi, length.out = mesh)
  theta0  <- seq(theta0.lo, theta0.hi, length.out = mesh)
  n       <- sampleN.TOST(alpha = alpha, CV = CV.tgt, theta0 = theta0.tgt,
                          theta1 = theta1, theta2 = theta2,
                          targetpower = target, design = design,
                          details = FALSE)[["Sample size"]]
  ns      <- n:floor(n * (1 - do.rate))
  windows(width = 6.5, height = 6.5)
  fig.col <- ceiling(sqrt(length(ns)))
  fig.row <- ceiling(length(ns)/fig.col)
  figs    <- c(fig.col, fig.row)
  op      <- par(no.readonly = TRUE)
  par(mar = c(3.5, 4, 0.1, 0.3))
  split.screen(figs)
  for (j in seq_along(ns)) {
    power <- power.TOST.vectorized(alpha = alpha, CV = CV, theta0 = theta0,
                                   n = ns[j], design = design)
    pwr   <- suppressMessages(
               power.TOST(alpha = alpha, CV = CV.tgt, theta0 = theta0.tgt,
                                   n = ns[j], design = design))
    screen(j)
    plot(theta0, CV, type = "n", xlab = "", ylab = "", cex.axis = 0.9, las = 1)
    axis(1, at = theta0.tgt, labels = FALSE)
    axis(2, at = CV.tgt, labels = FALSE)
    if (j %%figs[2] == 1) {       # y-label first columns
      mtext("CV", side = 2, line = 3)
    }
    if (j > prod(figs)-figs[2]) { # x-label last row
      mtext(expression(theta[0]), side = 1, line = 2.25)
    }
    grid(); box()
    pwrs <- as.vector(unlist(as.data.frame(power)))
    nl   <- length(pretty(pwrs, 20))
    clr  <- sapply(hcl.pals(type="sequential"),
                   hcl.colors, n = nl, rev = TRUE)[, "ag_Sunset"]
    contour(theta0, CV, power, col = clr, nlevels = nl, labcex = 0.8,
            labels = sprintf("%.2f", pretty(pwrs, nl)), add = TRUE)
    points(theta0.tgt, CV.tgt, cex = 1, pch = 21, col = "blue", bg = "#87CEFA")
    TeachingDemos::shadowtext(theta0.tgt, CV.tgt, col = "blue", bg = "white",
                              labels = paste0(signif(pwr, 3),
                                              " (n ", ns[j], ")"),
                              r = 0.25, adj = c(0.5, 1.6), cex = 0.8)
  }
  close.screen(all = TRUE)
  par(op)
}
#####################################################
# Specification of the study (mandatory values)     #
#####################################################
CV.tgt  <- 0.25 # assumed CV                        #
do.rate <- 0.10 # anticipated dropout-rate (10%)    #
#####################################################
# defaults (if not provided in named arguments)     #
#   alpha      = 0.05            common             #
#   CV.lo      = CV*0.80         best case          #
#   CV.hi      = CV*1.20         worst case         #
#   theta0.tgt = 0.95            assumed T/R-ratio  #
#   theta0.lo  = theta0.tgt*0.95 worst case         #
#   target     = 0.80            target power       #
#   theta1     = 0.80            lower BE limit     #
#   theta2     = 1.25            upper BE limit     #
#   design     = "2x2"           in known.designs() #
#   mesh       = 25              resolution         #
#####################################################
sensitivity(CV.tgt = CV.tgt, do.rate = do.rate)