adjusting α [RSABE / ABEL]
Hi all!
slow code to estimate sample sizes for adjusted α.
I set the expected ratio to 0.9 (more realistic than 0.95 for HVDs/HVDPs). Observations: Sample size penality due to αadj is higher in the fully replicated design. Generally no more penalty for CVWR > ~40%.
Bonus question: αadj depends on CVWR observed in the study, which regulators probably don’t like. In a deficiency letter on Potvin’s method C The Netherland’s MEB stated:
Let’s say in study planning we expect a CVWR of 40% (T/R 0.9, 90% power). In a fully replicate design we would estimate the sample size with 42 – but α will be inflated to 0.05555. So we adjust α to 0.04377 (91.25% CI), which would need 44 subjects. In the study CVWR turned out to be 35%, which would call for a more conservative αadj of 0.03812 (92.38% CI). If we would use the αadj from study planning, we would face an inflation to 0.05681. Maybe it is acceptable to regulators to apply a lower α (wider CI) than in study planning, but what if it is the other way round? If it is not acceptable to use a higher α (narrower CI) we would loose power.
slow code to estimate sample sizes for adjusted α.
library(PowerTOST)
CV <- 0.30 # intra-subject CV of reference
reg <- "EMA" # "EMA" for scABEL or "FDA" for RSABE
des <- "2x2x4" # full: "2x2x4"; partial: "2x3x3"
target<- 0.80 # target power
TR <- 0.90 # expected T/R ratio
alpha <- 0.05 # target alpha
print <- FALSE # intermediate results
nsims <- 1e6
if (reg == "EMA") { # scABEL (rounded switching cond. acc. to GL)
ifelse (CV <= 0.5, GMR <- exp(0.76*CV2se(CV)),
GMR <- exp(0.76*CV2se(0.5)))
if (CV <= 0.3) GMR <- 1.25
} else { # RSABE (exact switching cond.: 0.8925742…)
ifelse (CV > 0.3, GMR <- exp(log(1.25)/0.25*CV2se(CV)),
GMR <- 1.25)
}
if (reg == "EMA") { # first estimate (unadjusted alpha)
res <- sampleN.scABEL(alpha=alpha, theta0=TR, CV=CV,
targetpower=target, design=des,
print=FALSE, details=FALSE)
} else {
res <- sampleN.RSABE(alpha=alpha, theta0=TR, CV=CV,
targetpower=target, design=des,
print=FALSE, details=FALSE)
}
n1 <- res[["Sample size"]]
pwr <- res[["Achieved power"]]
infl <-NULL
if (reg == "EMA") {
pBE <- power.scABEL(alpha=alpha, CV=CV, theta0=GMR,
n=n1, design=des, nsims=nsims)
} else {
pBE <- power.RSABE(alpha=alpha, CV=CV, theta0=GMR,
n=n1, design=des, nsims=nsims)
}
if (pBE > alpha) infl <- "alpha-inflation!"
cat("regulator:", reg, " design:", des,
"\nexpected CV:", CV, "expected PE:", TR,
" target power:", target, " achieved power:", pwr,
"\nn:", n1, " target alpha:", alpha, " pBE at extended limit:",
round(pBE, 5), infl, "\n")
n.adj <- n1 # start of search: sample size with unadjusted alpha
pwr <- 0.5 # that's rather low!
prec <- 1e-8 # precision of bisection algo
iter1 <- 0
while (pwr < target) {
infl <- NULL
iter1 <- iter1 + 1
start <- alpha # unadjusted alpha
stop <- 0.05 # target alpha
nom <- c(0.001, start) # from conservative to unadjusted alpha
lower <- min(nom); upper <- max(nom)
delta <- upper - lower # interval
iter2 <- 0
while (abs(delta) > prec) { # until precision reached
iter2 <- iter2 + 1
adj <- (lower+upper)/2 # bisection
if (reg == "EMA") {
pBE <- power.scABEL(alpha=adj, theta0=GMR, CV=CV,
n=n.adj, design=des, nsims=nsims)
} else {
pBE <- power.RSABE(alpha=adj, theta0=GMR, CV=CV,
n=n.adj, esign=des, nsims=nsims)
}
if (abs(pBE - stop) <= prec) break # precision reached
if (pBE > stop) upper <- adj # move limit downwards
if(pBE < stop) lower <- adj # move limit upwards
delta <- upper - lower # new interval
}
if (reg == "EMA") { # current alpha and power
pwr <- power.scABEL(alpha=adj, theta0=TR, CV=CV,
n=n.adj, design=des, nsims=nsims)
pBE <- power.scABEL(alpha=adj, theta0=GMR, CV=CV,
n=n.adj, design=des, nsims=nsims)
} else {
pwr <- power.RSABE(alpha=adj, theta0=TR, CV=CV,
n=n.adj, design=des, nsims=nsims)
pBE <- power.RSABE(alpha=adj, theta0=GMR, CV=CV,
n=n.adj, design=des, nsims=nsims)
}
if (print) {
cat("iteration:", iter1, " n:", n.adj, " power:", pwr,
" adj. alpha:", adj, "\n")
if (.Platform$OS.type == "windows") flush.console()
}
if (pBE > alpha) infl <- "alpha-inflation!"
if (!is.null(infl) | pwr < target) { # increase sample size if inflation or insufficient power
ifelse (des == "2x2x4", n.adj <- n.adj+2,
n.adj <- n.adj+3)
}
}
if (n.adj > n1) {
cat("regulator:", reg, " design:", des,
"\nexpected CV:", CV, "expected PE:", TR,
" target power:", target, " achieved power:", pwr,
"\nn:", n.adj, " target alpha:", alpha, " adj. alpha:",
round(adj, 5), " pBE at extended limit:", round(pBE, 5), infl,
"\nSample size penalty due to adjusted alpha of",
round(adj, 5), "( ~", round(100*(1-2*adj), 2),
"% confidence interval):\n", n.adj-n1, "(",
round(100*(n.adj-n1)/n1, 1), "% )\n")
} else {
cat("regulator:", reg, " design:", des,
"\nexpected CV:", CV, "expected PE:", TR,
" target power:", target, " achieved power:", pwr,
"\nn:", n.adj, " target alpha:", alpha, " adj. alpha:",
round(adj, 5), " pBE at extended limit:", round(pBE, 5), infl,
"\nNo sample size penalty due to adjusted alpha of", round(adj, 5),
"(~", round(100*(1-2*adj), 2), "% confidence interval)\n")
}
I set the expected ratio to 0.9 (more realistic than 0.95 for HVDs/HVDPs). Observations: Sample size penality due to αadj is higher in the fully replicated design. Generally no more penalty for CVWR > ~40%.
Bonus question: αadj depends on CVWR observed in the study, which regulators probably don’t like. In a deficiency letter on Potvin’s method C The Netherland’s MEB stated:
“Adapting the confidence intervals based upon power is not acceptable and also not in accordance with the EMA guideline. Confidence intervals should be selected a priori, without evaluation of the power.”
(my emphases)Let’s say in study planning we expect a CVWR of 40% (T/R 0.9, 90% power). In a fully replicate design we would estimate the sample size with 42 – but α will be inflated to 0.05555. So we adjust α to 0.04377 (91.25% CI), which would need 44 subjects. In the study CVWR turned out to be 35%, which would call for a more conservative αadj of 0.03812 (92.38% CI). If we would use the αadj from study planning, we would face an inflation to 0.05681. Maybe it is acceptable to regulators to apply a lower α (wider CI) than in study planning, but what if it is the other way round? If it is not acceptable to use a higher α (narrower CI) we would loose power.
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- ‘alpha’ of scaled ABE? d_labes 2013-03-15 15:56 [RSABE / ABEL]
- ‘alpha’ of scaled ABE? Helmut 2013-03-15 17:27
- ’alpha’ of scaled ABE: design d_labes 2013-03-16 20:10
- N=24, 48 d_labes 2013-03-18 08:11
- adaptive design without adjustment martin 2013-03-25 20:47
- α adjustment? Helmut 2013-03-26 15:21
- iteratively adjusted alpha martin 2013-03-26 16:19
- throw away our sample size tables? Helmut 2013-03-26 18:02
- if real martin 2013-03-26 19:44
- regulators don’t care? Helmut 2013-03-26 21:18
- adjusting αHelmut 2013-08-11 16:19
- if real martin 2013-03-26 19:44
- target α in sim’s? Helmut 2013-03-27 15:38
- target α in sim’s? martin 2013-03-27 16:36
- If it is real: results Helmut 2013-03-28 23:20
- having alpha martin 2013-03-28 15:08
- fixed swr Helmut 2013-03-28 23:34
- fixed swr martin 2013-03-29 15:07
- Yes but no but yes but no but… Helmut 2013-03-29 16:10
- fixed swr martin 2013-03-29 15:07
- fixed swr Helmut 2013-03-28 23:34
- target α in sim’s? martin 2013-03-27 16:36
- throw away our sample size tables? Helmut 2013-03-26 18:02
- iteratively adjusted alpha martin 2013-03-26 16:19
- α adjustment? Helmut 2013-03-26 15:21
- adaptive design without adjustment martin 2013-03-25 20:47
- N=24, 48 d_labes 2013-03-18 08:11
- ’alpha’ of scaled ABE: design d_labes 2013-03-16 20:10
- ‘alpha’ of scaled ABE? Helmut 2013-03-15 17:27