Inflated type I error with fixed widened limits [RSABE / ABEL]
» » ... how could this type I error inflation happen?
» May be Helmut has a better (paedogogical) explanation because he is an exceptionally gifted preacher

Only you say so.

Let’s start with two power curves (first
Imagine an even more extreme case than the one Detlew mentioned. We observe in the study a CVwR of 30.01% although the true one is 30%. That means we will use the widened limits of 75.00–133.33% instead of the correct 80.00–125.00%. With the same \(\small{\theta_0=0.80}\) and \(\small{\theta_0=1.25}\) we jump to the red power curve and therefore, have a much higher chance of passing (~40% instead of 5%).
Now for the tricky part (sorry): In the previous posts we estimated an inflation of the TIE of ~20%. Why not 40%? Say, the true CVwR is 30% and acc. to the convention the drug is considered to be not highly variable. Hence, we would have to apply the conventional limits. There is a ~50% chance that in the actual study we will observe a CVwR >30% and misclassify the drug/formulation as highly variable and apply the widened limits. But there is also a ~50% chance that we observe a CVwR ≤30% and use the conventional limits. Both together gives the ~20% inflation (actually it is more complicated, but let’s leave this aside for the moment).
What if we estimate the sample size for the GCC’s approach? With 28 it will be lower (second
If you think about iteratively adjusted α like for the reference-scaling methods (third
CV = 0.150, n = 12 (power = 0.8496)
TIE = 0.050148: inflated (α-adjustment necessary).
2 iterations: adjusted α = 0.049854 (90.029% CI), TIE = 0.05 (power = 0.8493).
CV = 0.200, n = 18 (power = 0.8015)
TIE = 0.050910: inflated (α-adjustment necessary).
7 iterations: adjusted α = 0.049082 (90.184% CI), TIE = 0.05 (power = 0.7988).
CV = 0.250, n = 28 (power = 0.8220)
TIE = 0.072558: inflated (α-adjustment necessary).
5 iterations: adjusted α = 0.032075 (93.585% CI), TIE = 0.05 (power = 0.7644).
CV = 0.300, n = 28 (power = 0.8079)
TIE = 0.164829: inflated (α-adjustment necessary).
8 iterations: adjusted α = 0.009071 (98.186% CI), TIE = 0.05 (power = 0.5859).
CV = 0.301, n = 28 (power = 0.8085)
TIE = 0.022390: not inflated (no α-adjustment necessary).
However, we could face a substantial loss in power (for CV 30% and the adjusted α of ~0.0091 it would drop from 81% to 59%).
1. Power-curves for fixed limits
library(PowerTOST)
design <- "2x2x4" # any replicate - see known.designs()
CV <- 0.30
target <- 0.80
theta0 <- 0.90
n <- sampleN.TOST(CV = CV, theta0 = theta0, targetpower = target,
design = design, details = FALSE,
print = FALSE)[["Sample size"]]
theta0 <- c(theta0, seq(0.75*0.96, 1, length.out = 60))
theta0 <- sort(unique(c(theta0, 0.8, 1, 1.25, 1/theta0)))
res <- data.frame(theta0 = theta0)
for (j in seq_along(theta0)) {
res$pwr1[j] <- power.TOST(CV = CV, n = n, theta0 = theta0[j],
theta1 = 0.80, design = design)
res$pwr2[j] <- power.TOST(CV = CV, n = n, theta0 = theta0[j],
theta1 = 0.75, design = design)
}
if (is.null(attr(dev.list(), "names")))
windows(width = 4.5, height = 3.3, record = TRUE)
op <- par(no.readonly = TRUE)
par(mar = c(4, 4, 0, 0) + 0.1, cex.axis = 0.8)
plot(log(theta0), res$pwr1, type = "n", axes = FALSE, yaxs = "i",
xlim = log(c(0.75, 1/0.75)), ylim = c(0, 1.04),
xlab = expression(theta[0]), ylab = "power")
axis(2, las = 1)
axis(2, at = 0.05, las = 1)
axis(1, at = log(c(0.75, seq(0.8, 1.2, 0.1), 1.25, 1/0.75)),
labels = sprintf("%.2f", c(0.75, seq(0.8, 1.2, 0.1), 1.25, 1/0.75)))
abline(h = pretty(c(0, 1)), lty = 3, col = "lightgrey")
abline(h = 0.05, lty = 2)
abline(v = log(c(0.9, 1, 1.1, 1.2)), lty = 3, col = "lightgrey")
abline(v = log(c(0.75, 0.8, 1.25, 1/0.75)), lty = 2)
box()
lines(log(theta0), res$pwr1, lwd = 3, col = "blue")
lines(log(theta0), res$pwr2, lwd = 3, col = "red")
par(op)
2. Power curves for sample sizes acc. to the GCC’s approach
library(PowerTOST)
design <- "2x2x4" # only "2x2x4", "2x2x3", "2x3x3" implemented!
CV <- 0.30
target <- 0.80
theta0 <- 0.90
reg <- reg_const("user", r_const = log(1/0.75)/CV2se(0.3),
CVswitch = 0.3, CVcap = 0.3, pe_const = TRUE)
n <- sampleN.scABEL(CV = CV, theta0 = theta0, targetpower = target,
design = design, regulator = reg, details = FALSE,
print = FALSE)[["Sample size"]]
theta0 <- c(theta0, seq(0.75*0.96, 1, length.out = 60))
theta0 <- sort(unique(c(theta0, 0.8, 1, 1.25, 1/theta0)))
res <- data.frame(theta0 = theta0)
for (j in seq_along(theta0)) {
res$pwr[j] <- power.scABEL(CV = CV, n = n, theta0 = theta0[j],
theta1 = 0.80, design = design,
regulator = reg)
}
if (is.null(attr(dev.list(), "names")))
windows(width = 4.5, height = 3.3, record = TRUE)
op <- par(no.readonly = TRUE)
par(mar = c(4, 4, 0, 0) + 0.1, cex.axis = 0.8)
plot(log(theta0), res$pwr, type = "n", axes = FALSE, yaxs = "i",
xlim = log(c(0.75, 1/0.75)), ylim = c(0, 1.04),
xlab = expression(theta[0]), ylab = "power")
axis(2, las = 1)
axis(2, at = 0.05, las = 1)
axis(1, at = log(c(0.75, seq(0.8, 1.2, 0.1), 1.25, 1/0.75)),
labels = sprintf("%.2f", c(0.75, seq(0.8, 1.2, 0.1), 1.25, 1/0.75)))
abline(h = pretty(c(0, 1)), lty = 3, col = "lightgrey")
abline(h = 0.05, lty = 2)
abline(v = log(c(0.9, 1, 1.1, 1.2)), lty = 3, col = "lightgrey")
abline(v = log(c(0.8, 1.25)), lty = 2)
if (CV > 0.3) abline(v = log(c(0.75, 1/0.75)), lty = 2)
box()
lines(log(theta0), res$pwr, lwd = 3, col = "blue")
par(op)
3. Iteratively adjusted α for the GCC’s approach
library(PowerTOST)
opt <- function(x) {
if (sdsims) {
power.scABEL.sds(alpha = x, CV = CV, n = n, theta0 = U, design = design,
regulator = reg, nsims = 1e6, setseed = setseed,
progress = FALSE) - alpha
} else {
power.scABEL(alpha = x, CV = CV, n = n, theta0 = U, design = design,
regulator = reg, nsims = 1e6, setseed = setseed) - alpha
}
}
design <- "2x2x4" # only "2x2x4", "2x2x3", "2x3x3" implemented!
alpha <- 0.05 # nominal level of the test
CV <- 0.30 # can be a 2-element vector: CV[1] for T, CV[2] for R
theta0 <- 0.90 # assumed T/R-ratio
target <- 0.80 # target (desired) power
setseed <- TRUE # for reproducibility
sdsims <- FALSE # set to TRUE for partial replicate (much slower)
reg <- reg_const("user", r_const = log(1/0.75)/CV2se(0.3),
CVswitch = 0.3, CVcap = 0.3, pe_const = TRUE)
# power and sample size for the GCC’s approach
x <- sampleN.scABEL(CV = CV, theta0 = theta0, design = design,
targetpower = target, regulator = reg,
details = FALSE, print = FALSE)
power <- x[["Achieved power"]]
n <- x[["Sample size"]]
U <- scABEL(CV = CV, regulator = reg)[["upper"]]
power <- power.TOST(CV = CV, n = n, theta0 = theta0, design = design)
if (!sdsims) { # simulate underlying statistics
TIE <- power.scABEL(CV = CV, n = n, theta0 = U, design = design,
regulator = reg, nsims = 1e6)
} else { # simulate subject data
TIE <- power.scABEL.sds(CV = CV, n = n, theta0 = U, design = design,
regulator = reg, nsims = 1e6, progress = FALSE)
}
txt <- paste0("CV = ", sprintf("%.3f", CV), ", n = ", n,
" (power = ", sprintf("%.4f)", power))
if (TIE <= alpha) {
txt <- paste0(txt, "\nTIE = ", sprintf("%.6f", TIE), ": not inflated ",
"(no \u03B1-adjustment necessary).\n")
} else {
txt <- paste0(txt, "\nTIE = ", sprintf("%.6f", TIE), ": inflated ",
"(\u03B1-adjustment necessary).")
x <- uniroot(opt, interval = c(0, alpha), tol = 1e-8)
alpha.adj <- x$root
TIE.adj <- alpha + x$f.root
power.adj <- power.TOST(alpha = alpha.adj, CV = CV, n = n, theta0 = theta0,
design = design)
txt <- paste0(txt, "\n", sprintf("%2i", x$iter), " iterations: ",
"adjusted \u03B1 = ", sprintf("%.6f", alpha.adj), " (",
sprintf("%.3f%%", 100*(1-2*alpha.adj)), " CI), TIE = ",
TIE.adj, " (power = ", sprintf("%.4f)", power.adj), ".\n")
}
cat(txt)
Dif-tor heh smusma 🖖
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- Inflated type I error with fixed widened limits? Helmut 2020-11-25 10:50 [RSABE / ABEL]
- Inflated type I error with fixed widened limits? d_labes 2020-11-25 15:07
- Inflated type I error with fixed widened limits? Helmut 2020-11-26 00:09
- Inflated type I error with fixed widened limits d_labes 2020-11-26 15:38
- Inflated type I error: Not nice Helmut 2020-11-26 17:14
- Power of the GCC framework and power of PowerTOST d_labes 2020-11-28 11:20
- Sample sizes (ignoring the inflated TIE) Helmut 2020-11-29 11:21
- Power of the GCC framework and power of PowerTOST d_labes 2020-11-28 11:20
- Inflated type I error: Not nice Helmut 2020-11-26 17:14
- Inflated type I error with fixed widened limits d_labes 2020-11-26 15:38
- Inflated type I error with fixed widened limits? Helmut 2020-11-26 00:09
- Inflated type I error with fixed widened limits? wienui 2020-11-29 14:03
- Inflated type I error with fixed widened limits d_labes 2020-11-29 17:51
- Inflated type I error with fixed widened limitsHelmut 2020-11-30 00:14
- Inflated type I error with fixed widened limits wienui 2020-11-30 03:30
- Houston, we have a problem! Helmut 2020-11-30 14:42
- Paradox of tolerance Astea 2020-12-01 23:34
- Οὐτοπεία ∨ Εὐτοπεία Helmut 2020-12-02 01:37
- Uchronia Astea 2020-12-02 09:46
- Steampunk Helmut 2020-12-02 11:34
- Dieselpunk Astea 2020-12-02 15:28
- Dieselpunk Helmut 2020-12-02 16:18
- Steampunk? OT d_labes 2020-12-02 19:30
- Steampunk? OT Astea 2020-12-02 19:58
- Dieselpunk Astea 2020-12-02 15:28
- Steampunk Helmut 2020-12-02 11:34
- Uchronia Astea 2020-12-02 09:46
- Οὐτοπεία ∨ Εὐτοπεία Helmut 2020-12-02 01:37
- Paradox of tolerance Astea 2020-12-01 23:34
- PowerTOST 1.5.2.9000 on GitHub Helmut 2020-12-23 12:18
- Houston, we have a problem! Helmut 2020-11-30 14:42
- Inflated type I error with fixed widened limits wienui 2020-11-30 03:30
- Inflated type I error with fixed widened limitsHelmut 2020-11-30 00:14
- Inflated type I error with fixed widened limits d_labes 2020-11-29 17:51
- Inflated type I error with fixed widened limits? d_labes 2020-11-25 15:07