Power limbo [Power / Sample Size]
Hi ElMaestro,
Shit, expected that.
Such a case is not uncommon. Say, you have single dose studies (highest strength, fasting/fed), multiple dose studies with two strengths (fasting/fed). Then you get the n = 6 of my example.
If you want a certain overall power, then each study has to be powered to \(p_i=\sqrt[n]{p_\textrm{overall}}\).
With n = 6, for 80% → 96.35% and for 90% → 98.26%. Will an IEC accept that?*
I know.
I’m not concerned about a new formulation. The first one went to the waste bin → zero consumer risk. The study supporting the new formulation stands on its own. Hence, TIE ≤0.05.
I’m indeed concerned about repeating the study (same formulation) with more subjects. Then you get an inflated TIE for sure. Regulators don’t care. They trust in the second study more because it is larger and thus the result ‘more reliable’, I guess.
We have that everywhere. A value in the post-study exam is clinically significant out of range. Follow-up initiated and now all is good. Did the value really improve? There is always inaccuracy involved. Maybe the first one was correct and the second one not.
❝ Yes to 0.53.
Shit, expected that.
❝ The risk is up to you or your client.
Such a case is not uncommon. Say, you have single dose studies (highest strength, fasting/fed), multiple dose studies with two strengths (fasting/fed). Then you get the n = 6 of my example.
If you want a certain overall power, then each study has to be powered to \(p_i=\sqrt[n]{p_\textrm{overall}}\).
With n = 6, for 80% → 96.35% and for 90% → 98.26%. Will an IEC accept that?*
❝ I think there is no general awareness, but my real worry is the type I error, as I have indicated elsewhere.
I know.
❝ Related issue, the one that worries me more:
❝ You test one formulation, it fails on the 90% CI, you develop a new formulation, it passes on the 90% CI. What is the type I error?
I’m not concerned about a new formulation. The first one went to the waste bin → zero consumer risk. The study supporting the new formulation stands on its own. Hence, TIE ≤0.05.
❝ Well, strictly speaking that would be inflated. But noone seems to give a damn.
I’m indeed concerned about repeating the study (same formulation) with more subjects. Then you get an inflated TIE for sure. Regulators don’t care. They trust in the second study more because it is larger and thus the result ‘more reliable’, I guess.
We have that everywhere. A value in the post-study exam is clinically significant out of range. Follow-up initiated and now all is good. Did the value really improve? There is always inaccuracy involved. Maybe the first one was correct and the second one not.
My doctor gave me six months to live,
but when I couldn’t pay the bill
he gave me six months more. Walter Matthau
- The IEC assesses protocols of single studies. If we want to keep the desired overall power, sample sizes will be substantially higher:
library(PowerTOST)
Number of subjects for given
CV <- seq(0.2, 0.4, 0.1)
ns <- 1:6L
target <- c(0.8, 0.9)
theta0 <- 0.95
design <- "2x2x2"
res <- data.frame(CV = rep(CV, each = length(ns)),
target = rep(target, each = length(CV)*length(ns)),
n.single = NA, pwr.single = NA, studies = ns)
res$target.adj <- res$target^(1/res$studies)
for (j in 1:nrow(res)) {
res[j, 3:4] <- sampleN.TOST(CV = res$CV[j], theta0 = theta0,
design = design,
targetpower = res$target[j],
print = FALSE)[7:8]
tmp <- sampleN.TOST(CV = res$CV[j], theta0 = theta0,
design = design,
targetpower = res$target.adj[j],
print = FALSE)
res$n[j] <- tmp[["Sample size"]]
res$pwr.each[j] <- tmp[["Achieved power"]]^res$studies[j]
}
res$n.incr <- sprintf("%+.1f%%", 100*(res$n - res$n.single) / res$n)
res[, 1:8] <- signif(res[, 1:8], 5)
print(res, row.names = FALSE)
CV target n.single pwr.single studies target.adj n pwr.each n.incr
0.2 0.8 20 0.83468 1 0.80000 20 0.83468 +0.0%
0.2 0.8 20 0.83468 2 0.89443 24 0.80286 +16.7%
0.2 0.8 20 0.83468 3 0.92832 28 0.81705 +28.6%
0.2 0.8 20 0.83468 4 0.94574 30 0.80973 +33.3%
0.2 0.8 20 0.83468 5 0.95635 32 0.81341 +37.5%
0.2 0.8 20 0.83468 6 0.96349 34 0.82384 +41.2%
0.3 0.8 40 0.81585 1 0.80000 40 0.81585 +0.0%
0.3 0.8 40 0.81585 2 0.89443 52 0.81354 +23.1%
0.3 0.8 40 0.81585 3 0.92832 58 0.80091 +31.0%
0.3 0.8 40 0.81585 4 0.94574 64 0.80870 +37.5%
0.3 0.8 40 0.81585 5 0.95635 68 0.80855 +41.2%
0.3 0.8 40 0.81585 6 0.96349 72 0.81548 +44.4%
0.4 0.8 66 0.80525 1 0.80000 66 0.80525 +0.0%
0.4 0.8 66 0.80525 2 0.89443 88 0.81075 +25.0%
0.4 0.8 66 0.80525 3 0.92832 100 0.80737 +34.0%
0.4 0.8 66 0.80525 4 0.94574 108 0.80202 +38.9%
0.4 0.8 66 0.80525 5 0.95635 116 0.80809 +43.1%
0.4 0.8 66 0.80525 6 0.96349 122 0.81015 +45.9%
0.2 0.9 26 0.91763 1 0.90000 26 0.91763 +0.0%
0.2 0.9 26 0.91763 2 0.94868 32 0.92071 +18.8%
0.2 0.9 26 0.91763 3 0.96549 34 0.90766 +23.5%
0.2 0.9 26 0.91763 4 0.97400 36 0.90405 +27.8%
0.2 0.9 26 0.91763 5 0.97915 38 0.90639 +31.6%
0.2 0.9 26 0.91763 6 0.98259 40 0.91230 +35.0%
0.3 0.9 52 0.90197 1 0.90000 52 0.90197 +0.0%
0.3 0.9 52 0.90197 2 0.94868 66 0.90937 +21.2%
0.3 0.9 52 0.90197 3 0.96549 72 0.90304 +27.8%
0.3 0.9 52 0.90197 4 0.97400 78 0.90750 +33.3%
0.3 0.9 52 0.90197 5 0.97915 82 0.90779 +36.6%
0.3 0.9 52 0.90197 6 0.98259 84 0.90158 +38.1%
0.4 0.9 88 0.90041 1 0.90000 88 0.90041 +0.0%
0.4 0.9 88 0.90041 2 0.94868 110 0.90173 +20.0%
0.4 0.9 88 0.90041 3 0.96549 122 0.90008 +27.9%
0.4 0.9 88 0.90041 4 0.97400 132 0.90364 +33.3%
0.4 0.9 88 0.90041 5 0.97915 138 0.90121 +36.2%
0.4 0.9 88 0.90041 6 0.98259 144 0.90267 +38.9%
n.singleCV
andtarget
power in a single study
pwr.single
Achieved power with n.single subjects in a single study
target.adj
Adjusted target power for sample size estimation of multiple studies
n
Number of subjects for givenCV
and adjusted power in each of the studies
pwr.each
Overall power of the studies
n.incr
Penalty in sample size compared to a single study
Example:
We assume CV 20%, T/R-ratio 0.95, and desire a power of ≥80%. Then in a single study with 20 subjects we would achieve a power of ~83.5%. If we want to perform six studies, the overall power would drop to \(\small{0.80^6\approx 26.2\%}\) (total sample size 120). In order to pass all studies with ≥80% power, we have to adjust the target power to \(\small{\sqrt[6]{0.80}=0.96349}\). Now we need 34 subjects in each of the studies (204 in total).
Question: Would a sponsor accept an extremely high risk of failure (73.8%) or prefer a sample size penalty of 70%?
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- Probability to pass multiple studies Helmut 2021-02-19 12:02 [Power / Sample Size]
- Probability to pass multiple studies ElMaestro 2021-02-19 12:57
- Power limboHelmut 2021-02-19 13:37
- Probability to pass multiple studies ? Helmut 2022-06-24 14:03
- Probability to pass multiple studies ElMaestro 2021-02-19 12:57