Bioequivalence and Bioavailability Forum

Dear all,

in my workshops I try to argue against small pilot studies since – if the uncertainty of the estimated CV is taken into account – the pivotal studies will be larger (see also this thread). Regularly a question is raised:

“We have to pay for both studies. So which is the ‘ideal’ sample size of the pilot?”

Unfortunately it is not that easy. If we define the “ideal” pilot study as the one which results in the minimum total (pilot+pivotal) sample size, we need to have an educated guess about the CV. Only then we could play with this code:

############################################
# ‘Ideal’ sample sizes of pilot studies    #
# based on an educated guess about the CV. #
############################################
require(PowerTOST)
n.min   <- 12   # minimum pilot sample size
n.max   <- 48   # maximum pilot sample size
CV.min  <- 0.1  # minimum expected CV
CV.max  <- 0.3  # maximum expected CV
target  <- 0.8  # target power
GMR     <- 0.95 # expected GMR
alpha   <- 0.05 # type I error probability (default 0.05)
n.pilot <- seq(n.min, n.max, 2)
df      <- n.pilot-2
CV      <- seq(CV.max, CV.min, -0.05)
col     <- colorRampPalette(c("red", "blue"))(length(CV))
for(j in 1:length(CV)) {
  n.pivot <- NULL; n.total <- NULL
  for(k in 1:length(n.pilot)) {
    n <- as.numeric(sampleN.TOST(CV=as.numeric(CVCL(CV=CV[j], df=df[k], side="upper",
           alpha=alpha)[2]), targetpower=target, theta0=GMR, print=F, details=F)[7])
    if(n < 12) n <- 12 # minimum pivotal sample size acc. to GLs
    n.pivot <- c(n.pivot, n)
    n.total <- c(n.total, n.pilot[k]+n.pivot[k])
  }
  if(j == 1) {
    plot(n.pilot, n.total, type="b", pch=16, cex=1.2, xlim=c(n.min, n.max),
    ylim=c(n.min, max(n.total)), xlab="pilot sample size", col=col[j],
    ylab="total (pilot+pivotal) sample size", lwd=2, axes=F, frame.plot=T,
    main="Guessing ‘ideal’ pilot sample sizes (ABE, design 2×2×2)", cex.main=1)
    points(n.pilot, n.pivot, type="b", pch=1, cex=1, lwd=1, col=col[j])
    axis(1, at=seq(n.min, n.max, by=6))
    axis(1, at=seq(n.min+2, n.max-2, by=2), labels=F, tcl=-0.25)
    axis(2, at=seq(n.min, max(n.total), by=12), las=1)
    axis(2, at=seq(n.min+2, max(n.total)-2, by=2), labels=F, tcl=-0.25)
  } else {
  points(n.pilot, n.total, type="b", pch=16, cex=1.2, lwd=2, col=col[j])
  points(n.pilot, n.pivot, type="b", pch=1, cex=1, lwd=1, col=col[j])
  }
  cat(sprintf("%.0f%%:", CV[j]*100), n.total, "\n")
}
legend("bottomright", title=expression(CV[intra]), bg="white", lwd=2,
  legend=sprintf("%.0f%%", CV*100), pch=16, pt.cex=1.2, col=col, seg.len=1.5)



Full circles = total sample sizes, empty circles = pivotal sample sizes.

┌──────────────────────────────────────────────────────────────────┐
│                          pilot sample size                       │
├──────────────────────────────────────────────────────────────────┤
│     12  14 16 18 20 22 24 26 28 30 32 34 36 38 40  42  44  46  48│
├───┬──────────────────────────────────────────────────────────────┤
│CV%│             total (pilot+pivotal) sample size                │
├───┼──────────────────────────────────────────────────────────────┤
│ 30│108 102 98 94 94 92 92 92 92 94 94 96 96 98 98 100 102 104 104│
│ 25│ 80  76 74 72 72 72 72 74 74 76 76 78 80 80 82  84  86  86  88│
│ 20│ 58  56 54 54 54 56 56 58 58 60 62 64 64 66 68  70  72  74  76│
│ 15│ 38  38 38 40 40 42 44 46 46 48 50 52 54 56 58  58  60  62  64│
│ 10│ 26  26 28 30 32 34 36 38 40 42 44 46 48 50 52  54  56  58  60│
└───┴──────────────────────────────────────────────────────────────┘
Clearly for every CV there is a minimum, in others words beyond this value the ‘better’ upper CL of the estimated CV will not pay off any more (the pivotal sample sizes decrease less than sample sizes of the pilot increase). For an expected CV of 20% the ideal pilot sample size is 16–20 subjects (pivotal 34–38 based on the upper CL of the CV of 27.9–29.5%).

Now for the tricky part: In order to play with the minimum we need an educated guess of the CV. Let’s say we assume the CV to be 15–25%. 18 subjects are fine for 20–25% – although 16 would be enough for 15%. If the CV is only 10% 12 would be enough. For 30% 22 would be better.

If you are more adventurous you can increase the α-level. For 0.1 instead of the default 0.05 we get:
┌─────────────────────────────────────────────────────────────┐
│                        pilot sample size                    │
├─────────────────────────────────────────────────────────────┤
│    12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46  48│
├───┬─────────────────────────────────────────────────────────┤
│CV%│           total (pilot+pivotal) sample size             │
├───┼─────────────────────────────────────────────────────────┤
│ 30│90 86 84 84 84 84 84 86 86 88 88 90 90 92 94 96 96 98 100│
│ 25│68 66 66 66 66 66 68 68 70 70 72 74 76 76 78 80 82 84  86│
│ 20│50 48 48 50 50 52 52 54 56 58 58 60 62 64 66 68 70 72  74│
│ 15│34 34 36 36 38 40 42 44 44 46 48 50 52 54 56 58 60 62  64│
│ 10│24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58  60│
└───┴─────────────────────────────────────────────────────────┘

Would a two-stage design help? Maybe. But even then an educated guess of the CV would be helpful designing the study in such a way that we already have a reasonable chance of passing in the first stage.

If you have any hints that the formulation might be highly variable I would opt for a fully replicated design (TRT|RTR or TRTR|RTRT). For some thoughts see this post about applying the lower CL of the CV.