## Problems with PASS [Software]

Dear all,

a colleague received a sample size estimation of an CRO performed in PASS 15.0.5 for a fully replicated study (TRRT|RTTR), CV 0.50, T/R 0.95, power 0.80, ABE (unscaled). The result was N=54 (power 0.8053).
Since he is a fan of PowerTOST he tried

library(PowerTOST) sampleN.TOST(CV = 0.5, theta0 = 0.95, targetpower = 0.80, design = "2x2x4")

and got

+++++++++++ Equivalence test - TOST +++++++++++             Sample size estimation ----------------------------------------------- Study design: 2x2x4 (4 period full replicate) log-transformed data (multiplicative model) alpha = 0.05, target power = 0.8 BE margins = 0.8 ... 1.25 True ratio = 0.95,  CV = 0.5 Sample size (total)  n     power 50   0.812806

Lower sample than with PASS (and higher power; with one dropout it will be still >0.80).
He suspected that PASS does not use the exact method (default in most function of PowerTOST) but one of the approximations and tried the noncentral t (method = "nct") as well as the shifted central t (method = "shifted"):

 sampleN.TOST(CV = 0.5, theta0 = 0.95, targetpower = 0.80, design = "2x2x4",              method = "nct", print = FALSE, details = FALSE)[7:8]   Sample size Achieved power 1          50      0.8128063 sampleN.TOST(CV = 0.5, theta0 = 0.95, targetpower = 0.80, design = "2x2x4",              method = "shifted", print = FALSE, details = FALSE)[7:8]   Sample size Achieved power 1          50      0.8120118

Then he was worried and sent me an email together with the output of PASS…

We know that in a 2×2×4 design power is approximately equal to a 2×2×2 design with ½ of its sample size because the number of treatments is the same and the differing degrees of freedom play a lesser role. In this case: 98 / 2 = 49 → 50. This approach is used in package bear.
Since I’m not aware of reference tables for replicate design evaluated for ABE, I tried simulations (see this post for the code) and got for simulating statistics

  Sample size Achieved power 1          50        0.81228

and for simulating subjects

  Sample size Achieved power 1          50        0.81231

What the heck? The output of PASS gives a list of references (I numbered the list):
1. Chow, S.C. and Liu, J.P. 1999. Design and Analysis of Bioavailability and Bioequivalence Studies. Marcel Dekker. New York
2. Chow, S.C.; Shao, J.; Wang, H. 2003. Sample Size Calculations in Clinical Research. Marcel Dekker. New York.
3. Chen, K.W.; Chow, S.C.; and Li, G. 1997. 'A Note on Sample Size Determination for Bioequivalence Studies with Higher-Order Crossover Designs.' Journal of Pharmacokinetics and Biopharmaceutics, Volume 25, No. 6, pages 753-765.
#2 is known for many typos; hence, I ignored it.
#3 contains sample size tables and therefore, was a good candidate. Surprise: With increasing CV sample sizes were – generally – larger than expected. Unfortunately the tables don’t go beyond 40%. However, in Table VII 38 subjects are given, whereas I got 34. The underlying ABE-model is not specified; the authors refer to #1. OK, Chapter 9 is it. Gotcha, carry­over in the model! Stephen Senn devoted a good part of his book about crossover studies arguing against it. Not only that carryover is scientifically questionable, none of the guidelines recommend such models. BTW, #1 contains also tables where the sample sizes are (consequently) too large.

Given all that, I recommend to
• neither use PASS (at least for ABE in replicate designs)
• nor the sample size tables in #1 and #3.
I will download the trial version of PASS2020 to assess it further.*

PS: If you are with a CRO you might be tempted to sell the sponsor large studies. That might backfire like in this case where to sponsor knows PowerTOST

• v20.0.1
Splendid.
Sample size (power) for ABE {0.8000|1.2500}, CV 0.50, ratio 0.95, target power 0.80, α 0.15 (!)
Seems that in PASS for the replicate designs the shifted central t-distribution is implemented and for the 2×2×2 the noncentral t (closest match of power). Nothing given in the manual.
TRRT|TRRT
PASS          : 32 (0.8003)
sampleN.TOST(): 30 (0.8100)
TT|RR|TR|RT
PASS          : 232 (0.8019)
sampleN.TOST(): 232 (0.8020)
TTRR|RRTT|TRRT|RTTR
PASS          : 32 (0.8301)
sampleN.TOST(): 32 (0.8302)
TR|RT
PASS          : 58 (0.8005)
sampleN.TOST(): 58 (0.8005)

Even if we consider the crude relationship of the 2-sequence full replicate to the 2×2×2: 58 / 2 = 29 → 30 < 32.

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes