Problems with PASS [Software]

posted by Helmut Homepage – Vienna, Austria, 2020-05-15 12:31 (83 d 02:45 ago) – Posting: # 21440
Views: 2,397

Dear all,

a colleague received a sample size estimation of an CRO performed in PASS 15.0.5 for a fully replicated study (TRRT|RTTR), CV 0.50, T/R 0.95, power 0.80, ABE (unscaled). The result was N=54 (power 0.8053).
Since he is a fan of PowerTOST he tried

sampleN.TOST(CV = 0.5, theta0 = 0.95, targetpower = 0.80, design = "2x2x4")

and got

+++++++++++ Equivalence test - TOST +++++++++++
            Sample size estimation
Study design: 2x2x4 (4 period full replicate)
log-transformed data (multiplicative model)

alpha = 0.05, target power = 0.8
BE margins = 0.8 ... 1.25
True ratio = 0.95,  CV = 0.5

Sample size (total)
 n     power

50   0.812806

Lower sample than with PASS (and higher power; with one dropout it will be still >0.80).
54 subjects instead of 50 are good for the CRO, bad for the sponsor. 💰
He suspected that PASS does not use the exact method (default in most function of PowerTOST) but one of the approximations and tried the noncentral t (method = "nct") as well as the shifted central t (method = "shifted"):

sampleN.TOST(CV = 0.5, theta0 = 0.95, targetpower = 0.80, design = "2x2x4",
             method = "nct", print = FALSE, details = FALSE)[7:8]
  Sample size Achieved power
1          50      0.8128063
sampleN.TOST(CV = 0.5, theta0 = 0.95, targetpower = 0.80, design = "2x2x4",
             method = "shifted", print = FALSE, details = FALSE)[7:8]
  Sample size Achieved power
1          50      0.8120118

Then he was worried and sent me an email together with the output of PASS…

We know that in a 2×2×4 design power is approximately equal to a 2×2×2 design with ½ of its sample size because the number of treatments is the same and the differing degrees of freedom play a lesser role. In this case: 98 / 2 = 49 → 50. This approach is used in package bear.
Since I’m not aware of reference tables for replicate design evaluated for ABE, I tried simulations (see this post for the code) and got for simulating statistics

  Sample size Achieved power
1          50        0.81228

and for simulating subjects

  Sample size Achieved power
1          50        0.81231

What the heck? The output of PASS gives a list of references (I numbered the list):
  1. Chow, S.C. and Liu, J.P. 1999. Design and Analysis of Bioavailability and Bioequivalence Studies. Marcel Dekker. New York
  2. Chow, S.C.; Shao, J.; Wang, H. 2003. Sample Size Calculations in Clinical Research. Marcel Dekker. New York.
  3. Chen, K.W.; Chow, S.C.; and Li, G. 1997. 'A Note on Sample Size Determination for Bioequivalence Studies with Higher-Order Crossover Designs.' Journal of Pharmacokinetics and Biopharmaceutics, Volume 25, No. 6, pages 753-765.
#2 is known for many typos; hence, I ignored it.
#3 contains sample size tables and therefore, was a good candidate. Surprise: With increasing CV sample sizes were – generally – larger than expected. Unfortunately the tables don’t go beyond 40%. However, in Table VII 38 subjects are given, whereas I got 34. The underlying ABE-model is not specified; the authors refer to #1. OK, Chapter 9 is it. Gotcha, carry­over in the model! Stephen Senn devoted a good part of his book about crossover studies arguing against it. Not only that carryover is scientifically questionable, none of the guidelines recommend such models. BTW, #1 contains also tables where the sample sizes are (consequently) too large.

Given all that, I recommend toI will download the trial version of PASS2020 to assess it further.*

PS: If you are with a CRO you might be tempted to sell the sponsor large studies. That might backfire like in this case where to sponsor knows PowerTOST

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Complete thread:

 Admin contact
20,969 posts in 4,374 threads, 1,460 registered users;
online 13 (1 registered, 12 guests [including 9 identified bots]).
Forum time: Thursday 15:16 CEST (Europe/Vienna)

It is a good morning exercise for a research scientist
to discard a pet hypothesis every day before breakfast.
It keeps him young.    Konrad Lorenz

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz