Advantages – performance measure [Two-Stage / GS Designs]

posted by Helmut Homepage – Vienna, Austria, 2015-06-05 15:29 (3219 d 21:53 ago) – Posting: # 14918
Views: 17,957

Dear both!

Liu et al.

"Evaluating the adaptive performance of flexible sample size designs with treatment difference in an interval"

❝ Stat Med 2008; 27:584–596.


❝ But don't ask me for details!


Tricky. Essentially they compare adaptive designs in terms of the expected total sample and power to a fixed sample design. They call the latter “ideal” which is only true for a known CV. Such a comparison is not fair but the only one we probably have. ;-)
Below some stuff (the adjusted HP is 0.0413, not 0.416 – which is for OF). “Type 1”, T/R 0.95, target power 0.80. N is the sample size of the “ideal” design or the expected average sample size in the TSDs. f1 is the ratio of sample sizes and f2 the ratio of expected power:

 CV  method    alpha(s)       N    power   f1    f2  
0.15 fixed     0.05    –      12   0.8305  –      –  
     adjPotvin 0.0302 0.0302  13.5 0.8830 1.123 1.063
     adjHP     0.001  0.0413  14.2 0.8825 1.187 1.063
     adjOF     0.005  0.0416  14.1 0.8738 1.179 1.052
0.25 fixed     0.05    –      28   0.8074   –     –  
     adjPotvin 0.0302 0.0302  32.0 0.8126 1.143 1.006
     adjHP     0.001  0.0413  30.5 0.7967 1.090 0.987
     adjOF     0.005  0.0416  30.4 0.7966 1.087 0.987
0.40 fixed     0.05    –      66   0.8053   –     –  
     adjPotvin 0.0302 0.0302  78.2 0.7500 1.184 0.931
     adjHP     0.001  0.0413  70.7 0.7516 1.071 0.933
     adjOF     0.005  0.0416  70.5 0.7528 1.069 0.935
0.70 fixed     0.05    –     174   0.8031   –     –  
     adjPotvin 0.0302 0.0302 205.6 0.7252 1.181 0.903
     adjHP     0.001  0.0413 185.6 0.7296 1.067 0.908
     adjOF     0.005  0.0416 185.1 0.7293 1.064 0.908

How to calculate their APS (average performance score) combining the sample size and power is beyond me.
I think there is no general rule (i.e., independent from the expected CV and planned stage 1 sample size) to judge which method performs “best”. For low CVs the larger average sample size may be outweighed by higher power (leaving some “headroom” for the ratio). If the CV is higher than ~0.25 any TSD with such a small n1 will lack power. Let’s play the game with an expected CV of 0.3 and an n1 of 24:
method    alpha(s)       N    power   f1    f2 
fixed     0.05    –      40   0.8158  –      – 
adjPotvin 0.0302 0.0302  39.4 0.8284 0.984 1.015
adjHP     0.001  0.0413  42.3 0.8110 1.057 0.994
adjOF     0.005  0.0416  41.9 0.8104 1.048 0.993

The winner is the symmetric split for my personal favorite n1 ~0.75 of fixed.
Furthermore, in TSDs the distribution of expected total sample sizes is not necessarily nor­mal. Actually it might be even bimodal (with increasing α1 the distribution gets “con­ta­mi­nated” by the fraction of studies stopping in the first stage). 105 sim’s each:


[image]

Whether it makes sense to compare designs based on the arithmetic mean (or even the median) remains an open issue.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Complete thread:

UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
108 visitors (0 registered, 108 guests [including 9 identified bots]).
Forum time: 12:23 CET (Europe/Vienna)

With four parameters I can fit an elephant,
and with five I can make him wiggle his trunk.    John von Neumann

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5