Yvonne
☆    

2012-08-02 11:50
(4707 d 08:24 ago)

Posting: # 9018
Views: 16,319
 

 two-stage design power 90% second stage [Two-Stage / GS Designs]

Post reply
Dear all,

Can someone help me with the following question. We are currently planning for a two-stage design (potvin method C).

My question is, if we go into the second stage and we have to do a sample size calculation, can we use a power of 90% (instead of 80%)? And of course more importantly :-D, would a sample size calculation based on 90% power be accepted by the authorities?
In theory the power is the companies risk and not the consumers risk and thus no extra inflation of alpha would occur in a two stage design (because of a different use of power) or am I wrong here :confused:.


Regards,
Yvonne
 
ElMaestro
★★★

Denmark,
2012-08-02 12:17
(4707 d 07:57 ago)

@ Yvonne
Posting: # 9019
Views: 15,317
 

 two-stage design power 90% second stage

Post reply
Hello Yvonne,

❝ My question is, if we go into the second stage and we have to do a sample size calculation, can we use a power of 90% (instead of 80%)? And of course more importantly :-D, would a sample size calculation based on 90% power be accepted by the authorities?


Yes, this will be acceptable to many authorities.

❝ In theory the power is the companies risk and not the consumers risk and thus no extra inflation of alpha would occur in a two stage design (because of a different use of power) or am I wrong here :confused:.


You probably need to prove that the overall alpha is preserved under or at 0.05. A simulation study is recommended for this purpose.
—
Pass or fail!
ElMaestro
 
Yvonne
☆    

2012-08-02 13:47
(4707 d 06:27 ago)

@ ElMaestro
Posting: # 9022
Views: 15,065
 

 two-stage design power 90% second stage

Post reply
Thank you very much for your answer.

❝ You probably need to prove that the overall alpha is preserved under or at 0.05. A simulation study is recommended for this purpose.


I am not very familiar with simulations. I can uderstand that this is very complicated and might take long.

So in practice there are 2 (or 3) possibilities?
option a: a simulation is not that difficult and can be done relatively "easy" with some help (and of course accepted by authorities :-D). Then I think that this one is prefered

option b: simulations are relatively difficult and without knowing what will happen with the inflation of alpha, the only power that will be accepted by the authorities is 80%

or option c: simulations are relatively difficult but authorites will accept also sample size calculation in the second stage with 90% power (which I hope is the answer :-)).

Can you or someone else help me with this?

Kind regards,

Yvonne
 
ElMaestro
★★★

Denmark,
2012-08-02 13:54
(4707 d 06:20 ago)

@ Yvonne
Posting: # 9023
Views: 15,037
 

 two-stage design power 90% second stage

Post reply
Hello Yvonne,

❝ or option c: simulations are relatively difficult but authorites will accept also sample size calculation in the second stage with 90% power (which I hope is the answer :-)).


Yes, I think C is the right one here.
What I would do if I were you: Make a series of simulations that show that a target power of 0.9 adequately controls alpha at or below 0.05 under your conditions (do it for a good range of CV's and N1's) and show this documentation to regulators. A sc. advice is of course a good idea. They may ask for a copy of your software, e.g. to make sure it actually reproduces something known (like some of the values of Potvin 2008).
—
Pass or fail!
ElMaestro
 
d_labes
★★★

Berlin, Germany,
2012-08-02 12:33
(4707 d 07:41 ago)

@ Yvonne
Posting: # 9020
Views: 15,215
 

 two-stage design power 90% in sample size adaption

Post reply
Dear Yvonne,

❝ Can someone help me with the following question. We are currently planning for a two-stage design (potvin method C).


If you go for an EMA submission: Seems there is an Potvin C abhorrence. See Huanghe's and Helmut's discussion.

❝ My question is, if we go into the second stage and we have to do a sample size calculation, can we use a power of 90% (instead of 80%)? And of course more importantly :-D, would a sample size calculation based on 90% power be accepted by the authorities?

❝ In theory the power is the companies risk and not the consumers risk and thus no extra inflation of alpha would occur in a two stage design (because of a different use of power) or am I wrong here :confused:.


Using 90% power in the sample size adaption step leaves the validation range of the Potvin's methods. No one can guarantee that the patient's risk is bounded by 0.05 as long as he had not done extensive simulations.

Montague et.al. have shown that a deviation from the GMR=0.95 to GMR=0.90 lead to an alpha inflation if the Pocock nominal alpha's (0.0294, 0.0294 for stage 1,2) are used unchanged in methods B and C, respectively.

They argue that the same may take place if the target power is changed. Let me cite: "In particular, we have not considered cases where the desired power used in the method is anything other than 80%. Increasing the power used in the methods from 80 to 90% would be expected to have similar direction of effect as decreasing the GMR used in the methods from 0.95 to 0.90".

Montague TH, Potvin D, DiLiberti CE, Hauck WW, Parr AF, and DJ Schuirmann
Additional results for ‘Sequential design approaches for bioequivalence studies with crossover designs’
Pharmaceut Statist 11/1, 8–13 (2012), DOI: 10.1002/pst.483
—
Regards,

Detlew
 
ElMaestro
★★★

Denmark,
2012-08-02 13:29
(4707 d 06:45 ago)

@ d_labes
Posting: # 9021
Views: 15,062
 

 two-stage design power 90% in sample size adaption

Post reply
Hi Yvonne and dlabes,

❝ They argue that the same may take place if the target power is changed. Let me cite: "In particular, we have not considered cases where the desired power used in the method is anything other than 80%. Increasing the power used in the methods from 80 to 90% would be expected to have similar direction of effect as decreasing the GMR used in the methods from 0.95 to 0.90".


They are right. The alpha may go a little up.
Example: T/R=0.9 at CV=0.3 and N1=24 (method B). Alpha is around 0.0529 (Montague's paper) with target power 0.8 but with target power 0.9 alpha increases to 0.0543 (1 million sims).
—
Pass or fail!
ElMaestro
 
Yvonne
☆    

2012-08-02 14:06
(4707 d 06:08 ago)

@ ElMaestro
Posting: # 9024
Views: 15,073
 

 two-stage design power 90% in sample size adaption

Post reply
Hi

Thank you for your answer

❝ They are right. The alpha may go a little up.

❝ Example: T/R=0.9 at CV=0.3 and N1=24 (method B). Alpha is around 0.0529 (Montague's paper) with target power 0.8 but with target power 0.9 alpha increases to 0.0543 (1 million sims).


So as what I understand, you need to calculate your sample size in the second stage with an alpha < 0.0294 to control alpha inflation. But how far should we decrease (most likely dependent on the CV in the first stage) and is it accepted to decrease alpha in stage 2 (and thus deviate from potvin method C)?

Regards,

Yvonne
 
ElMaestro
★★★

Denmark,
2012-08-02 18:59
(4707 d 01:15 ago)

@ Yvonne
Posting: # 9025
Views: 14,953
 

 two-stage design power 90% in sample size adaption

Post reply
HYvonne,

❝ So as what I understand, you need to calculate your sample size in the second stage with an alpha < 0.0294 to control alpha inflation. But how far should we decrease (most likely dependent on the CV in the first stage) and is it accepted to decrease alpha in stage 2 (and thus deviate from potvin method C)?


You can with quite some likelihood pick any alpha you like at the second stage as long as your qualify (simulations!) that the overall alpha stays at or below 0.05. It comes down to the specific numbers you have.
—
Pass or fail!
ElMaestro
 
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-08-02 19:39
(4707 d 00:35 ago)

@ Yvonne
Posting: # 9026
Views: 15,644
 

 Are simulations sufficient? …lenghty post!

Post reply
Hi Yvonne,

Detlew has already pointed to this thread. See especially footnote #3 in this post.

You are posting from The Netherlands… Shall I feel guilty recommending Method C since 2007? :-D Quoting a deficiency letter by the MEB concerning Method C:

“Confidence intervals were adapted based upon the power of the pharmacokinetic variable. In this case for Cmax the power was below 80% and confidence intervals were adapted to 94.12%, instead of the usually applied 90%. However, adapting the confidence intervals based upon power is not acceptable and also not in accordance with the EMA guideline. Confidence intervals should be selected a priori, without evaluation of the power.”

My emphasis. BTW, the study passed with the 94.12% CI.

Potvin et al. started from Pocock’s α 0.0294 and showed in their simulations that the empiric α never exceeded 0.051 (i.e., maximum observed inflation 2%) in Methods B/C (or with α 0.028 in Method D). They considered a potential 4% increase of risk type I (i.e., to 0.052) as negligible beforehand. Note that in one million simulations only αemp. exceeding 0.05036 is significantly >0.05 (by the exact binominal test). One can never show in simulations αemp. ≤0.05), e.g., the upper critical value for 109 simulations is still 0.05001134. ;-)

Potvin et al. were somewhat unfortunate in presenting their findings in Table I. They formated empiric alphas larger than 0.052 in italics and ones significantly larger than 0.05036 (but still with the predefined acceptance boundary) in bold. So a quick look gives a false impression about what they considered important (>0.052) and what not (>0.05036). IMHO the table should have been formatted like this one (only the top block; no values >0.05036 below):

n1  CV    A       B       C       D

12  10          0.0297  0.0496  0.0498
24  10          0.0294  0.0500  0.0500
36  10          0.0294  0.0500  0.0504
48  10          0.0292  0.0501  0.0502
60  10          0.0294  0.0504  0.0501

12  20  0.0584  0.0463  0.0510  0.0499
24  20  0.0505  0.0320  0.0490  0.0493
36  20  0.0497  0.0294  0.0499  0.0499
48  20  0.0500  0.0292  0.0495  0.0497
60  20  0.0500  0.0297  0.0500  0.0500

12  30  0.0575  0.0437  0.0441  0.0415
24  30  0.0550  0.0475  0.0492  0.0475
36  30  0.0523  0.0397  0.0477  0.0471
48  30  0.0502  0.0324  0.0494  0.0495
60  30  0.0498  0.0296  0.0502  0.0499

It’s also clear from the table that in some scenarios αemp. was substantially below 0.05 – indicating that 0.0294 was lower than necessary. Of course no problems with risk I, but the penalty one has to pay in terms of the sample size is too high. See for example αemp. for n1 12, CVintra 10%: Method C 0.0496, D 0.0498, but B 0.0297… In other words, if you opt for Method B in this scenario you could increase αadj. and still maintain αemp. ≤0.05. For αadj. 0.045 (!), Method B, 106 simulations I got αemp. 0.04501 and 1–βemp. 98.69%. In this case (only ~1% of studies went to stage 2), the penalty in Method B is too high. But see Potvin’s discussion:

“This study did not seek to find the best possible two-stage design, but rather to find good ones that could be used by sponsors without further validation.”

Not sure what will happen if you come up with a 1–2αadj. = 91% confidence interval (though it should be acceptable according to the GL “[…] (with the confidence intervals accordingly using an adjusted coverage probability which will be higher than 90%)”. “Coverage probability”?1 Some Crypto-Bayesians2,3,4 at the EMA?

Some European authorities seem to accept Method B (they consider Method C as problematic) but want to see simulations – at least if your planned n1 and anticipated CVintra is not given in the tables. Note that Potvin et al. have simulated the CV-range of 20–30% in 1% increments (not given in the tables, but claimed in the discussion section); αemp. did not exceed 0.051 in any case.

So I think these are the options:
  1. It seems that Method C is not acceptable to some European authorities. Not sure whether I should even list it as an option here; have to adjust my future presentations on the topic.
    Note: Method C is explicitly preferred by the FDA and in Canada. :crying:
  2. If n1 / CVintra is exactly covered in the papers (80% power; θ 0.95: Potvin, θ 0.90: Montague) go with Method B (Potvin) or D (Montague). Even then I suggest to perform a posteriori simulations covering your actual n1 and CVintra. Recently I was asked for simulations where the study almost matched the tables (n1 49 and CVintra 30.65%; tables give n1 48, CVintra 30%). Guess the outcome…
  3. If you are unsure, go for a scientific advice. Most likely you would end up with a statement similar to MEB’s and suggesting the “rumour method” stated by Detlew. Might [sic] work, but IMHO here simulations are mandatory because you are boldly going “Where No One Has Gone Before”.
  4. If you want to leave the framework of the PQRI5 like in you original post (90% power) – as ElMaestro already pointed out – simulations are mandatory. III. above also applies.
  5. If you are tempted to go with a full adaptive design6 (i.e., re-estimate the sample size for stage 2 not by a fixed θ but the one observed in stage 1), beware! In my simulations (well, another method…) I saw extremely skewed distributions of sample sizes. This is not surprising because we have to deal with two random variables now. If by chance the observed θ is far away from 1 together with a high CV the penalty might be substantial. For Potvin’s Example 2 the second stage would need 12 subjects instead of 8.
    I would follow this concept only if the first stage is sufficiently large in order to get a reliable estimate of θ. No space for gambling here.
Generally I have some mixed feelings. I agree that it would be desirable to base sequential designs not entirely on simulations. A method based on a mathematical / statistical proof would supersede simulations. Unfortunately a closed adaptive sequential test in the bioequivalence setting has not been published yet. On the other hand disfavouring numerical methods smells of hypocrisy to me. If one insists on exact methods that would mean the end of pharmacokinetics (beyond an iv one-compartment model), weather forecasts, and designing aircraft. At least the latter has a very low risk type I – or would you climb on a plane with a 5% chance of a crash?

BTW, interesting topic!


  1. WP tells me: “The coverage probability of a confidence interval is the proportion of the time that the interval contains the true value of interest.” I see!
  2. Selwyn MR, Dempster AP, Hall NR. A Bayesian approach to bioequivalence for the 2 × 2 changeover design. Biometrics. 1981;37(1):11–21.
  3. Fluehler H, Grieve AP, Mandallaz D, Mau J, Moser HA. Bayesian Approach to Bioequivalence Assessment: An Example. J Pharm Sci. 1983;10:1178–81. doi:10.1002/jps.2600721018
  4. Selwyn MR and NR Hall NR. On Bayesian Methods for Bioequivalence. Biometrics. 1984;40(4):1103–8.
  5. Product Quality Research Institute, a non-profit organisation established in 1999. Members amongst others are the US Food and Drug Administration / Center for Drug Evaluation and Research (FDA/CDER), Health Canada, the US Pharmacopeial Convention (USP), the American Association of Pharmaceutical Scientists (AAPS), the Consumer Healthcare Products Association (CHPA), and the Pharmaceutical Research and Manufacturers of America (PhRMA). PQRI set up a working group on sequential designs in 2003 and supported the simulations.
  6. Fuglsang A. Controlling type I errors for two-stage bioequivalence study designs. Clin Res Reg Aff. 2011;28(4):100–5. doi:10.3109/10601333.2011.631547
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
 
d_labes
★★★

Berlin, Germany,
2012-08-03 13:33
(4706 d 06:41 ago)

@ Helmut
Posting: # 9029
Views: 14,910
 

 adjusted alpha = 0.045

Post reply
Dear Helmut!

Very well elaborated post. THX.

❝ It’s also clear from the table that in some scenarios αemp. was substantially below 0.05 – indicating that 0.0294 was lower than necessary. Of course no problems with risk I, but the penalty one has to pay in terms of the sample size is too high. See for example αemp. for n1 12, CVintra 10%: Method C 0.0496, D 0.0498, but B 0.0297…

❝ In other words, if you opt for Method B in this scenario you could increase αadj. and still maintain αemp. ≤0.05. For αadj. 0.045 (!), Method B, 106 simulations I got αemp. 0.04501 and 1–βemp. 98.69%. In this case (only ~1% of studies went to stage 2), the penalty in Method B is too high.


Wow! Cough :smoke: ... Very interesting result.
Where did the αadj.=0.045 came from? Luckily guess? Or some Hermetism?
—
Regards,

Detlew
 
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-08-03 16:26
(4706 d 03:48 ago)

@ d_labes
Posting: # 9032
Views: 14,860
 

 adjusted alpha = 0.038

Post reply
Dear Detlew!

❝ Where did the αadj.=0.045 came from? Luckily guess? Or some Hermetism?


More an educated guess. With n1 12 and CV 10% a study is already large enough (fixed sample design 98.8% power) and chances to require a second stage are very low. Since Methods C/D allow for stopping in stage 1 with a 90% CI in these methods αemp. ~ αnominal. In Method B we have to pay the penalty and αemp. ~ αadj.. That’s even more pronounced for higher n1 and the same CV (where <0.1% of studies reach stage 2). So why not increase the level of αadj.? 0.045 was a quick-shot – maybe one can even go to close <0.05.

I will try a more realistic example (n1 36, CV 30%; fixed design power 77.2% and 81.6% for n 40). With αadj. 0.0294 Potvin got αemp.: 0.0397 (B) and 0.0477 (C); studies in stage 2: 29.0% (B) and 22.7% (C).
106 sims of Method B with αadj. 0.0380 (= quick-shot obtained from 104 sims) are running…
BTW, where do you think Method B’s 0.0280 came from? :ponder:
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
 
d_labes
★★★

Berlin, Germany,
2012-08-03 17:48
(4706 d 02:26 ago)

@ Helmut
Posting: # 9034
Views: 14,876
 

 adjusted alpha by sims?

Post reply
Dear Helmut!

❝ 106 sims of Method B with αadj. 0.0380 (= quick-shot obtained from 104 sims) are running…


Don't overheat your machines :-D.

❝ BTW, where do you think Method B’s 0.0280 came from? :ponder:


If you mean Method D I guess: also from some trial and error.

Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe?
What if the CV comes out other than supposed in the planning?

BTW: It's a similar question a good friend of our ol' pirate EM was once asked in respect to his method (see paper 6 of your post above):
"Do you think that it is acceptable to any regulator in Europe that the alpha of stage 2 isn't chosen a priori but depending on the results of stage 1, again via some simulation studies?"
—
Regards,

Detlew
 
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-08-03 18:39
(4706 d 01:35 ago)

@ d_labes
Posting: # 9035
Views: 15,258
 

 Yeah but, no but, yeah but, no but…

Post reply
Dear Detlew!

❝ Don't overheat your machines :-D.


Thanks for reminding me. Have to look – ah, 49% done so far. :PCchaos:

❝ If you mean Method D


Oops, sure.

❝ I guess: also from some trial and error.


Here we are!

❝ Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe?


Why not – since we don’t have a method which is not based on simulations. Obviously some regulators don’t trust PQRI’s simulations and are asking for a posteriori simulations of the actual n1 and CV – why shouldn’t they accept them if performed before the study?

❝ It's a similar question a good friend of our ol' pirate EM was once asked in respect to his method (see paper 6 of your post above):

❝ "Do you think that it is acceptable to any regulator in Europe that the alpha of stage 2 isn't chosen a priori but depending on the results of stage 1, again via some simulation studies?"


If one comes up with an αadj. through simulations, it will be fixed in both stages and can be specified in the protocol. BTW, the lacking acceptance of an adjusted α2 gives me headaches.

Results of 106 sims, n1 36, CV 30%:
1 Method B, αadj. 0.0294
2 Method B on steroids wo intermediate power, αadj. 0.0380
              Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1  47.6  36   46   66   81.1    0.0397   │ 40.7  36   36   62   29.0   0.8379
2  44.3  36   42   60   70.63   0.047938 │ 39.0  36   36   56   22.43  0.84041
Risk I is maintained and we can expect a slight increase in power. ~23% less studies proceed to stage 2 (θ 0.95). May use 92.40% CI instead of 94.12%.

❝ What if the CV comes out other than supposed in the planning?


Good point! In order to be protected against that sims should cover a realistic (!) range of CVs (and n1 – drop outs!), not only the expected one. If the (mandatory?) a posteriori sim shows αemp. >0.05, bad luck. If we go with a fixed sample design instead and the CV turns out to be higher than expected we may fail as well. Part of the game. But let’s simulate. Method B on steroids wo power, αadj. 0.0380, but CV 35%.
           Ratio 1.25                 │             Ratio 0.95             
──────────────────────────────────────┼─────────────────────────────────────
                     % in    empiric  │                      % in   empiric
 n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
56.8  36   56   80   92.10   0.057127 │ 46.7  36   36   78   42.2   0.83226
Ouch! Bad idea. Should have payed more attention to our ol’ pirate’s Figure 3; below transformed into a contour plot:

[image]

Try this goodie (enlarge the plot and drag around):
require(rgl)
x <- seq(12, 60, by = 12)
y <- seq(10, 100, by = 10)
B <- matrix(data = c(
       0.0297,0.0463,0.0437,0.0344,0.0309,0.0297,0.0294,0.0292,0.0289,0.0291,
       0.0294,0.0320,0.0475,0.0433,0.0338,0.0307,0.0299,0.0298,0.0298,0.0298,
       0.0294,0.0294,0.0397,0.0485,0.0420,0.0333,0.0306,0.0303,0.0296,0.0298,
       0.0292,0.0292,0.0324,0.0458,0.0484,0.0399,0.0328,0.0303,0.0297,0.0297,
       0.0294,0.0297,0.0296,0.0409,0.0483,0.0466,0.0381,0.0318,0.0300,0.0301),
       nrow = 5, ncol = 10, byrow = TRUE, dimnames = NULL)
rownames(B) <- as.character(x)
colnames(B) <- paste(as.character(y),"%")
persp3d(x, y, B,
  xlim = c(12, 60), xlab = "n1",
  ylim = c(10, 100), ylab = "CV%",
  zlim = c(0.025, 0.05), zlab = "empiric alpha",
  main = "Potvin B (Table I)",
  aspect = c(1, 4/3, 1),
  color = rgb(0.7, 0.9, 1, 0.75), smooth = TRUE, lit = TRUE)

αemp. is ~0.05 if we are both close to the intended n1 and expected CV but drops rapidly to ~αadj. otherwise. In other words the level becomes conservative. I still think that it should be possible to raise αadj. but with great caution and only after a lot of simulations.

Example: What can we do if we plan for n1 36, CV 30% and what will happen if the CV is higher? From the above we guess that our αadj. for 30% is limited by the maximum possible αemp. at CV 40%.
1 Method B, αadj. 0.0294, 2 Method B on steroids wo power, αadj. 0.0303, 106 sims.
CV 30%        Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1  47.6  36   46   66   81.1    0.0397   │ 40.7  36   36   62   29.0   0.8379
2  47.1  36   46   66   79.82   0.040450 │ 40.5  36   36   60   28.08  0.83735
CV 40%        Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1  78.3  48   78  112   97.0    0.0485   │ 67.3  36   70  112   66.3   0.8236
2  66.3  36   68  110   96.95   0.049704 │ 66.3  36   68  110   65.47  0.82377
CV 50%        Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1 116.8  74  116  166   98.5    0.0420   │127.7  36  116  166   90.8   0.8052
2 115.5  72  114  166   98.41   0.043799 │111.1  36  114  166   90.25  0.80468
Have we gained something substantially? Not at all. With 50% we have crossed the ridge at 40% and don’t have to worry any more. Maybe that’s really a stupid idea and not worth the efforts – only to come up with a slightly narrower 93.94% CI.

After all this stuff coming back to your question again….

❝ Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe?


Taking the GL and the Dutch deficiency letter from above into account at least adaption of α2 seems to be very tough. Coincidentally I have heard the term ‘cookbook’ for the first time from a Dutch regulator back in 2004. Little chances for ‘true’ adaptive designs. :-(

Now for the positive part. If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n1 corresponding to max. αemp.. That would mean that we are already on ‘top of the ridge’. What could happen?
  • Drop-outs: We go left in the plot and see the test becomes conservative.
  • Higher/lower CV than expected: We go up/down towards conservative levels.
BTW, empiric power is also interesting:

[image]

If we plan the study with a n1 like a fixed design, power will be always >80% because we get a ‘second chance’ of showing BE in stage 2 (e.g., CV 30%, Power 81.6% for n 40 in a fixed design but ~85% with n1 40).
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
 
d_labes
★★★

Berlin, Germany,
2012-08-07 16:42
(4702 d 03:32 ago)

@ Helmut
Posting: # 9046
Views: 14,860
 

 Yeah but, no but ...

Post reply
Dear Helmut!

:clap:

❝ ❝ What if the CV comes out other than supposed in the planning?

❝

❝ Good point! In order to be protected against that sims should cover a realistic (!) range of CVs (and n1 – drop outs!), not only the expected one. If the (mandatory?) a posteriori sim shows αemp. >0.05, bad luck. If we go with a fixed sample design instead and the CV turns out to be higher than expected we may fail as well. Part of the game.

❝ But let’s simulate. Method B on steroids wo power,


:cool:

❝ αadj. 0.0380, but CV 35%.

❝            Ratio 1.25                 │             Ratio 0.95             

❝ ──────────────────────────────────────┼─────────────────────────────────────

❝                      % in    empiric  │                      % in   empiric
❝  n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 

❝ 56.8  36   56   80   92.10   0.057127 │ 46.7  36   36   78   42.2   0.83226

❝ Ouch! Bad idea. ...


Got you! :-D

❝ Now for the positive part. If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n1 corresponding to max. αemp.. That would mean that we are already on ‘top of the ridge’. What could happen?

  • Drop-outs: We go left in the plot and see the test becomes conservative.
  • Higher/lower CV than expected: We go up/down towards conservative levels.

Beeing on the ‘top of the ridge’ would mean IMHO that we are practically working with alpha1,2 = 0.0294.

❝ BTW, empiric power is also interesting:

❝

❝ [image]


Another rather clear message of this nice picture: Don't use n1 as low as 12 ('internal pilot' in the usual understanding of many sponsors) if the anticipated CV is >25%.
—
Regards,

Detlew
 
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-08-07 17:14
(4702 d 03:00 ago)

@ d_labes
Posting: # 9047
Views: 14,740
 

 Yeah but, no but ...

Post reply
Dear Detlew!

❝ ❝ […] If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n1 corresponding to max. αemp.. That would mean that we are already on ‘top of the ridge’.

❝

❝ Beeing on the ‘top of the ridge’ would mean IMHO that we are practically working with alpha1,2 = 0.0294.


Exactly. That’s what I mean by “if the study was planned with a n1 corresponding to max. αemp.” above.

❝ Another rather clear message of this nice picture: Don't use n1 as low as 12 ('internal pilot' in the usual understanding of many sponsors) if the anticipated CV is >25%.


Yep. I was always preaching (though in the context of ‘classical’ pilot studies) against too small sample sizes if the anticipated CV is large.
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
 
UA Flag
Activity
 Admin contact
23,424 posts in 4,927 threads, 1,673 registered users;
47 visitors (0 registered, 47 guests [including 16 identified bots]).
Forum time: 20:15 CEST (Europe/Vienna)

Medical researches can be divided into two sorts:
those who think that meta is better and those
who believe that pooling is fooling.    Stephen Senn

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5