Log in
Register|
Yvonne ☆ 2012-08-02 11:50 (4707 d 05:30 ago) Posting: # 9018 Views: 16,311 |
|
|
Dear all, Can someone help me with the following question. We are currently planning for a two-stage design (potvin method C). My question is, if we go into the second stage and we have to do a sample size calculation, can we use a power of 90% (instead of 80%)? And of course more importantly  , would a sample size calculation based on 90% power be accepted by the authorities?In theory the power is the companies risk and not the consumers risk and thus no extra inflation of alpha would occur in a two stage design (because of a different use of power) or am I wrong here  .Regards, Yvonne |
|
ElMaestro ★★★ Denmark, 2012-08-02 12:17 (4707 d 05:03 ago) @ Yvonne Posting: # 9019 Views: 15,308 |
|
|
Hello Yvonne, ❝ My question is, if we go into the second stage and we have to do a sample size calculation, can we use a power of 90% (instead of 80%)? And of course more importantly Yes, this will be acceptable to many authorities. ❝ In theory the power is the companies risk and not the consumers risk and thus no extra inflation of alpha would occur in a two stage design (because of a different use of power) or am I wrong here You probably need to prove that the overall alpha is preserved under or at 0.05. A simulation study is recommended for this purpose. — Pass or fail! ElMaestro |
|
Yvonne ☆ 2012-08-02 13:47 (4707 d 03:32 ago) @ ElMaestro Posting: # 9022 Views: 15,056 |
|
|
Thank you very much for your answer. ❝ You probably need to prove that the overall alpha is preserved under or at 0.05. A simulation study is recommended for this purpose. I am not very familiar with simulations. I can uderstand that this is very complicated and might take long. So in practice there are 2 (or 3) possibilities? option a: a simulation is not that difficult and can be done relatively "easy" with some help (and of course accepted by authorities ). Then I think that this one is preferedoption b: simulations are relatively difficult and without knowing what will happen with the inflation of alpha, the only power that will be accepted by the authorities is 80% or option c: simulations are relatively difficult but authorites will accept also sample size calculation in the second stage with 90% power (which I hope is the answer  ).Can you or someone else help me with this? Kind regards, Yvonne |
|
ElMaestro ★★★ Denmark, 2012-08-02 13:54 (4707 d 03:25 ago) @ Yvonne Posting: # 9023 Views: 15,028 |
|
|
Hello Yvonne, ❝ or option c: simulations are relatively difficult but authorites will accept also sample size calculation in the second stage with 90% power (which I hope is the answer Yes, I think C is the right one here. What I would do if I were you: Make a series of simulations that show that a target power of 0.9 adequately controls alpha at or below 0.05 under your conditions (do it for a good range of CV's and N1's) and show this documentation to regulators. A sc. advice is of course a good idea. They may ask for a copy of your software, e.g. to make sure it actually reproduces something known (like some of the values of Potvin 2008). — Pass or fail! ElMaestro |
|
d_labes ★★★ Berlin, Germany, 2012-08-02 12:33 (4707 d 04:47 ago) @ Yvonne Posting: # 9020 Views: 15,207 |
|
|
Dear Yvonne, ❝ Can someone help me with the following question. We are currently planning for a two-stage design (potvin method C). If you go for an EMA submission: Seems there is an Potvin C abhorrence. See Huanghe's and Helmut's discussion. ❝ My question is, if we go into the second stage and we have to do a sample size calculation, can we use a power of 90% (instead of 80%)? And of course more importantly ❝ In theory the power is the companies risk and not the consumers risk and thus no extra inflation of alpha would occur in a two stage design (because of a different use of power) or am I wrong here Using 90% power in the sample size adaption step leaves the validation range of the Potvin's methods. No one can guarantee that the patient's risk is bounded by 0.05 as long as he had not done extensive simulations. Montague et.al. have shown that a deviation from the GMR=0.95 to GMR=0.90 lead to an alpha inflation if the Pocock nominal alpha's (0.0294, 0.0294 for stage 1,2) are used unchanged in methods B and C, respectively. They argue that the same may take place if the target power is changed. Let me cite: "In particular, we have not considered cases where the desired power used in the method is anything other than 80%. Increasing the power used in the methods from 80 to 90% would be expected to have similar direction of effect as decreasing the GMR used in the methods from 0.95 to 0.90". Montague TH, Potvin D, DiLiberti CE, Hauck WW, Parr AF, and DJ Schuirmann Additional results for ‘Sequential design approaches for bioequivalence studies with crossover designs’ Pharmaceut Statist 11/1, 8–13 (2012), DOI: 10.1002/pst.483 — Regards, Detlew |
|
ElMaestro ★★★ Denmark, 2012-08-02 13:29 (4707 d 03:51 ago) @ d_labes Posting: # 9021 Views: 15,054 |
|
|
Hi Yvonne and dlabes, ❝ They argue that the same may take place if the target power is changed. Let me cite: "In particular, we have not considered cases where the desired power used in the method is anything other than 80%. Increasing the power used in the methods from 80 to 90% would be expected to have similar direction of effect as decreasing the GMR used in the methods from 0.95 to 0.90". They are right. The alpha may go a little up. Example: T/R=0.9 at CV=0.3 and N1=24 (method B). Alpha is around 0.0529 (Montague's paper) with target power 0.8 but with target power 0.9 alpha increases to 0.0543 (1 million sims). — Pass or fail! ElMaestro |
|
Yvonne ☆ 2012-08-02 14:06 (4707 d 03:14 ago) @ ElMaestro Posting: # 9024 Views: 15,065 |
|
|
Hi Thank you for your answer ❝ They are right. The alpha may go a little up. ❝ Example: T/R=0.9 at CV=0.3 and N1=24 (method B). Alpha is around 0.0529 (Montague's paper) with target power 0.8 but with target power 0.9 alpha increases to 0.0543 (1 million sims). So as what I understand, you need to calculate your sample size in the second stage with an alpha < 0.0294 to control alpha inflation. But how far should we decrease (most likely dependent on the CV in the first stage) and is it accepted to decrease alpha in stage 2 (and thus deviate from potvin method C)? Regards, Yvonne |
|
ElMaestro ★★★ Denmark, 2012-08-02 18:59 (4706 d 22:21 ago) @ Yvonne Posting: # 9025 Views: 14,945 |
|
|
HYvonne, ❝ So as what I understand, you need to calculate your sample size in the second stage with an alpha < 0.0294 to control alpha inflation. But how far should we decrease (most likely dependent on the CV in the first stage) and is it accepted to decrease alpha in stage 2 (and thus deviate from potvin method C)? You can with quite some likelihood pick any alpha you like at the second stage as long as your qualify (simulations!) that the overall alpha stays at or below 0.05. It comes down to the specific numbers you have. — Pass or fail! ElMaestro |
|
Helmut ★★★   Vienna, Austria, 2012-08-02 19:39 (4706 d 21:41 ago) @ Yvonne Posting: # 9026 Views: 15,635 |
|
|
Hi Yvonne, Detlew has already pointed to this thread. See especially footnote #3 in this post. You are posting from The Netherlands… Shall I feel guilty recommending Method C since 2007? Quoting a deficiency letter by the MEB concerning Method C:“Confidence intervals were adapted based upon the power of the pharmacokinetic variable. In this case for Cmax the power was below 80% and confidence intervals were adapted to 94.12%, instead of the usually applied 90%. However, adapting the confidence intervals based upon power is not acceptable and also not in accordance with the EMA guideline. Confidence intervals should be selected a priori, without evaluation of the power.” My emphasis. BTW, the study passed with the 94.12% CI.Potvin et al. started from Pocock’s α 0.0294 and showed in their simulations that the empiric α never exceeded 0.051 (i.e., maximum observed inflation 2%) in Methods B/C (or with α 0.028 in Method D). They considered a potential 4% increase of risk type I (i.e., to 0.052) as negligible beforehand. Note that in one million simulations only αemp. exceeding 0.05036 is significantly >0.05 (by the exact binominal test). One can never show in simulations αemp. ≤0.05), e.g., the upper critical value for 109 simulations is still 0.05001134.  Potvin et al. were somewhat unfortunate in presenting their findings in Table I. They formated empiric alphas larger than 0.052 in italics and ones significantly larger than 0.05036 (but still with the predefined acceptance boundary) in bold. So a quick look gives a false impression about what they considered important (>0.052) and what not (>0.05036). IMHO the table should have been formatted like this one (only the top block; no values >0.05036 below): n1 CV A B C DIt’s also clear from the table that in some scenarios αemp. was substantially below 0.05 – indicating that 0.0294 was lower than necessary. Of course no problems with risk I, but the penalty one has to pay in terms of the sample size is too high. See for example αemp. for n1 12, CVintra 10%: Method C 0.0496, D 0.0498, but B 0.0297… In other words, if you opt for Method B in this scenario you could increase αadj. and still maintain αemp. ≤0.05. For αadj. 0.045 (!), Method B, 106 simulations I got αemp. 0.04501 and 1–βemp. 98.69%. In this case (only ~1% of studies went to stage 2), the penalty in Method B is too high. But see Potvin’s discussion: “This study did not seek to find the best possible two-stage design, but rather to find good ones that could be used by sponsors without further validation.” Not sure what will happen if you come up with a 1–2αadj. = 91% confidence interval (though it should be acceptable according to the GL “[…] (with the confidence intervals accordingly using an adjusted coverage probability which will be higher than 90%)”. “Coverage probability”?1 Some Crypto-Bayesians2,3,4 at the EMA?Some European authorities seem to accept Method B (they consider Method C as problematic) but want to see simulations – at least if your planned n1 and anticipated CVintra is not given in the tables. Note that Potvin et al. have simulated the CV-range of 20–30% in 1% increments (not given in the tables, but claimed in the discussion section); αemp. did not exceed 0.051 in any case. So I think these are the options:
BTW, interesting topic!
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png) Helmut Schütz ![[image]](https://static.bebac.at/img/CC by.png) The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
d_labes ★★★ Berlin, Germany, 2012-08-03 13:33 (4706 d 03:47 ago) @ Helmut Posting: # 9029 Views: 14,901 |
|
|
Dear Helmut! Very well elaborated post. THX. ❝ It’s also clear from the table that in some scenarios αemp. was substantially below 0.05 – indicating that 0.0294 was lower than necessary. Of course no problems with risk I, but the penalty one has to pay in terms of the sample size is too high. See for example αemp. for n1 12, CVintra 10%: Method C 0.0496, D 0.0498, but B 0.0297… ❝ In other words, if you opt for Method B in this scenario you could increase αadj. and still maintain αemp. ≤0.05. For αadj. 0.045 (!), Method B, 106 simulations I got αemp. 0.04501 and 1–βemp. 98.69%. In this case (only ~1% of studies went to stage 2), the penalty in Method B is too high. Wow! Cough  ... Very interesting result.Where did the αadj.=0.045 came from? Luckily guess? Or some Hermetism? — Regards, Detlew |
|
Helmut ★★★ Vienna, Austria, 2012-08-03 16:26 (4706 d 00:54 ago) @ d_labes Posting: # 9032 Views: 14,852 |
|
|
Dear Detlew! ❝ Where did the αadj.=0.045 came from? Luckily guess? Or some Hermetism? More an educated guess. With n1 12 and CV 10% a study is already large enough (fixed sample design 98.8% power) and chances to require a second stage are very low. Since Methods C/D allow for stopping in stage 1 with a 90% CI in these methods αemp. ~ αnominal. In Method B we have to pay the penalty and αemp. ~ αadj.. That’s even more pronounced for higher n1 and the same CV (where <0.1% of studies reach stage 2). So why not increase the level of αadj.? 0.045 was a quick-shot – maybe one can even go to close <0.05. I will try a more realistic example (n1 36, CV 30%; fixed design power 77.2% and 81.6% for n 40). With αadj. 0.0294 Potvin got αemp.: 0.0397 (B) and 0.0477 (C); studies in stage 2: 29.0% (B) and 22.7% (C). 106 sims of Method B with αadj. 0.0380 (= quick-shot obtained from 104 sims) are running… BTW, where do you think Method B’s 0.0280 came from?  — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
d_labes ★★★ Berlin, Germany, 2012-08-03 17:48 (4705 d 23:31 ago) @ Helmut Posting: # 9034 Views: 14,867 |
|
|
Dear Helmut! ❝ 106 sims of Method B with αadj. 0.0380 (= quick-shot obtained from 104 sims) are running… Don't overheat your machines .❝ BTW, where do you think Method B’s 0.0280 came from? If you mean Method D I guess: also from some trial and error. Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe? What if the CV comes out other than supposed in the planning? BTW: It's a similar question a good friend of our ol' pirate EM was once asked in respect to his method (see paper 6 of your post above): "Do you think that it is acceptable to any regulator in Europe that the alpha of stage 2 isn't chosen a priori but depending on the results of stage 1, again via some simulation studies?" — Regards, Detlew |
|
Helmut ★★★ Vienna, Austria, 2012-08-03 18:39 (4705 d 22:41 ago) @ d_labes Posting: # 9035 Views: 15,250 |
|
|
Dear Detlew! ❝ Don't overheat your machines Thanks for reminding me. Have to look – ah, 49% done so far.  ❝ If you mean Method D Oops, sure. ❝ I guess: also from some trial and error. Here we are! ❝ Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe? Why not – since we don’t have a method which is not based on simulations. Obviously some regulators don’t trust PQRI’s simulations and are asking for a posteriori simulations of the actual n1 and CV – why shouldn’t they accept them if performed before the study? ❝ It's a similar question a good friend of our ol' pirate EM was once asked in respect to his method (see paper 6 of your post above): ❝ "Do you think that it is acceptable to any regulator in Europe that the alpha of stage 2 isn't chosen a priori but depending on the results of stage 1, again via some simulation studies?" If one comes up with an αadj. through simulations, it will be fixed in both stages and can be specified in the protocol. BTW, the lacking acceptance of an adjusted α2 gives me headaches. Results of 106 sims, n1 36, CV 30%: 1 Method B, αadj. 0.02942 Method B on steroids wo intermediate power, αadj. 0.0380 Ratio 1.25 │ Ratio 0.95 Risk I is maintained and we can expect a slight increase in power. ~23% less studies proceed to stage 2 (θ 0.95). May use 92.40% CI instead of 94.12%. ❝ What if the CV comes out other than supposed in the planning? Good point! In order to be protected against that sims should cover a realistic (!) range of CVs (and n1 – drop outs!), not only the expected one. If the (mandatory?) a posteriori sim shows αemp. >0.05, bad luck. If we go with a fixed sample design instead and the CV turns out to be higher than expected we may fail as well. Part of the game. But let’s simulate. Method B on steroids wo power, αadj. 0.0380, but CV 35%. Ratio 1.25 │ Ratio 0.95 Ouch! Bad idea. Should have payed more attention to our ol’ pirate’s Figure 3; below transformed into a contour plot: ![[image]](img/uploaded/image114.png) Try this goodie (enlarge the plot and drag around): require(rgl)αemp. is ~0.05 if we are both close to the intended n1 and expected CV but drops rapidly to ~αadj. otherwise. In other words the level becomes conservative. I still think that it should be possible to raise αadj. but with great caution and only after a lot of simulations. Example: What can we do if we plan for n1 36, CV 30% and what will happen if the CV is higher? From the above we guess that our αadj. for 30% is limited by the maximum possible αemp. at CV 40%. 1 Method B, αadj. 0.0294, 2 Method B on steroids wo power, αadj. 0.0303, 106 sims.CV 30% Ratio 1.25 │ Ratio 0.95 CV 40% Ratio 1.25 │ Ratio 0.95 CV 50% Ratio 1.25 │ Ratio 0.95 Have we gained something substantially? Not at all. With 50% we have crossed the ridge at 40% and don’t have to worry any more. Maybe that’s really a stupid idea and not worth the efforts – only to come up with a slightly narrower 93.94% CI. After all this stuff coming back to your question again…. ❝ Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe? Taking the GL and the Dutch deficiency letter from above into account at least adaption of α2 seems to be very tough. Coincidentally I have heard the term ‘cookbook’ for the first time from a Dutch regulator back in 2004. Little chances for ‘true’ adaptive designs.  Now for the positive part. If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n1 corresponding to max. αemp.. That would mean that we are already on ‘top of the ridge’. What could happen?
![[image]](img/uploaded/image115.png) If we plan the study with a n1 like a fixed design, power will be always >80% because we get a ‘second chance’ of showing BE in stage 2 (e.g., CV 30%, Power 81.6% for n 40 in a fixed design but ~85% with n1 40). — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
d_labes ★★★ Berlin, Germany, 2012-08-07 16:42 (4702 d 00:37 ago) @ Helmut Posting: # 9046 Views: 14,852 |
|
|
Dear Helmut!  ❝ ❝ What if the CV comes out other than supposed in the planning? ❝ ❝ Good point! In order to be protected against that sims should cover a realistic (!) range of CVs (and n1 – drop outs!), not only the expected one. If the (mandatory?) a posteriori sim shows αemp. >0.05, bad luck. If we go with a fixed sample design instead and the CV turns out to be higher than expected we may fail as well. Part of the game. ❝ But let’s simulate. Method B on steroids wo power,  ❝ αadj. 0.0380, but CV 35%. ❝ ❝ ──────────────────────────────────────┼───────────────────────────────────── ❝ n (5%, 50%, 95%) stage 2 α │ n (5%, 50%, 95%) stage 2 1-β ❝ 56.8 36 56 80 92.10 0.057127 │ 46.7 36 36 78 42.2 0.83226 ❝ Ouch! Bad idea. ... Got you! ❝ Now for the positive part. If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n1 corresponding to max. αemp.. That would mean that we are already on ‘top of the ridge’. What could happen?
Beeing on the ‘top of the ridge’ would mean IMHO that we are practically working with alpha1,2 = 0.0294. ❝ BTW, empiric power is also interesting: ❝ ❝ Another rather clear message of this nice picture: Don't use n1 as low as 12 ('internal pilot' in the usual understanding of many sponsors) if the anticipated CV is >25%. — Regards, Detlew |
|
Helmut ★★★ Vienna, Austria, 2012-08-07 17:14 (4702 d 00:06 ago) @ d_labes Posting: # 9047 Views: 14,732 |
|
|
Dear Detlew! ❝ ❝ […] If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n1 corresponding to max. αemp.. That would mean that we are already on ‘top of the ridge’. ❝ ❝ Beeing on the ‘top of the ridge’ would mean IMHO that we are practically working with alpha1,2 = 0.0294. Exactly. That’s what I mean by “if the study was planned with a n1 corresponding to max. αemp.” above. ❝ Another rather clear message of this nice picture: Don't use n1 as low as 12 ('internal pilot' in the usual understanding of many sponsors) if the anticipated CV is >25%. Yep. I was always preaching (though in the context of ‘classical’ pilot studies) against too small sample sizes if the anticipated CV is large. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Ing. Helmut Schütz