ElMaestro
★★★

Denmark,
2012-07-03 20:33
(4286 d 18:36 ago)

Posting: # 8882
Views: 21,400
 

 Sequential designs, draft FDA guidance Loteprednol [Two-Stage / GS Designs]

Hi all,

I am reading FDA's draft guidance on Loteprednol, revised June 2012.

"If using the cross-over study design and within-subject variability is a concern, the applicant may apply an appropriately designed adaptive group-sequential bioequivalence study design. For general information on this approach, please refer to Potvin et al. Sequential design approaches for bioequivalence studies with crossover designs, Pharm. Stat. 7:245-262 (2008) and Montague et al. Additional results for ‘Sequential design approaches for bioequivalence studies with crossover designs’, Pharm. Stat. 2011 Feb 10. Please note that if there is no intention to separately submit to the Agency the results of the first group, then a second group may be enrolled following analysis of data from the first group, and statistical analysis conducted using “group” in the model without using the sequential design constraints."

Whiskey Tango Foxtrot?
What does that mean for an applicant and does anyone have some insight into the logic behind it?

Note: Same passage exists in the dexa/tobi guidance document. Also revised June 2012.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-03 21:25
(4286 d 17:44 ago)

@ ElMaestro
Posting: # 8883
Views: 19,545
 

 What the heck?

Hi ElMaestro!

❝ I am reading FDA's draft guidance on Loteprednol, revised June 2012.


At least that’s the first official statement that the FDA accepts sequential designs and prefers Potvin etc. Can revise my presentations from “personal communication with Barbara Davit”. :-D

❝ "[…] if there is no intention to separately submit to the Agency the results of the first group, then a second group may be enrolled following analysis of data from the first group, and statistical analysis conducted using “group” in the model without using the sequential design constraints."


❝ Whiskey Tango Foxtrot?


Precisely.

❝ What does that mean for an applicant and does anyone have some insight into the logic behind it?


What the heck does the “if – then – without” part mean? Why should anybody want to submit the results of stage 1 if he/she/it has to proceed to stage 2? Any if not, what does “without using the sequential design constraints” mean? Unbelievable – with FDA’s Donald Schuirmann as co-author of both papers.
Reading between the lines my interpretation would be: FDA accepts a pilot study (classical design, n ≥ 12, properly planned and evaluated, etc.) demonstrating already BE. So the pilot is treated as a pivotal study. If a study passes in stage 1, FDA calls this one pivotal.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Shuanghe
★★  

Spain,
2012-07-04 11:07
(4286 d 04:02 ago)

@ Helmut
Posting: # 8887
Views: 19,853
 

 some unofficial opinion with this regard in Europe

Hi all,

I was about to make a new post when I saw this, so ElMaestro, I just borrow your place to make some comments and ask a few questions.;-)

I had some unofficial feedback regarding sequential design just a few days ago. We send a protocol synopsis with statistical analysis details with Method C from Potvin's article and were told that "Potvin's method is not valid in Europe". I have no idea if that means all method ABCD (well, A is not recommended so I could probably take it off) or just method C since that is the one we used in a synopsis. Long story short, after some discussion back and forth we end up with one of the examples mentioned in EMA's BE guideline and got OK from agency. Basically, we do stage 1 BE, analysis with alpha=0.0294, if BE, stop, if not, calculate sample size for stage 2 and analyse all data with alpha=0.0294 again.

I got the opinion of "not valid in Europe" unofficially from certain agency who in turn got it from BSWP (which we assume is BioStatistics Working Party). I'm not sure it will be reflected somewhere in an official document (e.g QA) or not.

so my first question is: You guys have any insight with this regard?

and Hi Helmut, it was nice talking to you in Budapest last month by the way. For those of you didn't know, Helmut has so many fans, esp. from India, who want to have a picture with him I couldn't remember how many times I was asked to help with taking pictures. :lol: :ok:

Anyway, I remember that you had several protocols accepted by certain agency using adaptive sequential design. I thought they were based on method C? Now since I have this feedback, I started to doubt my memory. maybe I mixed up what I read and what I heard. Is it OK if I ask for your confirmation? you don't have to go in detail about study design in case of confidential issue. maybe just a rough idea?

Thanks,

Shuanghe

All the best,
Shuanghe
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-04 17:43
(4285 d 21:26 ago)

@ Shuanghe
Posting: # 8888
Views: 20,109
 

 Great!

Hi Shuanghe!

❝ I had some unofficial feedback regarding sequential design just a few days ago. We send a protocol synopsis with statistical analysis details with Method C from Potvin's article and were told that "Potvin's method is not valid in Europe".


Phantastic! Should be put in the same drawer like

TOST is not valid in Nanjing – use the I Ching instead.”

See also this thread and ElMaestro’s wonderful Pt. BTW, you are not alone (see this post).

❝ I have no idea if that means all method ABCD (well, A is not recommended so I could probably take it off) or just method C since that is the one we used in a synopsis.


I would guess methods A & C. Seems that some regulators believe [sic] in B, but then they should be happy with the even more strict method D as well. Or are they really thinking that α 0.0294 is a universal constant?

❝ […] after some discussion back and forth we end up with one of the examples mentioned in EMA's BE guideline and got OK from agency. Basically, we do stage 1 BE, analysis with alpha=0.0294, if BE, stop, if not, calculate sample size for stage 2 and analyse all data with alpha=0.0294 again.


Great. This is not even Method B (abandoning the power estimation step). You are leaving the validated range of Potvin’s paper; the patient’s is risk is not controlled any more. But let’s look at the GL again:

Two-stage design
It is acceptable to use a two-stage approach when attempting to demonstrate bioequi­va­lence. An initial group of subjects can be treated and their data analysed. If bioequivalence has not been demonstrated an additional group can be recruited and the results from both groups combined in a final analysis. If this approach is adopted appropriate steps must be taken to preserve the overall type I error of the experiment and the stopping criteria should be clearly defined prior to the study. The analysis of the first stage data should be treated as an interim analysis and both analyses conducted at adjusted significance levels (with the confidence intervals accordingly using an adjusted coverage probability which will be higher than 90%). For example, using 94.12% confidence intervals for both the analysis of stage 1 and the combined data from stage 1 and stage 2 would be acceptable, but there are many acceptable alternatives and the choice of how much alpha to spend at the interim analysis is at the company’s discretion. The plan to use a two-stage approach must be pre-spe­cified in the protocol along with the adjusted significance levels to be used for each of the analyses.
When analysing the combined data from the two stages, a term for stage should be included in the ANOVA model.

  • An initial group of subjects can be treated and their data analysed. If bioequivalence has not been demonstrated an additional group can be recruited …
    No power estimation in the interim analysis! See this thread. I saw a scientific advice from EMA (2011) where O’Brian-Fleming was suggested by the applicant and EMA wanted to see simulations showing that empiric α will not exceed 0.05. If the agency accepted your method without an intermediate power estimation they should ask themselves which risk to patients they are actually accepting.
  • … and the results from both groups combined in a final analysis.
    OK, doesn’t speak against Potvin; no poolability criterion.
  • … both analyses conducted at adjusted significance levels (with the confidence intervals accordingly using an adjusted coverage probability which will be higher than 90%).
    Aha, here we are (both analyses conducted at adjusted significance levels Method C)!
  • For example, using 94.12% confidence intervals for both the analysis of stage 1 and the combined data from stage 1 and stage 2 would be acceptable, …
    That’s Method B!
  • … but there are many acceptable alternatives and the choice of how much alpha to spend at the interim analysis is at the company’s discretion.
    F.i. Potvin et al. and Montague et al. Method D. See the request in the scientific advice above for O’Brian-Fleming.
  • The plan to use a two-stage approach must be pre-specified in the protocol along with the adjusted significance levels to be used for each of the analyses.
    Note the plural (significance levels)! If EMA is accepting O’Brian-Fleming (after demonstration of empiric α ≤0.05), why do they have problems with Method C, where this was already demonstrated and is recommended by both the FDA and in Canada?*

❝ I got the opinion of "not valid in Europe" unofficially from certain agency who in turn got it from BSWP (which we assume is BioStatistics Working Party). I'm not sure it will be reflected somewhere in an official document (e.g QA) or not.


I don’t know of any document except the ambiguous GL itself.

❝ Hi Helmut, it was nice talking to you in Budapest last month by the way.


Same with me. Enjoyed especially the talks outside in the smoker’s Ghetto. :smoke:

❝ I remember that you had several protocols accepted by certain agency using adeptive sequential design. I thought they were based on method C? Now since I have this feedback, I started to doubt my memory. maybe I mixed up what I read and what I heard.


Your memory is correct; see this post. The BfArM (not in Europe according to the BSWP?) had no problems with Method C. Maybe it’s worthwhile noticing that two of the studies were sub­mitted to Berlin’s IECs. Member was Prof. Joachim Röhmel (1990–2004 head of the bio­sta­tis­ti­cal department of the BfArM and an expert in adaptive designs).


  • To be more provocative: The papers were published in a peer-reviewed journal (IF 2.067), based on work supported by the “Product Quality Research Institute” (PQRI, a non-profit organization founded in 1999). Members amongst others are FDA/CDER, HC, USP, AAPS, CHPA, PhRMA. A working group on sequential designs in BE was founded in 2003 and it took them four years to get the work done. Amongst the authors are statistical ‘heavy weights’ like Walter Hauck and Donald Schuirmann. They recommended Method C. Full stop. Now {EMA/BSWP/whoever/——} pick out one part believing in the universal constant Pt 0.0294. On the other hand they seem to accept any other method supported by simulations performed by an obscure backyard statistician. I don’t get the point. :angry:
    End of rant.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Shuanghe
★★  

Spain,
2012-07-05 15:48
(4284 d 23:21 ago)

@ Helmut
Posting: # 8892
Views: 19,532
 

 Great!

Hi Helmut,

❝ Phantastic! Should be put in the same drawer like TOST is not valid in Nanjing – use the I Ching instead.”

:-D

❝ See also this thread and ElMaestro’s wonderful Pt.


The post is about method B or C. However, here the thing is bit different. What I forgot to mention is that after sending out synopsis based on method C one of the comments I got, apart from "not valid in Europe :blahblah:", is that "power does not count in order to make a decision. The decision can not depend on the results of the power."

I don't know how many ways we can interpret this phrase. Since Potvin's methods are all somehow based on power evaluation in its flowchart so may be this is the reason they say "not valid ..."? Anyway, this is the reason we changed synopsis to use another method without power evaluation as I mentioned.

❝ BTW, you are not alone (see this post).


Don't know if that's comforting or frustrating. :-|

❝ Seems that some regulators believe sic in B, but then they should be happy with the even more strict method D as well. Or are they really thinking that α 0.0294 is a universal constant?


If they don't like power evaluation in the flowchart, B and D should also be unacceptable to them.

❝ Great. This is not even Method B (abandoning the power estimation step).


Nope. We knew that and did it deliberately to avoid power evaluation.

❝ If the agency accepted your method without an intermediate power estimation they should ask themselves which risk to patients they are actually accepting.


Don't know if there's some kind of pact here (this is a BE for certain SUPAC and I'm not the one contact the agency). We were warned that the agency accept this study only in this specific country. It implied that the design might not be valid in other EU countries. It's a little bit complicate so I don't want to go to details.

For example, using 94.12% confidence intervals for both the analysis of stage 1 and the combined data from stage 1 and stage 2 would be acceptable, …

❝ That’s Method B!


My understanding is different. B involve power evaluation as well (After BE failed in stage 1, there's a step of evaluation power to determine if it should be stopped (Pt>= 80%) or continue (Pt<80%). What I understood from the guideline is that no power evaluation is involved in this example.

❝ ❝ Hi Helmut, it was nice talking to you in Budapest last month by the way.

❝ Same with me. Enjoyed especially the talks outside in the smoker’s Ghetto. :smoke:

It's a surprise that we actually talked about BE until 2 or 3 AM :-D :clap:.

❝ Your memory is correct; see this post. The BfArM (not in Europe according to the BSWP?) had no problems with Method C. Maybe it’s worthwhile noticing that two of the studies were submitted to Berlin’s IECs. Member was Prof. Joachim Röhmel (1990–2004 head of the biostatistical department of the BfArM and an expert in adaptive designs).


That's good to know. I didn't find this post before. Thanks.

Shuanghe

All the best,
Shuanghe
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-05 16:57
(4284 d 22:12 ago)

@ Shuanghe
Posting: # 8893
Views: 20,031
 

 Potvin & Montague not acceptable at all?!

Hi Shuanghe!

❝ ❝ See also this thread […]


❝ The post is about method B or C. However, here the thing is bit different. What I forgot to mention is that after sending out synopsis based on method C one of the comments I got, apart from "not valid in Europe :blahblah:", is that "power does not count in order to make a decision. The decision can not depend on the results of the power."


Phantastic № 2! Of course post hoc power is irrelevant in BE, but obviously these guys (of the agency or – even worse – the BSWP) did|do not comprehend that the intermediate [sic] power estimation is part of the framework of Potvin et al. & Montague et al.

❝ I don't know how many ways we can interpret this phrase.


Stupidity? Especially “The decision can not depend on the results of the power.” WTF does the apodictic “can not” mean? OK, the framework showed that α is maintained. Of course this does not imply that skipping this step will not work at well. We just don’t have any evidence (i.e., a published reference).1 I don’t have the means to set up simulations but would be a rather interesting task.2 ElMaestro?

❝ Since Potvin's methods are all somehow based on power evaluation in its flowchart so may be this is the reason they say "not valid ..."?


Only in the intermediate step! Finally (i.e., after requests of the type “The applicant showed BE, but power was only 70%. Please justify.”) they have realized that post hoc power is irrelevant. But this is not an issue in this framework. See the last sentence in the description of Methods B–C:

Stop here whether BE is met or not and regardless of the power achieved.

(my emphasis)

❝ Anyway, this is the reason we changed synopsis to use another method without power evaluation as I mentioned.


Great. Quod licet Iovi, non licet bovi. Unless I have seen simulations showing otherwise I have to assume that the patient’s risk is not controlled.

❝ ❝ Seems that some regulators believe sic in B, but then they should be happy with the even more strict method D as well. Or are they really thinking that α 0.0294 is a universal constant?

❝ If they don't like power evaluation in the flowchart, B and D should also be unacceptable to them.


According to what you said it seems so.3

❝ ❝ Great. This is not even Method B (abandoning the power estimation step).

❝ Nope. We knew that and did it deliberately to avoid power evaluation.


I understand. α?

❝ ❝ If the agency accepted your method without an intermediate power estimation they should ask themselves which risk to patients they are actually accepting.


❝ Don't know if there's some kind of pact here (this is a BE for certain SUPAC […]). We were warned that the agency accept this study only in this specific country. It implied that the design might not be valid in other EU countries.


❝ ❝ For example, using 94.12% confidence intervals for both the analysis of stage 1 and the combined data from stage 1 and stage 2 would be acceptable, …

❝ ❝ That’s Method B!


❝ My understanding is different. B involve power evaluation as well (After BE failed in stage 1, there's a step of evaluation power to determine if it should be stopped (Pt>= 80%) or continue (Pt<80%). What I understood from the guideline is that no power evaluation is involved in this example.


I meant that this is Method B, because 94.12% CIs are mentioned for both stages. In Method A 94.12% CIs are only applicable if power in step 1 <80%.

❝ ❝ Enjoyed especially the talks outside in the smoker’s Ghetto. :smoke:

❝ It's a surprise that we actually talked about BE until 2 or 3 AM :-D :clap:.


3 a.m. is not particular for me, but you and some other guys showed a lot of stamina!


  1. Appeal to ignorance—the claim that whatever has not been proved false must be true, and vice versa (e.g., There is no compelling evidence that UFOs are not visiting the Earth; therefore UFOs exist—and there is intelligent life elsewhere in the Universe. Or: There may be seventy kazillion other worlds, but not one is known to have the moral advancement of the Earth, so we’re still central to the Universe.) This impatience with ambiguity can be criticized in the phrase: absence of evidence is not evidence of absence.
    Carl Sagan
    The Demon-Haunted World: Science as a Candle in the Dark
    Ballantine, New York, p213 (1st ed. 1997)
    (my emphasis)
  2. Edit: In the meantime I can do better. See this post.
  3. Edit: End of July I saw a couple of deficiency letters. Some countries would accept Potvin B/C and Montague D if their applicability is justified for the particular study (aka a posteriori simulations) whilst others are not willing to accept intermediate power estimations at all! :angry:

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-07-05 18:11
(4284 d 20:58 ago)

@ Helmut
Posting: # 8894
Views: 19,525
 

 Powerless Potvin & Montague?

Dear Shuanghe, dear Helmut!

The whole thing you discuss here is a pure tragedy.
Such a lot of ignorance to scientific work and results is unbelievable :no:.
I'm at loss for words.

Regarding the intermediate power calculations:

Of course you can formulate the Potvin's decision scheme B without them.
But then it will happen that the sample size estimation step will give you a number which is smaller or at best equal to that you have already used for step 1.

Now what then?
Natural would be: STOP study. Bioequivalence not proven!
Exactly this is what the intermediate power calculation is for in scheme B.

I've heard some rumour from regulators that you should not stop but recruit 2 more subjects for stage 2 in that case. But this lacks any scientific justification IMHO.

Scheme C (D) is more complicated. At moment I don't have any idea if one can formulate it without intermediate power calculation. Seems a very integral part of that scheme.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-06 04:06
(4284 d 11:03 ago)

@ d_labes
Posting: # 8897
Views: 19,232
 

 Example w.o. intermediate power

Dear Detlew and all!

❝ Of course you can formulate the Potvin's decision scheme B without them.

❝ But then it will happen that the sample size estimation step will give you a number which is smaller or at best equal to that you have already used for step 1.


❝ Now what then?

❝ Natural would be: STOP study. Bioequivalence not proven!

❝ Exactly this is what the intermediate power calculation is for in scheme B.


Not for you, but a numerical example for non-initiates. Let’s consider Example 1 from Potvin’s paper (n1 12 failed in Stage 1 and ntotal 14):
  • Method B:
    Stage 1:
    94.12% CI: 104.27–134.17%, CVintra 14.56%
    Power at T/R 95%: 76.26% (exact; in the paper 75.6% based on shifted t)
    sampleN.TOST(alpha=0.0294, CV=0.1456, theta0=0.95) > ntotal 14 (n2 2)
    Pooled data:
    94.12% CI: 102.83–129.71%, CVintra 14.65%
  • No intermediate power:
    Outcome exactly as above.
Now I manipulated part of the data:
                        original
Subject Treatment Response manipulated
   1        T        3.3   →   2.9
   1        R        2.9       2.9
   2        T        5.3   →   6.1
   2        R        6.1       6.1
   3        T        6.2   →   6.3
   3        R        7.1       7.1
  • Method B:
    Stage 1:
    94.12% CI: 105.21–133.57%, CVintra 13.77%
    Power at T/R 95%: 80.99% (80.5% based on shifted t)
    Since power in Stage 1 ≥80% :stop: (fail)
  • No intermediate power:
    sampleN.TOST(alpha=0.0294, CV=0.1377, theta0=0.95) > ntotal 12 (n2 0) Yess!

❝ I've heard some rumour from regulators that you should not stop but recruit 2 more subjects for stage 2 in that case. But this lacks any scientific justification IMHO.


Yeah, but why not 2.718282 or 3.141593?

BTW, one regulator called Montague’s paper a “correction” of Potvin’s.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-07-06 10:13
(4284 d 04:56 ago)

@ Helmut
Posting: # 8898
Views: 19,223
 

 Example w.o. intermediate power

Dear Helmut!

❝ ❝ I've heard some rumour from regulators that you should not stop but recruit 2 more subjects for stage 2 in that case. But this lacks any scientific justification IMHO.


❝ Yeah, but why not 2.718282 or 3.141593?


:rotfl:

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-26 18:20
(4263 d 20:49 ago)

@ d_labes
Posting: # 8983
Views: 19,166
 

 Sims w.o. intermediate power

Dear Detlew & all!

❝ I've heard some rumour from regulators that you should not stop but recruit 2 more subjects for stage 2 in that case. But this lacks any scientific justification IMHO.


SCNR. Implemented your “rumour scheme”. In contrast to Potvin B, where a study might already fail in stage 1 (not BE at α 0.0294 and power ≥80%) we must continue to stage 2. Oh, only a few more subjects – presumably not an ethical concern to (some?) regulators… :not really:

[image]

Example 1: CV 20%, T/R 0.95, α 0.0294, expected power with n1 24 is 83.6%. Sample size for a fixed design, 80% power, α 0.05 would be 20. Run on two machines (R 2.15.1, PowerTOST 0.9-10), 106 simulations each.
Potvin B
            Ratio 1.25               │             Ratio 0.95             
─────────────────────────────────────┼─────────────────────────────────────
                      % in   empiric │                       % in   empiric
  n  ( 5%, 50%, 95%) stage 2    α    │   n  ( 5%, 50%, 95%) stage 2   1-β 
26.0  24   24   34    34.5   0.0320  │ 24.6  24   24   28     8.6    0.8810

“rumour scheme”
            Ratio 1.25               │             Ratio 0.95             
─────────────────────────────────────┼─────────────────────────────────────
                      % in   empiric │                       % in   empiric
  n  ( 5%, 50%, 95%) stage 2    α    │   n  ( 5%, 50%, 95%) stage 2   1-β 
27.2  26   26   34    97.1   0.0379  │ 24.7  24   24   28    16.3    0.9037
27.2  26   26   34    97.1   0.0379  │ 24.7  24   24   28    16.4    0.9030


[image]


Example 2: CV 30%, T/R 0.95, α 0.0294, expected power with n1 24 is 41.1%. Sample size for a fixed design, 80% power, α 0.05 would be 40. Sims as above.
Potvin B
            Ratio 1.25               │             Ratio 0.95             
─────────────────────────────────────┼─────────────────────────────────────
                      % in   empiric │                       % in   empiric
  n  ( 5%, 50%, 95%) stage 2    α    │   n  ( 5%, 50%, 95%) stage 2   1-β 
46.9  24   46   72    95.0   0.0475  │ 39.9  24   38   70    58.3    0.8305

“rumour scheme”
            Ratio 1.25               │             Ratio 0.95             
─────────────────────────────────────┼─────────────────────────────────────
                      % in   empiric │                       % in   empiric
  n  ( 5%, 50%, 95%) stage 2    α    │   n  ( 5%, 50%, 95%) stage 2   1-β 
46.8  26   46   72    97.1   0.0480  │ 39.8  24   36   70    58.8    0.8303
46.8  26   46   70    97.1   0.0482  │ 39.9  24   36   70    58.9    0.8303


[image]

Good news: In both examples the patient’s risk is preserved. But note that empiric α is slightly lower for Method B as compared to the ‘powerless method’.1 In other words, EMA’s ‘method’ is liberal. In the first example twice as much studies are send to the second stage (caused by the ‘n2=n1+2 rule’). Studies which would fail already in stage 1 according to Method B are forced to the second stage. :angry:

Differences in the second example are less pronounced, because n1 is likely too small to claim BE already in the first stage.

Lesson learned: I always thought of Two-Stage designs to offer an opportunity to ‘save’ a study if assumptions about the variance turn out to be incorrect. I was never a friend of playing chances and go with an interim analysis at a sample size where presumable you would fail and have to proceed to the second stage anyway.* Doubles the study time and even if your assumptions were correct you have to pay the penalty in the total sample size (10–20% more subjects as compared to a fixed design). Since exhaustive simulations of the ‘rumour scheme’ are not published (has EMA performed some at all or is this just a ‘gut-feeling-approach’?), IMHO it would require simulations for every single study [sic] to demonstrate that αemp. ≤0.05.2

P.S.: Thanks to Detlew for finding a bug in my code. ;-)


  1. Instead of ‘powerless’ can we introduce the term ‘weak’?
  2. For 106 simulations the one-sided level of significance based on the exact binominal test is 0.05035995. Only an αemp. larger than the critical value is significantly >0.05. Also mentioned in the note below Table I of Potvin’s paper. If you are patient and opt for 107 sims: 0.05011351… For R-freaks:
    sims <- 1E6
    x    <- 0.05
    binom.test(x*sims, sims, alternative='less')

  • For similar resons I fail to understand the application of O’Brien/Fleming in BE.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-07-05 18:20
(4284 d 20:49 ago)

@ Helmut
Posting: # 8895
Views: 19,299
 

 Potvin & Montague not acceptable at all?!

Hi all,

❝ Phantastic № 2! Of course post hoc power is irrelevant in BE, but obviously these guys (of the agency or – even worse – the BSWP) did|do not comprehend the intermediate [sic] power estimation is part of the framework of Potvin et al. & Montague et al.


❝ ❝ I don't know how many ways we can interpret this phrase.


❝ Stupidity? Especially “The decision can not depend on the results of the power.” WTF does the apodictic “can not” mean? OK, the framework showed that α is maintained. Of course this does not imply that skipping this step will not work at well. We just don’t have any evidence (i.e., a published reference). I don’t have the means to set up simulations but would be a rather interesting task. ElMaestro?


Yes, I can certainly set up some new simulations if necessary although this will be a spare time project and may take a lot of time. The author of the crackpot paper used at least what corresponds to two full months work time just to do the necessary programming, so I've heard.

One thing I learned from the 2010 revision (subjects as fixed even in replicate designs) is that regulators are not likely to admit when they have issued wobbly statements (I could use the term 'made mistakes' but have deliberately chosen not to). This is possibly just the human nature?!? Therefore, I am inclined to think that we need to think of a way to create a win-win situation so that whatever is the result of future simulations is something that allows regulators to lean back in the armchair with a cognac and think "Yeah, we were right. Again."
Probably they just mean that classically, power is not associated with regulatory risk (patient's risk) and therefore it isn't what their concern is. But for 2-stage designs, one may argue exceptions to this true general rule exist.
I am bewildered though, I am not sure what to simulate. If you guys can propose an Al Gore Rhythm (please make a graphic) then this would be a good starting point. Is it possible to write one which does not involve intermediary power steps? I mean, at the end we do need to think about how many subjects to include at stage 2 and that means power doesn't it (and don't we)? Lemme hear some more thoughts, please; I find it very interesting and this discussion very inspiring.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-05 18:36
(4284 d 20:33 ago)

@ ElMaestro
Posting: # 8896
Views: 19,351
 

 Sims?

Hi ElMaestro!

❝ ❝ […] OK, the framework showed that α is maintained. Of course this does not imply that skipping this step will not work at well. We just don’t have any evidence (i.e., a published reference). I don’t have the means to set up simulations but would be a rather interesting task. ElMaestro?


❝ Yes, I can certainly set up some new simulations if necessary although this will be a spare time project and may take a lot of time. The author of the crackpot paper used at least what corresponds to two full months work time just to do the necessary programming, so I've heard.


I can offer you just a beer – and fame!

❝ Probably they just mean that classically, power is not associated with regulatory risk (patient's risk) and therefore it isn't what their concern is.


Yep.

❝ But for 2-stage designs, one may argue exceptions to this true general rule exist.


I would say so. They mixed up post hoc power with intermediate power.

❝ I am bewildered though, I am not sure what to simulate. If you guys can propose an Al Gore Rhythm (please make a graphic) then this would be a good starting point.


Somefink similar to what Detlew suggested last year (but α1,2 0.0294 instead of α1 0.005 and α2 0.048).

❝ Is it possible to write one which does not involve intermediary power steps? I mean, at the end we do need to think about how many subjects to include at stage 2 and that means power doesn't it (and don't we)?


See also Detlew’s comments above.


Edit: See the simulations above.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
120 visitors (0 registered, 120 guests [including 4 identified bots]).
Forum time: 14:09 CET (Europe/Vienna)

With four parameters I can fit an elephant,
and with five I can make him wiggle his trunk.    John von Neumann

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5