d_labes
★★★

Berlin, Germany,
2011-10-12 13:10
(4551 d 07:45 ago)

Posting: # 7474
Views: 18,514
 

 2-stage design - power evaluation [Two-Stage / GS Designs]

Dear All!

In this thread I had already raised some questions concerning the decision scheme according to Potvin Method B if one is willing to use "unsymmetrical" nominal alpha values, f.i. according to O'Brian-Fleming alpha1=0.005 and alpha2=0.048.

Now sitting here and wondering again especially about the power check step.
  • What is the rationale behind that step?
  • What alpha value should therefore used? alpha1 or alpha2?
  • How should the decision scheme read in case of "unsymmetrical" nominal alpha values?
As you see from these questions I'm very unsure. But meanwhile had arrived at the following (O'Brian-Fleming alphas):
  Evaluate BE at stage 1
  at alpha=alpha1=0.005
      ↓              ↓
if BE met:       if BE not met:
stop: success    Evaluate power with alpha2=0.048
                 at GMR=0.95
                     ↓                    ↓
                if power>80%           if power<80%
                Evaluate BE at stage1  Calculate sample size nest with GMR=0.95
                with alpha2=0.048      alpha2=0.048 and variance from stage1
                     ↓                    ↓
                stop: success          continue to stage 2 with n2=nest-n1 subjects
                      or fail          evaluate after stage 2 with alpha2=0.048
                                       using data from both stages
                                       success or fail

Any opinions about that?

Bonus question: Do you have an idea how decision scheme C should read?

More questions arise if the point estimator of stage 1 has to be taken into consideration. But that's another story.

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2011-10-26 11:14
(4537 d 09:42 ago)

@ d_labes
Posting: # 7543
Views: 16,819
 

 No opinions out there?

Dear All!

No opinions, no answers?

Meanwhile I had attempted to get an opinion from the authors of the Potvin et al. paper (mailed to the correspondence author Walter Hauck). But unfortunately don't got a response up to now.

Let me answer myself
(dropped this from the first post not to prejudice other opinions :cool:):

The decision scheme I have given for the case of 'unsymmetrical' alpha values for the two stages arose from my understanding (?) that the power check step is complementary to the sample size adaptation step.
If we have power >80% (or whatever target power we wish to achieve) the sample size calculation step with alpha2 (and this is undoubtedly the right alpha value) will give a sample size lower or equal to the size of stage 1. Thus we have no possibility to improve our BE result by raising the number of subjects and have to stay with the stage 1 data.

Now the question is stay with the result 'Not BE' with alpha1 = 0.005 f.i. or evaluate BE with alpha2. In my opinion the latter is natural in the light of the BE evaluation if a second stage is necessary.

In case of symmetrical alpha values (f.i. Pocock's alpha1 = alpha2 = 0.0294) this new BE evaluation is covered already by the very first BE evaluation and collapses to the result 'Stop: fail' with alpha2.
This is then the decision scheme B as given by Potvin et al.

Any body out there to prove my point of view wrong?
Any opinion would be highly appreciated!

Regards,

Detlew
ElMaestro
★★★

Denmark,
2011-10-26 13:15
(4537 d 07:40 ago)

@ d_labes
Posting: # 7544
Views: 16,769
 

 Opinion yes, answer no

Dear d_labes,

❝ (...) the power check step is complementary to the sample size adaptation step.


I tried to imagine what a decision tree could look like in the absence of a power evaluation. I did not come up with much useful. The ideal design imho minimises type I errors and maximises power (=minimises futility), generally. By all means, if you have a proposal for a scheme that works somehow without the power step then I think it would be worth publishing. There is too much interest in this topic and too few publication out there to let such an opportunity pass unnoticed.

Best regards,
EM.
d_labes
★★★

Berlin, Germany,
2011-10-26 15:00
(4537 d 05:56 ago)

@ ElMaestro
Posting: # 7545
Views: 16,990
 

 Decision scheme without (?) power check

Dear Ol Captain,

What about this one (variant of Potvin B):
  Evaluate BE at stage 1
  at alpha=alpha1=0.005
      ↓              ↓
if BE met:       if BE not met:
stop: success    Calculate sample size nest
                 with GMR=0.95, alpha2=0.048
                 and variance (CV) from stage 1
                     ↓                     ↓
                 if nest≤n1            if nest>n1
                     ↓                     ↓
             Evaluate BE at stage1     Continue to stage 2 with
             with alpha2=0.048          n2=nest - n1 additional subjects
                     ↓                     ↓
             Stop: sucess or fail      Evaluate after stage 2 with alpha2=0.048
                                       using data from both stages
                                               ↓
                                       success or fail


To be honest, this scheme of course is not without a power check :-D. The power check step is only hidden in testing nest ≤ or > n1.

Regards,

Detlew
ElMaestro
★★★

Denmark,
2011-10-26 15:20
(4537 d 05:36 ago)

@ d_labes
Posting: # 7546
Views: 16,817
 

 Decision scheme without (?) power check

Dear d_labes,

❝ What about this one (variant of Potvin B):


Alpha2 only kicks in when and if you know you are going into stage 2. Until then it is alpha1 everywhere. That would at least be least my understanding and intuition.

Best regards,
EM.
d_labes
★★★

Berlin, Germany,
2011-10-26 16:51
(4537 d 04:04 ago)

@ ElMaestro
Posting: # 7547
Views: 16,746
 

 Decision scheme 'without' power check

Dear ElMaestro,

❝ Alpha2 only kicks in when and if you know you are going into stage 2. Until then it is alpha1 everywhere. That would at least be least my understanding and intuition.


That would mean what in my last scheme hiding the power check step?
  • If we are going to stage 2 is clear (nest > n1).
  • Calculate sample size with alpha1 because we here don't know if we are going to stage 2? That would be against my intuition.
  • if nest ≤ n1 -> stop: fail with BE evaluated at alpha1?

Regards,

Detlew
ElMaestro
★★★

Denmark,
2011-10-26 17:24
(4537 d 03:31 ago)

@ d_labes
Posting: # 7548
Views: 16,866
 

 Decision scheme 'without' power check

Dear d_labes,

❝ That would mean what in my last scheme hiding the power check step?

  • If we are going to stage 2 is clear (nest > n1).

  • Calculate sample size with alpha1 because we here don't know if we are going to stage 2? That would be against my intuition.

  • if nest ≤ n1 -> stop: fail with BE evaluated at alpha1?

Oh bugger: I lost a code somewhere. How do I make the following appear black?

As long as "if nest≤n1" is essentially similar (pun intended) to "power>80 at alpha1" I think my walnut-sized brain agrees with you.

You got mail.

Best regards,
EM.


Edit: Sorry for the formating troubles in the quoted list. The regex in the PHP-script translating BBcodes to (X)HTML/CSS is not trivial. I couldn’t do better. Bug filed. ;-) [Helmut]
d_labes
★★★

Berlin, Germany,
2011-10-26 18:28
(4537 d 02:28 ago)

@ ElMaestro
Posting: # 7549
Views: 16,806
 

 Numerical example

Dear EM,

let's take a numerical example (again the OBF alpha's alpha1=0.005, alpha2=0.048):
Imagine after stage 1 with n=24 subjects we got CV1=0.2, PE=0.90.
The (1-2*alpha1 CI) is 0.7660 ... 1.0574 if I don't make a mistake. Thus 'not BE' if we use the common acceptance range 0.8 ... 1.25.
We know the stage 1 size is higher than the necessary sample size for a one-stage design with alpha=0.05 (n=20 for this alpha, CV=0.2 and true ratio 0.95).

Using the famous :cool: R-package PowerTOST we obtain:
power.TOST(alpha=0.005, CV=0.2, theta0=0.95, n=24) -> 0.5489
power.TOST(alpha=0.048, CV=0.2, theta0=0.95, n=24) -> 0.8919


sampleN.TOST(alpha=0.005,CV=0.2,theta0=0.95) -> 36
sampleN.TOST(alpha=0.048,CV=0.2,theta0=0.95) -> 20


In my schemes (using alpha2 for power check and/or sample size adaptation) you have to stop with stage 1. because the study is powered enough for the end alpha.
The choice to stay with the first BE evaluation with alpha1 will produce 'Not BE'. But for that alpha the power is <80%.
If you choose 'Evaluate BE with alpha2=0.048' the CI is 0.8148 ... 0.9941, i.e. 'BE shown' (except in Denmark :no:).

If you use alpha1 in all steps except stage 2 BE evaluation you end in an study with n=36, highly overpowered.

If you use alpha1 only in the power check step you have to re-calculate the sample size because power < 80%, but come out with nest=20 using alpha2=0.048 although you have already 24 subjects in the study. This is contradictory.

To get rid of all these curiosities I had arrived at the decision schemes mentioned in the posts above.

Regards,

Detlew
ElMaestro
★★★

Denmark,
2011-10-26 18:49
(4537 d 02:07 ago)

@ d_labes
Posting: # 7550
Views: 16,738
 

 Numerical example

Dear d_labes,

❝ Using the famous :cool: R-package PowerTOST we obtain:


That package is a great resource.

power.TOST(alpha=0.005, CV=0.2, theta0=0.95, n=24) -> 0.5489

power.TOST(alpha=0.048, CV=0.2, theta0=0.95, n=24) -> 0.8919


sampleN.TOST(alpha=0.005,CV=0.2,theta0=0.95) -> 36

sampleN.TOST(alpha=0.048,CV=0.2,theta0=0.95) -> 20


Hmmmm I get really uncertain now. I think you will hafta go with the 0.5489 value and procced to st. 2 with n=36 then?!?
I guess the real answer should come from a simulation of the scenario.

❝ If you use alpha1 in all steps except stage 2 BE evaluation you end in an study with n=36, highly overpowered.


Power 0.8919 is not so bad; it certainly looks within the range of commonly accepted values by ethics committees. Did you sim and check final power. It will be a little lower, won't it?

This isn't simple :confused:. Why don't we discuss method C in stead where the authors make a clear discussion between alpha1 and alpha2; it is for many practical purposes performing just like method B anyway, at least under the conditions tested by Potvin :-D.

And yeah, what the heck is wrong with the Danish people? What are they, nuts?

best regards,
em.
d_labes
★★★

Berlin, Germany,
2011-10-27 16:54
(4536 d 04:02 ago)

@ ElMaestro
Posting: # 7555
Views: 16,749
 

 Potvin C with unsymmetrical alphas

Dear ElMaestro,

❝ Hmmmm I get really uncertain now. I think you will hafta go with the 0.5489 value and procced to st. 2 with n=36 then?!?


No. My decision would be 'stay with stage 1'.

❝ This isn't simple :confused:.


I think so :-D.

❝ Why don't we discuss method C in stead where the authors make a clear discussion between alpha1 and alpha2; it is for many practical purposes performing just like method B anyway, at least under the conditions tested by Potvin :-D.


That's my bonus question from above.
I can not see the 'clear discussion' for alpha1 and alpha2 in the Potvin et al. paper. May be I'm some sort of blind :cool:.

For the acceptance of Potvin C see this post.
Oh, I see you have noticed it already.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-28 00:52
(4535 d 20:04 ago)

@ d_labes
Posting: # 7557
Views: 16,936
 

 Potvin B, C, or D?

Dear D. Labes!

❝ ❝ Why don't we discuss method C in stead where the authors make a clear discussion between alpha1 and alpha2; it is for many practical purposes performing just like method B anyway, at least under the conditions tested by Potvin :-D.


❝ I can not see the 'clear discussion' for alpha1 and alpha2 in the Potvin et al. paper.


Me not either. I’ve heard that regulators read this forum; so if they have no access to the paper here’s a snippet from its Section 5 (Discussion and Recommendations)

The goal of our group was to validate at least one method for two-stage designs that could be used for BE studies. Methods B and C meet our criteria of not more than minimal inflation of type I error rate. We recommend that regulatory agencies accept either. It is our understanding that the FDA has accepted studies with designs like those considered here.
For sponsors, there is a small power advantage to Method C over Method B, so we consider Method C as the method of choice. Another advantage of Method C is that it was designed so that if the study were found to have adequate power at the first stage, the α for that study would be the same as if it were designed to be single-stage study.


❝ For the acceptance of Potvin C see this post.


Terrible.
BTW, FDA recommends Method C (for T/R 0.95) and Method D (for T/R 0.90)… That’s not surprising with Donald Schuirmann as a co-author.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-31 03:36
(4532 d 16:19 ago)

@ d_labes
Posting: # 7561
Views: 16,765
 

 Two-stage (classical Pocock) and the FDA

Dear all!

Have you ever wondered what the last sentence is based upon?

Methods B and C meet our criteria of not more than minimal inflation of type I error rate. We recommend that regulatory agencies accept either. It is our understanding that the FDA has accepted studies with designs like those considered here.

Mystery revealed here.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2011-11-30 19:04
(4502 d 00:52 ago)

@ d_labes
Posting: # 7754
Views: 16,506
 

 Weirdo paper

Hmmm,

looks like some weirdo published a paper about alpha preservation for two-stage designs. He should be strapped to a gurney. Completely nuts.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-12-01 00:45
(4501 d 19:11 ago)

@ ElMaestro
Posting: # 7755
Views: 16,229
 

 Weirdo paper

My Capt’n!

Do you know whether the “PDF Plus” contains the simulation code for stage 2 or do we have to start over from scratch?

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2011-12-01 08:16
(4501 d 11:40 ago)

@ Helmut
Posting: # 7756
Views: 15,820
 

 Weirdo paper

Hi HS,

not sure what you mean?
The source code and excutable isn't in the public domain. Source written in C so needs a compiler.

Pass or fail!
ElMaestro
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
89 visitors (0 registered, 89 guests [including 13 identified bots]).
Forum time: 19:56 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5