Bioequivalence and Bioavailability Forum

d_labes
★★★

Berlin, Germany,
2011-10-12 13:10
(4946 d 14:08 ago)

Posting: # 7474
Views: 20,445

2-stage design - power evaluation [Two-Stage / GS Designs]

Dear All!

In this thread I had already raised some questions concerning the decision scheme according to Potvin Method B if one is willing to use "unsymmetrical" nominal alpha values, f.i. according to O'Brian-Fleming alpha₁=0.005 and alpha₂=0.048.

Now sitting here and wondering again especially about the power check step.

What is the rationale behind that step?
What alpha value should therefore used? alpha₁ or alpha₂?
How should the decision scheme read in case of "unsymmetrical" nominal alpha values?

As you see from these questions I'm very unsure. But meanwhile had arrived at the following (O'Brian-Fleming alphas):

  Evaluate BE at stage 1 

  at alpha=alpha₁=0.005

      ↓              ↓

if BE met:       if BE not met:

stop: success    Evaluate power with alpha₂=0.048

                 at GMR=0.95

                     ↓                    ↓

                if power>80%           if power<80%

                Evaluate BE at stage1  Calculate sample size n_est with GMR=0.95

                with alpha₂=0.048      alpha₂=0.048 and variance from stage1

                     ↓                    ↓

                stop: success          continue to stage 2 with n₂=n_est-n₁ subjects

                      or fail          evaluate after stage 2 with alpha2=0.048

                                       using data from both stages

                                       success or fail

Any opinions about that?

Bonus question: Do you have an idea how decision scheme C should read?

More questions arise if the point estimator of stage 1 has to be taken into consideration. But that's another story.

—
Regards,

Detlew

d_labes
★★★

Berlin, Germany,
2011-10-26 11:14
(4932 d 16:05 ago)

@ d_labes
Posting: # 7543
Views: 18,387

No opinions out there?

Post reply

Dear All!

No opinions, no answers?

Meanwhile I had attempted to get an opinion from the authors of the Potvin et al. paper (mailed to the correspondence author Walter Hauck). But unfortunately don't got a response up to now.

Let me answer myself
(dropped this from the first post not to prejudice other opinions :cool:

):

The decision scheme I have given for the case of 'unsymmetrical' alpha values for the two stages arose from my understanding (?) that the power check step is complementary to the sample size adaptation step.
If we have power >80% (or whatever target power we wish to achieve) the sample size calculation step with alpha₂ (and this is undoubtedly the right alpha value) will give a sample size lower or equal to the size of stage 1. Thus we have no possibility to improve our BE result by raising the number of subjects and have to stay with the stage 1 data.

Now the question is stay with the result 'Not BE' with alpha₁ = 0.005 f.i. or evaluate BE with alpha₂. In my opinion the latter is natural in the light of the BE evaluation if a second stage is necessary.

In case of symmetrical alpha values (f.i. Pocock's alpha₁ = alpha₂ = 0.0294) this new BE evaluation is covered already by the very first BE evaluation and collapses to the result 'Stop: fail' with alpha₂.
This is then the decision scheme B as given by Potvin et al.

Any body out there to prove my point of view wrong?
Any opinion would be highly appreciated!

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2011-10-26 13:15
(4932 d 14:03 ago)

@ d_labes
Posting: # 7544
Views: 18,352

Opinion yes, answer no

Post reply

Dear d_labes,

❝ (...) the power check step is complementary to the sample size adaptation step.

I tried to imagine what a decision tree could look like in the absence of a power evaluation. I did not come up with much useful. The ideal design imho minimises type I errors and maximises power (=minimises futility), generally. By all means, if you have a proposal for a scheme that works somehow without the power step then I think it would be worth publishing. There is too much interest in this topic and too few publication out there to let such an opportunity pass unnoticed.

Best regards,
EM.

d_labes
★★★

Berlin, Germany,
2011-10-26 15:00
(4932 d 12:19 ago)

@ ElMaestro
Posting: # 7545
Views: 18,615

Decision scheme without (?) power check

Post reply

Dear Ol Captain,

What about this one (variant of Potvin B):

  Evaluate BE at stage 1 

  at alpha=alpha₁=0.005

      ↓              ↓

if BE met:       if BE not met:

stop: success    Calculate sample size n_est

                 with GMR=0.95, alpha₂=0.048 

                 and variance (CV) from stage 1

                     ↓                     ↓

                 if n_est≤n₁            if n_est>n₁

                     ↓                     ↓

             Evaluate BE at stage1     Continue to stage 2 with

             with alpha₂=0.048          n₂=n_est - n₁ additional subjects

                     ↓                     ↓

             Stop: sucess or fail      Evaluate after stage 2 with alpha2=0.048

                                       using data from both stages

                                               ↓

                                       success or fail

To be honest, this scheme of course is not without a power check :-D

. The power check step is only hidden in testing n_est ≤ or > n₁.

—
Regards,

Detlew

ElMaestro ★★★ Denmark, 2011-10-26 15:20 (4932 d 11:59 ago) @ d_labes Posting: # 7546 Views: 18,396	Decision scheme without (?) power check Post reply
	Dear d_labes, ❝ What about this one (variant of Potvin B): Alpha2 only kicks in when and if you know you are going into stage 2. Until then it is alpha1 everywhere. That would at least be least my understanding and intuition. Best regards, EM.

d_labes
★★★

Berlin, Germany,
2011-10-26 16:51
(4932 d 10:27 ago)

@ ElMaestro
Posting: # 7547
Views: 18,320

Decision scheme 'without' power check

Post reply

Dear ElMaestro,

❝ Alpha2 only kicks in when and if you know you are going into stage 2. Until then it is alpha1 everywhere. That would at least be least my understanding and intuition.

That would mean what in my last scheme hiding the power check step?

If we are going to stage 2 is clear (n_est > n₁).
Calculate sample size with alpha₁ because we here don't know if we are going to stage 2? That would be against my intuition.
if n_est ≤ n₁ -> stop: fail with BE evaluated at alpha₁?

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2011-10-26 17:24
(4932 d 09:54 ago)

@ d_labes
Posting: # 7548
Views: 18,442

Decision scheme 'without' power check

Post reply

Dear d_labes,

❝ That would mean what in my last scheme hiding the power check step?

If we are going to stage 2 is clear (n_est > n₁).
Calculate sample size with alpha₁ because we here don't know if we are going to stage 2? That would be against my intuition.
❝
if n_est ≤ n₁ -> stop: fail with BE evaluated at alpha₁?

Oh bugger: I lost a code somewhere. How do I make the following appear black?

As long as "if n_est≤n1" is essentially similar (pun intended) to "power>80 at alpha₁" I think my walnut-sized brain agrees with you.

You got mail.

Best regards,
EM.

Edit: Sorry for the formating troubles in the quoted list. The regex in the PHP-script translating BBcodes to (X)HTML/CSS is not trivial. I couldn’t do better. Bug filed. ;-)

[Helmut]

d_labes
★★★

Berlin, Germany,
2011-10-26 18:28
(4932 d 08:51 ago)

@ ElMaestro
Posting: # 7549
Views: 18,394

Numerical example

Post reply

Dear EM,

let's take a numerical example (again the OBF alpha's alpha₁=0.005, alpha₂=0.048):
Imagine after stage 1 with n=24 subjects we got CV1=0.2, PE=0.90.
The (1-2*alpha₁ CI) is 0.7660 ... 1.0574 if I don't make a mistake. Thus 'not BE' if we use the common acceptance range 0.8 ... 1.25.
We know the stage 1 size is higher than the necessary sample size for a one-stage design with alpha=0.05 (n=20 for this alpha, CV=0.2 and true ratio 0.95).

Using the famous :cool:

R-package PowerTOST we obtain:


power.TOST(alpha=0.005, CV=0.2, theta0=0.95, n=24) -> 0.5489 

power.TOST(alpha=0.048, CV=0.2, theta0=0.95, n=24) -> 0.8919

sampleN.TOST(alpha=0.005,CV=0.2,theta0=0.95) -> 36

sampleN.TOST(alpha=0.048,CV=0.2,theta0=0.95) -> 20

In my schemes (using alpha₂ for power check and/or sample size adaptation) you have to stop with stage 1. because the study is powered enough for the end alpha.
The choice to stay with the first BE evaluation with alpha1 will produce 'Not BE'. But for that alpha the power is <80%.
If you choose 'Evaluate BE with alpha2=0.048' the CI is 0.8148 ... 0.9941, i.e. 'BE shown' (except in Denmark :no:

).

If you use alpha₁ in all steps except stage 2 BE evaluation you end in an study with n=36, highly overpowered.

If you use alpha1 only in the power check step you have to re-calculate the sample size because power < 80%, but come out with n_est=20 using alpha₂=0.048 although you have already 24 subjects in the study. This is contradictory.

To get rid of all these curiosities I had arrived at the decision schemes mentioned in the posts above.

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2011-10-26 18:49
(4932 d 08:30 ago)

@ d_labes
Posting: # 7550
Views: 18,337

Numerical example

Post reply

Dear d_labes,

❝ Using the famous :cool: R-package PowerTOST we obtain:

That package is a great resource.

❝ power.TOST(alpha=0.005, CV=0.2, theta0=0.95, n=24) -> 0.5489

❝ power.TOST(alpha=0.048, CV=0.2, theta0=0.95, n=24) -> 0.8919

❝

❝ sampleN.TOST(alpha=0.005,CV=0.2,theta0=0.95) -> 36

❝ sampleN.TOST(alpha=0.048,CV=0.2,theta0=0.95) -> 20

Hmmmm I get really uncertain now. I think you will hafta go with the 0.5489 value and procced to st. 2 with n=36 then?!?
I guess the real answer should come from a simulation of the scenario.

❝ If you use alpha₁ in all steps except stage 2 BE evaluation you end in an study with n=36, highly overpowered.

Power 0.8919 is not so bad; it certainly looks within the range of commonly accepted values by ethics committees. Did you sim and check final power. It will be a little lower, won't it?

This isn't simple :confused:

. Why don't we discuss method C in stead where the authors make a clear discussion between alpha1 and alpha2; it is for many practical purposes performing just like method B anyway, at least under the conditions tested by Potvin :-D

.

And yeah, what the heck is wrong with the Danish people? What are they, nuts?

best regards,
em.

d_labes
★★★

Berlin, Germany,
2011-10-27 16:54
(4931 d 10:24 ago)

@ ElMaestro
Posting: # 7555
Views: 18,341

Potvin C with unsymmetrical alphas

Post reply

Dear ElMaestro,

❝ Hmmmm I get really uncertain now. I think you will hafta go with the 0.5489 value and procced to st. 2 with n=36 then?!?

No. My decision would be 'stay with stage 1'.

❝ This isn't simple :confused: .

I think so :-D

❝ Why don't we discuss method C in stead where the authors make a clear discussion between alpha1 and alpha2; it is for many practical purposes performing just like method B anyway, at least under the conditions tested by Potvin :-D .

That's my bonus question from above.
I can not see the 'clear discussion' for alpha1 and alpha2 in the Potvin et al. paper. May be I'm some sort of blind :cool:

.

For the acceptance of Potvin C see this post.
Oh, I see you have noticed it already.

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2011-10-28 00:52
(4931 d 02:27 ago)

@ d_labes
Posting: # 7557
Views: 18,519

Potvin B, C, or D?

Post reply

Dear D. Labes!

❝ ❝ Why don't we discuss method C in stead where the authors make a clear discussion between alpha1 and alpha2; it is for many practical purposes performing just like method B anyway, at least under the conditions tested by Potvin :-D .

❝

❝ I can not see the 'clear discussion' for alpha1 and alpha2 in the Potvin et al. paper.

Me not either. I’ve heard that regulators read this forum; so if they have no access to the paper here’s a snippet from its Section 5 (Discussion and Recommendations)

The goal of our group was to validate at least one method for two-stage designs that could be used for BE studies. Methods B and C meet our criteria of not more than minimal inﬂation of type I error rate. We recommend that regulatory agencies accept either. It is our understanding that the FDA has accepted studies with designs like those considered here.
For sponsors, there is a small power advantage to Method C over Method B, so we consider Method C as the method of choice. Another advantage of Method C is that it was designed so that if the study were found to have adequate power at the ﬁrst stage, the α for that study would be the same as if it were designed to be single-stage study.

❝ For the acceptance of Potvin C see this post.

Terrible.
BTW, FDA recommends Method C (for T/R 0.95) and Method D (for T/R 0.90)… That’s not surprising with Donald Schuirmann as a co-author.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Helmut
★★★

Vienna, Austria,
2011-10-31 03:36
(4927 d 22:42 ago)

@ d_labes
Posting: # 7561
Views: 18,372

Two-stage (classical Pocock) and the FDA

Post reply

Dear all!

Have you ever wondered what the last sentence is based upon?

Methods B and C meet our criteria of not more than minimal inﬂation of type I error rate. We recommend that regulatory agencies accept either. It is our understanding that the FDA has accepted studies with designs like those considered here.

Mystery revealed here.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro ★★★ Denmark, 2011-11-30 19:04 (4897 d 07:15 ago) @ d_labes Posting: # 7754 Views: 18,111	Weirdo paper Post reply
	Hmmm, looks like some weirdo published a paper about alpha preservation for two-stage designs. He should be strapped to a gurney. Completely nuts. — Pass or fail! ElMaestro

Helmut ★★★ Vienna, Austria, 2011-12-01 00:45 (4897 d 01:34 ago) @ ElMaestro Posting: # 7755 Views: 17,826	Weirdo paper Post reply
	My Capt’n! Do you know whether the “PDF Plus” contains the simulation code for stage 2 or do we have to start over from scratch? — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes

ElMaestro ★★★ Denmark, 2011-12-01 08:16 (4896 d 18:03 ago) @ Helmut Posting: # 7756 Views: 17,406	Weirdo paper Post reply
	Hi HS, not sure what you mean? The source code and excutable isn't in the public domain. Source written in C so needs a compiler. — Pass or fail! ElMaestro

2-stage design - power evaluation [Two-Stage / GS Designs]

No opinions out there?

Opinion yes, answer no

Decision scheme without (?) power check

Decision scheme without (?) power check

Decision scheme 'without' power check

Decision scheme 'without' power check

Numerical example

Numerical example

Potvin C with unsymmetrical alphas

Potvin B, C, or D?

Two-stage (classical Pocock) and the FDA

Weirdo paper

Weirdo paper

Weirdo paper