Bioequivalence and Bioavailability Forum

Dear Detlew!

❝ Don't overheat your machines .

Thanks for reminding me. Have to look – ah, 49% done so far.

❝ If you mean Method D

Oops, sure.

❝ I guess: also from some trial and error.

Here we are!

❝ Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe?

Why not – since we don’t have a method which is not based on simulations. Obviously some regulators don’t trust PQRI’s simulations and are asking for a posteriori simulations of the actual n₁ and CV – why shouldn’t they accept them if performed before the study?

❝ It's a similar question a good friend of our ol' pirate EM was once asked in respect to his method (see paper 6 of your post above):

❝ "Do you think that it is acceptable to any regulator in Europe that the alpha of stage 2 isn't chosen a priori but depending on the results of stage 1, again via some simulation studies?"

If one comes up with an α_adj. through simulations, it will be fixed in both stages and can be specified in the protocol. BTW, the lacking acceptance of an adjusted α₂ gives me headaches.

---

Results of 10⁶ sims, n₁ 36, CV 30%:
1 Method B, α_adj. 0.0294
2 Method B on steroids wo intermediate power, α_adj. 0.0380
              Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1  47.6  36   46   66   81.1    0.0397   │ 40.7  36   36   62   29.0   0.8379
2  44.3  36   42   60   70.63   0.047938 │ 39.0  36   36   56   22.43  0.84041
Risk I is maintained and we can expect a slight increase in power. ~23% less studies proceed to stage 2 (θ 0.95). May use 92.40% CI instead of 94.12%.

❝ What if the CV comes out other than supposed in the planning?

Good point! In order to be protected against that sims should cover a realistic (!) range of CVs (and n₁ – drop outs!), not only the expected one. If the (mandatory?) a posteriori sim shows α_emp. >0.05, bad luck. If we go with a fixed sample design instead and the CV turns out to be higher than expected we may fail as well. Part of the game. But let’s simulate. Method B on steroids wo power, α_adj. 0.0380, but CV 35%.
           Ratio 1.25                 │             Ratio 0.95             
──────────────────────────────────────┼─────────────────────────────────────
                     % in    empiric  │                      % in   empiric
 n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
56.8  36   56   80   92.10   0.057127 │ 46.7  36   36   78   42.2   0.83226
Ouch! Bad idea. Should have payed more attention to our ol’ pirate’s Figure 3; below transformed into a contour plot:



Try this goodie (enlarge the plot and drag around):
require(rgl)
x <- seq(12, 60, by = 12)
y <- seq(10, 100, by = 10)
B <- matrix(data = c(
       0.0297,0.0463,0.0437,0.0344,0.0309,0.0297,0.0294,0.0292,0.0289,0.0291,
       0.0294,0.0320,0.0475,0.0433,0.0338,0.0307,0.0299,0.0298,0.0298,0.0298,
       0.0294,0.0294,0.0397,0.0485,0.0420,0.0333,0.0306,0.0303,0.0296,0.0298,
       0.0292,0.0292,0.0324,0.0458,0.0484,0.0399,0.0328,0.0303,0.0297,0.0297,
       0.0294,0.0297,0.0296,0.0409,0.0483,0.0466,0.0381,0.0318,0.0300,0.0301),
       nrow = 5, ncol = 10, byrow = TRUE, dimnames = NULL)
rownames(B) <- as.character(x)
colnames(B) <- paste(as.character(y),"%")
persp3d(x, y, B,
  xlim = c(12, 60), xlab = "n1",
  ylim = c(10, 100), ylab = "CV%",
  zlim = c(0.025, 0.05), zlab = "empiric alpha",
  main = "Potvin B (Table I)",
  aspect = c(1, 4/3, 1),
  color = rgb(0.7, 0.9, 1, 0.75), smooth = TRUE, lit = TRUE)

α_emp. is ~0.05 if we are both close to the intended n₁ and expected CV but drops rapidly to ~α_adj. otherwise. In other words the level becomes conservative. I still think that it should be possible to raise α_adj. but with great caution and only after a lot of simulations.

Example: What can we do if we plan for n₁ 36, CV 30% and what will happen if the CV is higher? From the above we guess that our α_adj. for 30% is limited by the maximum possible α_emp. at CV 40%.
1 Method B, α_adj. 0.0294, 2 Method B on steroids wo power, α_adj. 0.0303, 10⁶ sims.
CV 30%        Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1  47.6  36   46   66   81.1    0.0397   │ 40.7  36   36   62   29.0   0.8379
2  47.1  36   46   66   79.82   0.040450 │ 40.5  36   36   60   28.08  0.83735
CV 40%        Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1  78.3  48   78  112   97.0    0.0485   │ 67.3  36   70  112   66.3   0.8236
2  66.3  36   68  110   96.95   0.049704 │ 66.3  36   68  110   65.47  0.82377
CV 50%        Ratio 1.25                 │             Ratio 0.95             
─────────────────────────────────────────┼─────────────────────────────────────
                        % in    empiric  │                      % in   empiric
    n   (5%, 50%, 95%) stage 2     α     │  n   (5%, 50%, 95%) stage 2   1-β 
1 116.8  74  116  166   98.5    0.0420   │127.7  36  116  166   90.8   0.8052
2 115.5  72  114  166   98.41   0.043799 │111.1  36  114  166   90.25  0.80468
Have we gained something substantially? Not at all. With 50% we have crossed the ridge at 40% and don’t have to worry any more. Maybe that’s really a stupid idea and not worth the efforts – only to come up with a slightly narrower 93.94% CI.

After all this stuff coming back to your question again….

❝ Do you think doing some simulations in the planning phase and deciding the alpha's based on them will be acceptable to the mighty oracles in Europe?

Taking the GL and the Dutch deficiency letter from above into account at least adaption of α₂ seems to be very tough. Coincidentally I have heard the term ‘cookbook’ for the first time from a Dutch regulator back in 2004. Little chances for ‘true’ adaptive designs.

Now for the positive part. If one follows Method B, IMHO a posteriori simulations are not necessary if the study was planned with a n₁ corresponding to max. α_emp.. That would mean that we are already on ‘top of the ridge’. What could happen?

• Drop-outs: We go left in the plot and see the test becomes conservative.
  
• Higher/lower CV than expected: We go up/down towards conservative levels.

BTW, empiric power is also interesting:



If we plan the study with a n₁ like a fixed design, power will be always >80% because we get a ‘second chance’ of showing BE in stage 2 (e.g., CV 30%, Power 81.6% for n 40 in a fixed design but ~85% with n₁ 40).