Bioequivalence and Bioavailability Forum

Hi Shuanghe!

❝ I had some unofficial feedback regarding sequential design just a few days ago. We send a protocol synopsis with statistical analysis details with Method C from Potvin's article and were told that "Potvin's method is not valid in Europe".

Phantastic! Should be put in the same drawer like

“TOST is not valid in Nanjing – use the I Ching instead.”

See also this thread and ElMaestro’s wonderful P_t. BTW, you are not alone (see this post).

❝ I have no idea if that means all method ABCD (well, A is not recommended so I could probably take it off) or just method C since that is the one we used in a synopsis.

I would guess methods A & C. Seems that some regulators believe [sic] in B, but then they should be happy with the even more strict method D as well. Or are they really thinking that α 0.0294 is a universal constant?

❝ […] after some discussion back and forth we end up with one of the examples mentioned in EMA's BE guideline and got OK from agency. Basically, we do stage 1 BE, analysis with alpha=0.0294, if BE, stop, if not, calculate sample size for stage 2 and analyse all data with alpha=0.0294 again.

Great. This is not even Method B (abandoning the power estimation step). You are leaving the validated range of Potvin’s paper; the patient’s is risk is not controlled any more. But let’s look at the GL again:

Two-stage design
It is acceptable to use a two-stage approach when attempting to demonstrate bioequi­va­lence. An initial group of subjects can be treated and their data analysed. If bioequivalence has not been demonstrated an additional group can be recruited and the results from both groups combined in a final analysis. If this approach is adopted appropriate steps must be taken to preserve the overall type I error of the experiment and the stopping criteria should be clearly defined prior to the study. The analysis of the first stage data should be treated as an interim analysis and both analyses conducted at adjusted significance levels (with the confidence intervals accordingly using an adjusted coverage probability which will be higher than 90%). For example, using 94.12% confidence intervals for both the analysis of stage 1 and the combined data from stage 1 and stage 2 would be acceptable, but there are many acceptable alternatives and the choice of how much alpha to spend at the interim analysis is at the company’s discretion. The plan to use a two-stage approach must be pre-spe­cified in the protocol along with the adjusted significance levels to be used for each of the analyses.
When analysing the combined data from the two stages, a term for stage should be included in the ANOVA model.

• An initial group of subjects can be treated and their data analysed. If bioequivalence has not been demonstrated an additional group can be recruited …
  No power estimation in the interim analysis! See this thread. I saw a scientific advice from EMA (2011) where O’Brian-Fleming was suggested by the applicant and EMA wanted to see simulations showing that empiric α will not exceed 0.05. If the agency accepted your method without an intermediate power estimation they should ask themselves which risk to patients they are actually accepting.
  
• … and the results from both groups combined in a final analysis.
  OK, doesn’t speak against Potvin; no poolability criterion.
  
• … both analyses conducted at adjusted significance levels (with the confidence intervals accordingly using an adjusted coverage probability which will be higher than 90%).
  Aha, here we are (both analyses conducted at adjusted significance levels ≠ Method C)!
  
• For example, using 94.12% confidence intervals for both the analysis of stage 1 and the combined data from stage 1 and stage 2 would be acceptable, …
  That’s Method B!
  
• … but there are many acceptable alternatives and the choice of how much alpha to spend at the interim analysis is at the company’s discretion.
  F.i. Potvin et al. and Montague et al. Method D. See the request in the scientific advice above for O’Brian-Fleming.
  
• The plan to use a two-stage approach must be pre-specified in the protocol along with the adjusted significance levels to be used for each of the analyses.
  Note the plural (significance levels)! If EMA is accepting O’Brian-Fleming (after demonstration of empiric α ≤0.05), why do they have problems with Method C, where this was already demonstrated and is recommended by both the FDA and in Canada?^*

❝ I got the opinion of "not valid in Europe" unofficially from certain agency who in turn got it from BSWP (which we assume is BioStatistics Working Party). I'm not sure it will be reflected somewhere in an official document (e.g QA) or not.

I don’t know of any document except the ambiguous GL itself.

❝ Hi Helmut, it was nice talking to you in Budapest last month by the way.

Same with me. Enjoyed especially the talks outside in the smoker’s Ghetto.

❝ I remember that you had several protocols accepted by certain agency using adeptive sequential design. I thought they were based on method C? Now since I have this feedback, I started to doubt my memory. maybe I mixed up what I read and what I heard.

Your memory is correct; see this post. The BfArM (not in Europe according to the BSWP?) had no problems with Method C. Maybe it’s worthwhile noticing that two of the studies were sub­mitted to Berlin’s IECs. Member was Prof. Joachim Röhmel (1990–2004 head of the bio­sta­tis­ti­cal department of the BfArM and an expert in adaptive designs).

---

• To be more provocative: The papers were published in a peer-reviewed journal (IF 2.067), based on work supported by the “Product Quality Research Institute” (PQRI, a non-profit organization founded in 1999). Members amongst others are FDA/CDER, HC, USP, AAPS, CHPA, PhRMA. A working group on sequential designs in BE was founded in 2003 and it took them four years to get the work done. Amongst the authors are statistical ‘heavy weights’ like Walter Hauck and Donald Schuirmann. They recommended Method C. Full stop. Now {EMA/BSWP/whoever/——} pick out one part believing in the universal constant P_t 0.0294. On the other hand they seem to accept any other method supported by simulations performed by an obscure backyard statistician. I don’t get the point.
  End of rant.