Bioequivalence and Bioavailability Forum

Dear Esteemed Madam,¹

❝ One last suggestions I need from your end, if we go to repeat the whole study without any solid reason except the unexpected PE compared to the pilot study. What will be regulatory queries consequence with the regulatory approval. Will they approve our products based on the new data if it qualifies after repeat.

• Your pilot passed and the pivotal failed. Although asked for, you didn’t give us the PEs and CIs (or CVs), the target power (π) in study planning, and the final sample size(s). Let’s say you planned for 90% power and expected ≥80% power taking drop-outs into account. Producer’s risk β = 1 – π. In other words, with 80% power one out of five studies are expected to fail on pure chance. That’s life!
  
• Post hoc power is meaningless and should go straight into the statistical waste bin (remove it from your reports, it’s plain nonsense). The 100% you gave are outright wrong anyway. I strongly suggest to revise your estimation method. 100% power trans­lates into 0% chance of failure. But your study failed. Don’t you think that this is a contradiction? If you insist (for any wacky reason) to estimate post hoc power you must plug in the study’s PE into the formula. You might see that 79% will not even produce a result in some software. By definition power at the limits of the acceptance range will not exceed the level of the test α (patient’s risk). Hence, with any PE at or outside of the acceptance range – and with any CV! – power will be ≤5%.² You don’t even need soft­ware; this a consequence of the 90% confidence interval or TOSTs per­formed at 5%.
  
• Based on the above repeating the study would be both bad science³ and unethical.
  
• A PE outside the acceptance range from a properly powered study calls for re­for­mula­tion, IMHO.
  
• BTW, in my experience more studies fail in fed state than in fasting state. It is an in­com­prehensible mystery to me why many sponsors perform the fasting study first.
  
  
• Let’s assume you ignore our advice,⁴ repeat the study, and pass now. You have to sub­mit the synopsis of the failed study to the authority, which – since results are contra­dic­tory – immediately will ask for the complete report of the failed study. Authorities make their decision based on the whole body of evidence. Why should they trust results of the second pivotal study more than the first’s? The results of the pilot are supportive at its best. Or will you try to convince them that something (Namely what? Do you have any evidence?) went wrong in the first study (since you proposed to repeat the study in another CRO), rendering it less credible? In the best (!) case this will trigger an inspec­tion – not only of this study, but all others as well. Potential outcome:
  • No findings. Both studies are credible. Since the authority is interested in pro­tect­ing public health (not in the profit of the generic industry!) they will stay on the safe side and don’t approve the product.
    
  • Form 483 (FDA) or PSRtPH (EMA) issued – confirming the first study not credible. Maybe the second study will “count more” now.
    But: According to GCP it is the responsibility of the sponsor to guarantee proper study conduct (pre-study audit, qualification and training of personnel, pro­ce­dures, SOPs, QAU, study monitoring, …). Obviously you did not comply with these requirements. In the worst case it will retrospectively affect already accepted studies performed at this CRO.

1. See also this post.
   
2. Since you gave us no suitable numbers of the pivotal study my crystal ball whispered CV=24.5% and n=60. With a PE of 79% post hoc power is 2.70%. That’s not what I call an awful lot.
   
3. […] our greatest mistake would be to forget that data is used for serious decisions in the very real world, and bad information causes suffering and death. Ben Goldacre
   
4. The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. John W. Tukey