Bioequivalence and Bioavailability Forum

Hi Hötzi,

❝ Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 10⁶ sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require two-stage designs to be more conservative than fixed sample designs. Why?

(...)

❝ If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all – including your own – papers?

I get your point, but I see it a little differently. If the type I error rate is 0.0502 then this means that 'most likely' the true type I error rate exceeds the goal post of 0.05(000000000000 etc). But at 1e6 sims we just don't have statistical significance to refute a null hyp like P_t1e≤0.05. We could just as well increase the number of sims and discuss another new level of significance and so forth. Thus for practical purposes I would aim for 0.05 without too deep speculation into statistical significance at a given number of sims per scenario. If someone could come up with a convincing concept about clnically relevant inflation then I'd welcome it; until then 0.05(000000000000 etc) must be my own goalpost. Refutation of my own papers will not be necessary, I am sure they'll be forgotton before long anyway. And there are only two of them so I am a very small fish .