Helmut Hero Vienna, Austria, 20131030 18:48 Posting: # 11822 Views: 3,788 

Dear all, the neverending story (this time RMS Austria, AGES 2013/10): The Applicant has preplanned for a 2stage sequential design and such is principally acceptable. The Applicant has prespecified to follow the approach as outlined in Potvin et al. (2007), where the type I error control is based on simulations and a small inflation (of 0.2%, onesided) was considered acceptable by the authors. However, this would mean that the trial would be allowed a larger type I error than in a more conventional design. This general consideration could in principle turn out to be critical in the case the bioequivalence criteria are met only sharply. However, as the study was stopped for inequivalence, any further considerations on type I error control become obsolete. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro Hero Denmark, 20131030 19:26 @ Helmut Posting: # 11823 Views: 3,310 

Hi Hötzi, I must say I have some sympathy for this view from AGES. To me this proves that occasionally even Austrians say something meaningful... The whole idea about 0.052 as a kind of clinically relevant limit which popped up in Potvin's paper is taken out of the clean blue Canadian air. If we want to control the type I error rate at 5% then my interpretation is we should mean 5% and this should not so much be a discussion of decimals or even the 0.05036 binomial limit at one million sims. In perspective: eu regulators want the CI's to be within 80.00% to 125.00%  four or five significant digits or two decimals? If the latter is the case then we could try to argue accordingly that 0.054999 is not inflation. Yuck 0.05 rules. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
Helmut Hero Vienna, Austria, 20131031 15:01 @ ElMaestro Posting: # 11841 Views: 3,231 

Hi ElMaestro, » […] occasionally even Austrians say something meaningful... True. But this is a bad example. » The whole idea about 0.052 as a kind of clinically relevant limit which popped up in Potvin's paper is taken out of the clean blue Canadian air. Well, NorthAmerica’s air. The majority of authors are from the US. Irrelevant employers like FDA, USP… » If we want to control the type I error rate at 5% then my interpretation is we should mean 5% and this should not so much be a discussion of decimals or even the 0.05036 binomial limit at one million sims. Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 10^{6} sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require twostage designs to be more conservative than fixed sample designs. Why? Every year my book shelf gains about 1 kg since I’m a subscriber to Biometrics. Many, many simulations of incomprehensibly complicated models. Quite often n=10,000 (sign. limit 0.0537). If a value of 0.064 pops up, in the discussion it is called ‘minimal inflation’. These methods are applied in phase III. Regulators don’t care. Slowly I get tired about this double standard. Consider going for a hike to Similaun. » eu regulators want the CI's to be within 80.00% to 125.00%  four or five significant digits or two decimals? If the latter is the case… Seems so. IMHO this entire rounding business is bullshit. According to the (admittedly arbitrary) definition we accept a ∆ [sic] of 20% as clinically not relevant. After long discussions about the distribution / transformation the limits were set to AL_{lo} = 1∆ = 0.8 and AL_{hi} = (1∆)^{1} = 1.25 precisely. Why not AL_{hi} = 1+∆ = 1.20 and AL_{lo} = (1+∆)^{1} = 83.3? Because in the former case the numbers look nicer and are easier to remember (seriously: this was the winning argument in the mid 1980s). That’s what I call a proper justification. Every software could check whether CI_{lo} < AL_{lo} and CI_{hi} > AL_{hi} in full precision. Phoenix/WinNonlin in its standard format reports the CI to four decimals and gives a verbatim statement (based on the full precision comparison) in the next line. Looks like this:
Canadians regulators have a special love affair with nice numbers and set the upper limit for NTIDs instead of 111.11 to 112.0% (one decimal)… Should we do triple rounding in ABEL? OK, I have learned that k (derived from ln(1.25)/√ln(0.3^{2}+1 = 0.760128298…) should be treated as 0.76 (two decimals). For CV_{WR} 50% the scaled limits are 69.84–143.19% (rounding #2). Now assume different lower CLs. AL (exact) AL (rounded) AL (0.76) AL (0.76 rounded) Crap. I guess EMA wants the triple rounding (red in the 4^{th} column). Red in the 1^{st} column shows a disagreement with the exact comparison. » … then we could try to argue accordingly that 0.054999 is not inflation. Yuck Don’t get it. » 0.05 rules. If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all – including your own – papers? Happy simulating, Hötzi — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro Hero Denmark, 20131031 18:16 @ Helmut Posting: # 11846 Views: 3,184 

Hi Hötzi, » Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 10^{6} sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require twostage designs to be more conservative than fixed sample designs. Why? (...) » If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all – including your own – papers? I get your point, but I see it a little differently. If the type I error rate is 0.0502 then this means that 'most likely' the true type I error rate exceeds the goal post of 0.05(000000000000 etc). But at 1e6 sims we just don't have statistical significance to refute a null hyp like P_{t1e}≤0.05. We could just as well increase the number of sims and discuss another new level of significance and so forth. Thus for practical purposes I would aim for 0.05 without too deep speculation into statistical significance at a given number of sims per scenario. If someone could come up with a convincing concept about clnically relevant inflation then I'd welcome it; until then 0.05(000000000000 etc) must be my own goalpost. Refutation of my own papers will not be necessary, I am sure they'll be forgotton before long anyway. And there are only two of them so I am a very small fish . — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
Shuanghe Regular Spain, 20131031 10:48 @ Helmut Posting: # 11836 Views: 3,173 

Hi, » .. is principally acceptable. ... When I saw this I knew something bad gonna happen. » ... However, as the study was stopped for inequivalence, any further considerations on type I error control become obsolete. OK, that's not what I was expecting. I don't know if Elmaestro and I were reading the same thing but the hidden line I've got is that: ok, you are "lucky" that your formulation screwed so you have to stop after stage 1; if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc... How's that "... occasionally even Austrians say something meaningful. " — All the best, Shuanghe 
Helmut Hero Vienna, Austria, 20131031 12:51 @ Shuanghe Posting: # 11838 Views: 3,175 

Hi Shuanghe, » […] the hidden line I've got is that: » » ok, you are "lucky" that your formulation screwed so you have to stop after stage 1; if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc... This is exactly my interpretation. BTW, it was ‘Method C’ as usual. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro Hero Denmark, 20131031 13:04 @ Shuanghe Posting: # 11839 Views: 3,162 

Hi Shuanghe, » (...) if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc... » How's that "... occasionally even Austrians say something meaningful. " This is exactly the practical application of type I error rates. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 