Log in
Register|
Helmut ★★★   Vienna, Austria, 2013-10-30 19:48 (3823 d 06:57 ago) Posting: # 11822 Views: 6,788 |
|
|
Dear all, the neverending story (this time RMS Austria, AGES 2013/10): The Applicant has preplanned for a 2-stage sequential design and such is principally acceptable. The Applicant has prespecified to follow the approach as outlined in Potvin et al. (2007), where the type I error control is based on simulations and a small inflation (of 0.2%, one-sided) was considered acceptable by the authors. However, this would mean that the trial would be allowed a larger type I error than in a more conventional design. This general consideration could in principle turn out to be critical in the case the bioequivalence criteria are met only sharply. However, as the study was stopped for inequivalence, any further considerations on type I error control become obsolete. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png) Helmut Schütz ![[image]](https://static.bebac.at/img/CC by.png) The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
ElMaestro ★★★ Denmark, 2013-10-30 20:26 (3823 d 06:19 ago) @ Helmut Posting: # 11823 Views: 5,925 |
|
|
Hi Hötzi, I must say I have some sympathy for this view from AGES. To me this proves that occasionally even Austrians say something meaningful...  The whole idea about 0.052 as a kind of clinically relevant limit which popped up in Potvin's paper is taken out of the clean blue Canadian air. If we want to control the type I error rate at 5% then my interpretation is we should mean 5% and this should not so much be a discussion of decimals or even the 0.05036 binomial limit at one million sims. In perspective: eu regulators want the CI's to be within 80.00% to 125.00% - four or five significant digits or two decimals? If the latter is the case then we could try to argue accordingly that 0.054999 is not inflation. Yuck 0.05 rules. — Pass or fail! ElMaestro |
|
Helmut ★★★ Vienna, Austria, 2013-10-31 16:01 (3822 d 10:45 ago) @ ElMaestro Posting: # 11841 Views: 6,020 |
|
|
Hi ElMaestro, ❝ […] occasionally even Austrians say something meaningful... True. But this is a bad example. ❝ The whole idea about 0.052 as a kind of clinically relevant limit which popped up in Potvin's paper is taken out of the clean blue Canadian air. Well, North-America’s air. The majority of authors are from the US. Irrelevant employers like FDA, USP… ❝ If we want to control the type I error rate at 5% then my interpretation is we should mean 5% and this should not so much be a discussion of decimals or even the 0.05036 binomial limit at one million sims. Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 106 sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require two-stage designs to be more conservative than fixed sample designs. Why? Every year my book shelf gains about 1 kg since I’m a subscriber to Biometrics. Many, many simulations of incomprehensibly complicated models. Quite often n=10,000 (sign. limit 0.0537). If a value of 0.064 pops up, in the discussion it is called ‘minimal inflation’. These methods are applied in phase III. Regulators don’t care. Slowly I get tired about this double standard. Consider going for a hike to Similaun. ❝ eu regulators want the CI's to be within 80.00% to 125.00% - four or five significant digits or two decimals? If the latter is the case… Seems so. IMHO this entire rounding business is bullshit. According to the (admittedly arbitrary) definition we accept a ∆ [sic] of 20% as clinically not relevant. After long discussions about the distribution / transformation the limits were set to ALlo = 1-∆ = 0.8 and ALhi = (1-∆)-1 = 1.25 precisely. Why not ALhi = 1+∆ = 1.20 and ALlo = (1+∆)-1 = 83.3? Because in the former case the numbers look nicer and are easier to remember (seriously: this was the winning argument in the mid 1980s). That’s what I call a proper justification. Every software could check whether CIlo < ALlo and CIhi > ALhi in full precision. Phoenix/WinNonlin in its standard format reports the CI to four decimals and gives a verbatim statement (based on the full precision comparison) in the next line. Looks like this:
Canadians regulators have a special love affair with nice numbers and set the upper limit for NTIDs instead of 111.11 to 112.0% (one decimal)… Should we do triple rounding in ABEL? OK, I have learned that k (derived from ln(1.25)/√ln(0.32+1 = 0.760128298…) should be treated as 0.76 (two decimals). For CVWR 50% the scaled limits are 69.84–143.19% (rounding #2). Now assume different lower CLs. AL (exact) AL (rounded) AL (0.76) AL (0.76 rounded)Crap. I guess EMA wants the triple rounding (red in the 4th column). Red in the 1st column shows a disagreement with the exact comparison. ❝ … then we could try to argue accordingly that 0.054999 is not inflation. Yuck ![[image]](img/uploaded/image65.jpg) Don’t get it.  ❝ 0.05 rules. If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all – including your own – papers? Happy simulating, Hötzi — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
ElMaestro ★★★ Denmark, 2013-10-31 19:16 (3822 d 07:30 ago) @ Helmut Posting: # 11846 Views: 5,794 |
|
|
Hi Hötzi, ❝ Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 106 sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require two-stage designs to be more conservative than fixed sample designs. Why? ❝ If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all – including your own – papers? I get your point, but I see it a little differently. If the type I error rate is 0.0502 then this means that 'most likely' the true type I error rate exceeds the goal post of 0.05(000000000000 etc). But at 1e6 sims we just don't have statistical significance to refute a null hyp like Pt1e≤0.05. We could just as well increase the number of sims and discuss another new level of significance and so forth. Thus for practical purposes I would aim for 0.05 without too deep speculation into statistical significance at a given number of sims per scenario. If someone could come up with a convincing concept about clnically relevant inflation then I'd welcome it; until then 0.05(000000000000 etc) must be my own goalpost. Refutation of my own papers will not be necessary, I am sure they'll be forgotton before long anyway. And there are only two of them so I am a very small fish .— Pass or fail! ElMaestro |
|
Shuanghe ★★ Spain, 2013-10-31 11:48 (3822 d 14:57 ago) @ Helmut Posting: # 11836 Views: 5,754 |
|
|
Hi, ❝ .. is principally acceptable. ... When I saw this I knew something bad gonna happen. ❝ ... However, as the study was stopped for inequivalence, any further considerations on type I error control become obsolete. OK, that's not what I was expecting. I don't know if Elmaestro and I were reading the same thing but the hidden line I've got is that: ok, you are "lucky" that your formulation screwed so you have to stop after stage 1; if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc... How's that "... occasionally even Austrians say something meaningful. " — All the best, Shuanghe |
|
Helmut ★★★ Vienna, Austria, 2013-10-31 13:51 (3822 d 12:55 ago) @ Shuanghe Posting: # 11838 Views: 5,731 |
|
|
Hi Shuanghe, ❝ […] the hidden line I've got is that: ❝ ❝ ok, you are "lucky" that your formulation screwed so you have to stop after stage 1; if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc... This is exactly my interpretation. BTW, it was ‘Method C’ as usual. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
ElMaestro ★★★ Denmark, 2013-10-31 14:04 (3822 d 12:41 ago) @ Shuanghe Posting: # 11839 Views: 5,769 |
|
|
Hi Shuanghe, ❝ (...) if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc... ❝ How's that "... occasionally even Austrians say something meaningful. " This is exactly the practical application of type I error rates. — Pass or fail! ElMaestro |
Ing. Helmut Schütz