Bioequivalence and Bioavailability Forum

Main page Policy/Terms of Use Abbreviations Latest Posts

 Log-in |  Register |  Search

Back to the forum  Query: 2018-02-18 05:14 UTC (UTC+2h)
 
Helmut
Hero
Homepage
Vienna, Austria,
2013-10-30 17:48

Posting: # 11822
Views: 3,372
 

 Potvin’s TSDs in the EU [Two-Stage / GS Designs]

Dear all,

the neverending story (this time RMS Austria, AGES 2013/10):

The Applicant has preplanned for a 2-stage sequential design and such is principally acceptable. The Applicant has prespecified to follow the approach as outlined in Potvin et al. (2007), where the type I error control is based on simulations and a small inflation (of 0.2%, one-sided) was considered acceptable by the authors. However, this would mean that the trial would be allowed a larger type I error than in a more conventional design. This general consideration could in principle turn out to be critical in the case the bioequivalence criteria are met only sharply. However, as the study was stopped for inequivalence, any further considerations on type I error control become obsolete.


[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
Hero

Denmark,
2013-10-30 18:26

@ Helmut
Posting: # 11823
Views: 2,920
 

 Partial thumbs up to AGES

Hi Hötzi,

I must say I have some sympathy for this view from AGES. To me this proves that occasionally even Austrians say something meaningful... :-D
The whole idea about 0.052 as a kind of clinically relevant limit which popped up in Potvin's paper is taken out of the clean blue Canadian air. If we want to control the type I error rate at 5% then my interpretation is we should mean 5% and this should not so much be a discussion of decimals or even the 0.05036 binomial limit at one million sims. In perspective: eu regulators want the CI's to be within 80.00% to 125.00% - four or five significant digits or two decimals? If the latter is the case then we could try to argue accordingly that 0.054999 is not inflation. Yuck :-D

0.05 rules.

I could be wrong, but…


Best regards,
ElMaestro

- Bootstrapping for dissolution data is a relatively new hobby of mine.
Helmut
Hero
Homepage
Vienna, Austria,
2013-10-31 14:01

@ ElMaestro
Posting: # 11841
Views: 2,830
 

 AGES & numeric precision

Hi ElMaestro,

» […] occasionally even Austrians say something meaningful...

True. But this is a bad example.

» The whole idea about 0.052 as a kind of clinically relevant limit which popped up in Potvin's paper is taken out of the clean blue Canadian air.

Well, North-America’s air. The majority of authors are from the US. Irrelevant employers like FDA, USP…

» If we want to control the type I error rate at 5% then my interpretation is we should mean 5% and this should not so much be a discussion of decimals or even the 0.05036 binomial limit at one million sims.

Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 106 sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require two-stage designs to be more conservative than fixed sample designs. Why?

Every year my book shelf gains about 1 kg since I’m a subscriber to Biometrics. Many, many simulations of incomprehensibly complicated models. Quite often n=10,000 (sign. limit 0.0537). If a value of 0.064 pops up, in the discussion it is called ‘minimal inflation’. These methods are applied in phase III. Regulators don’t care. Slowly I get tired about this double standard. Consider going for a hike to Similaun.

» eu regulators want the CI's to be within 80.00% to 125.00% - four or five significant digits or two decimals? If the latter is the case…

Seems so. IMHO this entire rounding business is bullshit. According to the (admittedly arbitrary) definition we accept a [sic] of 20% as clinically not relevant. After long discussions about the distribution / transformation the limits were set to ALlo = 1- = 0.8 and ALhi = (1-)-1 = 1.25 precisely. Why not ALhi = 1+ = 1.20 and ALlo = (1+)-1 = 83.3? Because in the former case the numbers look nicer and are easier to remember (seriously: this was the winning argument in the mid 1980s). That’s what I call a proper justification. Every software could check whether CIlo < ALlo and CIhi > ALhi in full precision. Phoenix/WinNonlin in its standard format reports the CI to four decimals and gives a verbatim statement (based on the full precision comparison) in the next line. Looks like this:

CI  90% = (   79.9950,  114.1593)
Failed to show average bioequivalence for confidence=90.00 and percent=20.0.

Hurrah!
Canadians regulators have a special love affair with nice numbers and set the upper limit for NTIDs instead of 111.11 to 112.0% (one decimal)…
Should we do triple rounding in ABEL? OK, I have learned that k (derived from ln(1.25)/√ln(0.32+1 = 0.760128298…) should be treated as 0.76 (two decimals). For CVWR 50% the scaled limits are 69.84–143.19% (rounding #2). Now assume different lower CLs.
                      AL (exact)  AL (rounded)  AL (0.76)  AL (0.76 rounded)
                      69.83255%      69.83%     69.83678%       69.84%
69.83254% ≥ AL (“BE”)   FALSE         TRUE        FALSE          FALSE
↓ to 69.83%             FALSE         TRUE        FALSE          FALSE
69.83255%               TRUE          TRUE        FALSE          FALSE
↓ to 69.83%             FALSE         TRUE        FALSE          FALSE
69.83677%               TRUE          TRUE        FALSE          FALSE
↑ to 69.84%             TRUE          TRUE        TRUE           TRUE
69.83679%               TRUE          TRUE        TRUE           FALSE
↑ to 69.84%             TRUE          TRUE        TRUE           TRUE

Crap. I guess EMA wants the triple rounding (red in the 4th column). Red in the 1st column shows a disagreement with the exact comparison.

» … then we could try to argue accordingly that 0.054999 is not inflation. Yuck :-D

[image]Don’t get it. :confused:

» 0.05 rules.

If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all – including your own – papers?

Happy simulating,
Hötzi

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
Hero

Denmark,
2013-10-31 17:16

@ Helmut
Posting: # 11846
Views: 2,796
 

 AGES & numeric precision

Hi Hötzi,

» Nope. Everything ≤0.05036 can be pure chance (at the 5% level). If you don’t accept such values in 106 sim’s you would have to adjust α further down in such a way that the “true” risk I is expected to be <0.05 (~0.04964). In other words, you would require two-stage designs to be more conservative than fixed sample designs. Why?
(...)
» If so, not even ‘Method B’ (maximum inflation 0.0504 would be acceptable. Are you suggesting to throw away all including your own – papers?

I get your point, but I see it a little differently. If the type I error rate is 0.0502 then this means that 'most likely' the true type I error rate exceeds the goal post of 0.05(000000000000 etc). But at 1e6 sims we just don't have statistical significance to refute a null hyp like Pt1e≤0.05. We could just as well increase the number of sims and discuss another new level of significance and so forth. Thus for practical purposes I would aim for 0.05 without too deep speculation into statistical significance at a given number of sims per scenario. If someone could come up with a convincing concept about clnically relevant inflation then I'd welcome it; until then 0.05(000000000000 etc) must be my own goalpost. Refutation of my own papers will not be necessary, I am sure they'll be forgotton before long anyway. And there are only two of them so I am a very small fish :-D.

I could be wrong, but…


Best regards,
ElMaestro

- Bootstrapping for dissolution data is a relatively new hobby of mine.
Shuanghe
Regular

Spain,
2013-10-31 09:48

@ Helmut
Posting: # 11836
Views: 2,788
 

 Potvin’s TSDs in the EU

Hi,

» .. is principally acceptable. ...

When I saw this I knew something bad gonna happen. :-D

» ... However, as the study was stopped for inequivalence, any further considerations on type I error control become obsolete.

OK, that's not what I was expecting.

I don't know if Elmaestro and I were reading the same thing but the hidden line I've got is that:

ok, you are "lucky" that your formulation screwed so you have to stop after stage 1; if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc...

How's that "... occasionally even Austrians say something meaningful. " :-D

All the best,
Shuanghe
Helmut
Hero
Homepage
Vienna, Austria,
2013-10-31 11:51

@ Shuanghe
Posting: # 11838
Views: 2,793
 

 Exegesis

Hi Shuanghe,

» […] the hidden line I've got is that:
»
» ok, you are "lucky" that your formulation screwed so you have to stop after stage 1; if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc...

This is exactly my interpretation. BTW, it was ‘Method C’ as usual.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
Hero

Denmark,
2013-10-31 12:04

@ Shuanghe
Posting: # 11839
Views: 2,777
 

 Potvin’s TSDs in the EU

Hi Shuanghe,

» (...) if you go to stage 2 and the result "appears" OK I'm gonna ask about the alpha inflation, higher patient risk, etc etc...

» How's that "... occasionally even Austrians say something meaningful. " :-D

This is exactly the practical application of type I error rates.

I could be wrong, but…


Best regards,
ElMaestro

- Bootstrapping for dissolution data is a relatively new hobby of mine.
Back to the forum Activity
 Thread view
Bioequivalence and Bioavailability Forum | Admin contact
17,916 Posts in 3,813 Threads, 1,117 registered users;
29 users online (0 registered, 29 guests).

The purpose of models is not to fit the data,
but to sharpen the questions.    Samuel Karlin

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
XHTML/CSS RSS Feed