Mikalai ★ Belarus, 20191105 18:59 (448 d 12:24 ago) Posting: # 20750 Views: 2,270 

Dear all, 1. I am not a statistician and struggle to grasp the concept of type one error inflation in the referencescaled approach. We basically do the same things as in the usual average bioequivalence where we are able to preserve TIE at 5%, but when we expand CI, we have got TIE inflation. What is behind this inflation, philosophically and mathematically? I also bumped into this discussion where some ever argue about whether this concept even exists. https://daniellakens.blogspot.com/2016/12/whytype1errorsaremoreimportant.html I assume this is not related to multiple testing. Also, we have a statistical concept, but do we have any real proof of this concept? Does anyone know products that were initially registered and then withdrawn from the market because their initial bioequivalence had been due to the inflation TIE. 2 Maybe this is not related to SABE but twostage design, but anyway. What prevents us from using Bonferroni correction in the twostage adaptive design, instead of rather complicated other statistical approaches? Thanks in advance 
Helmut ★★★ Vienna, Austria, 20191105 22:50 (448 d 08:33 ago) @ Mikalai Posting: # 20751 Views: 2,002 

Hi Mikalai, » I am not a statistician […] So am I. » We basically do the same things as in the usual average bioequivalence where we are able to preserve TIE at 5%, [... ] No, we aren’t. In ABE we have fixed limits of the acceptance range, i.e. a prespecified Null Hypothesis. In ABEL the limits are random variables or in other words, the Null is generated ‘in face of the data’. That means that each study sets it own standards and if we have a couple of HVDPs, each of them was approved according to different rules. » What is behind this inflation, philosophically and mathematically? Maybe this presentation helps. In short: Referencescaling is based on the true population parameters (hence the Greek letters \(\theta_s,\,\mu_T,\,\mu_R,\,\sigma_{wR}\)). The true standard deviation \(\sigma_{wR}\) of the reference is unknown. We have only its estimate \(s_{wR}\) from the study. Imagine: The true withinsubject CV of the reference is 27%. Hence, it is not an HVD(P) and we should use the conventional limits of 80.00125.00%. However, by chance in our study we get an estimate of 35% and we expand the limits. Since the PE and the 90% are not affected it means that the chance of passing BE increases. The chance to falsely not accepting the Null increases and this is the inflated type I error. » I also bumped into this discussion where some ever argue about whether this concept even exists. https://daniellakens.blogspot.com/2016/12/whytype1errorsaremoreimportant.html I assume this is not related to multiple testing. Nice one. Your assumption is correct. » Also we have a statistical concept, but do we have any real proof of this concept? Does any» W know products that were initially registered and then withdrawn from the market because their initial bioequivalence had been due to the inflation TIE? No (twice). But these questions diserve a detailed discussion. More when I’ll be back from Athens. — Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
Mikalai ★ Belarus, 20191106 15:13 (447 d 16:10 ago) @ Helmut Posting: # 20756 Views: 1,918 

» Maybe this presentation helps. In short: Referencescaling is based on the true population parameters (hence the Greek letters \(\theta_s,\,\mu_T,\,\mu_R,\,\sigma_{wR}\)). The true standard deviation \(\sigma_{wR}\) of the reference is unknown. We have only its estimate \(s_{wR}\) from the study. Imagine: The true withinsubject CV of the reference is 27%. Hence, it is not an HVD(P) and we should use the conventional limits of 80.00125.00%. However, by chance in our study we get an estimate of 35% and we expand the limits. Since the PE and the 90% are not affected it means that the chance of passing BE increases. The chance to falsely not accepting the Null increases and this is the inflated type I error. Dear Helmut, I may be wrong but I cannot see how we can get a true withinsubject CV of any drug. I may be wrong, but even with simulations (I suppose with simulations some assumptions regarding variance should be made), it is very difficult or even impossible. Usually, we have very scarce data on withinsubject CVs. How can we control in this situation TIE and what regulators say on this subject? I do not remember any reflection on this matter in official documents (EMA, FDA)? Regards, Mikalai 
Helmut ★★★ Vienna, Austria, 20191108 14:52 (445 d 16:31 ago) @ Mikalai Posting: # 20766 Views: 1,873 

Hi Mikalai, » I may be wrong but I cannot see how we can get a true withinsubject CV of any drug. Of course, you are right. The true CV_{wR} is unknown. But referencescaling should be done for true HVD(P)s, i.e., where the population’s CV_{wR} >30%. However, \(s_{wR}\) is the best unbiased estimate of \(\sigma_{wR}\). The former is used in the expansion formula, i.e., treating \(s_{wR}\) as a true value. » I may be wrong, but even with simulations (I suppose with simulations some assumptions regarding variance should be made), it is very difficult or even impossible. There are essentially two options:
Still: The expansion is based on the observed CV_{wR}. We once had the crazy idea of using a very conservative (99.9%) CI instead. Doesn’t work because then we would practically never be allowed to scale… » […] what regulators say on this subject? Nothing. I raised this issue at numerous conferences. Dead silence. Armin Koch (coauthor of one of the papers^{3} noting the inflated TIE) is a member of the EMA’s Biostatistical Working Party. Sent him an email in 2016. No answer. » I do not remember any reflection on this matter in official documents (EMA, FDA)? EMA = zero. At the 2^{nd} GBHI conference (Sep 2016, Rockville) László Endrenyi gave a presentation “Features, Constraints, and Extensions of the Scaling Approach” where he showed examples of the TIE, both for the EMA’s and the FDA’s approaches. Donald Schuirmann said “There is a recent paper in Pharm Res. showing how to deal with the inflation of the type I error. This is an excellent and applicable approach.” and told me in a coffeebreak “… if this is correct, we have to modify our method”. Didn’t happen. Will ask him again next month at the 4th GBHI in Bethesda.
— Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
Helmut ★★★ Vienna, Austria, 20191110 11:33 (443 d 19:51 ago) @ Mikalai Posting: # 20780 Views: 1,744 

Hi Mikalai, » […] what regulators say on this subject? I do not remember any reflection on this matter in official documents (EMA, FDA)? Some slides of Terry Hyslop (Director, Division of Biostatistics) of his presentation “Bioequivalence (BE) for Highly Variable Drugs” at the AAPS Workshop (New Orleans, Nov 2010) – after the progesterone guidance was published…
Check:
Hence, the FDA was well aware of the inflated type I error and decided to ignore it. — Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
PharmCat ★ Russia, 20191105 23:06 (448 d 08:18 ago) @ Mikalai Posting: # 20752 Views: 1,972 

» Dear all, » Thanks in advance Dear Mikalai, Bonferroni correction performed when two o more independent tests done. And this correction is very rude (More delicate is Sidak correction). But in case of adaptive design we have one test, and then another with part of same data. We don't have independent comparation and we should spend our alpha: one part in first test, another at second. And we can spend any proportion of alpha as we wish, but overal alpha should not be greater, for example 0.05. We should use an application of alphaspending function. Pocock boundary, Haybittle–Peto boundary, O'Brien–Fleming boundary  there are many approaches to work with interim analysis. Range of CI itself don't influence on TIE, it is only convention. But when CI dynamically changing I think there is no good definition for TIE. For fixed CI TIE means that real GMR may be outside permissible range with this chance. Very touching assumption to consider that TIE for RSABE is a chance when GMR outside 0.81.25 and with this comprehension make CI range wider. Really in this situation TIE not the same as in fixed case. But people want to make ABE for highvariable drug and try to do this it's like attempt to trick statistics... But I could be wrong... 
Helmut ★★★ Vienna, Austria, 20191108 15:21 (445 d 16:03 ago) @ PharmCat Posting: # 20767 Views: 1,850 

Hi PharmCat, » But when CI dynamically changing I think there is no good definition for TIE. I guess you mean that the acceptance range changes (depending on the \(s_{wR}\)). The CI is not affected. » For fixed CI TIE means that real GMR may be outside permissible range with this chance. Yep. For fixed limits the TIE is defined based on the Null of bioinquivalence. Directly accessible as the power for GMR exactly at one of the limits.
You can plug in any CV, n, design and the TIE will never exceed nominal α. » Very touching assumption to consider that TIE for RSABE is a chance when GMR outside 0.81.25 and with this comprehension make CI range wider. Really in this situation TIE not the same as in fixed case. Here the trouble starts (see what I wrote above). What we are doing here is actually HARKing (Hypothesizing After the Results are Known). Not exactly but we definitely generate the Null from the data. Apart from the TIEissues every product approved by RSABE/ABEL followed its own rules. From a consumer’s perspective this is not fortunate. » But people want to make ABE for highvariable drug and try to do this it's like attempt to trick statistics... Not sure what you mean here. Can you elaborate? — Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
PharmCat ★ Russia, 20191108 18:15 (445 d 13:08 ago) @ Helmut Posting: # 20769 Views: 1,854 

Hello! Sorry for my bad english! » Here the trouble starts (see what I wrote above). What we are doing here is actually HARKing (Hypothesizing After the Results are Known). Not exactly but we definitely generate the Null from the data. Apart from the TIEissues every product approved by RSABE/ABEL followed its own rules. From a consumer’s perspective this is not fortunate. Yes, we generate hypothesis, but we loss any link of TIE with reality. We form hypothesis from variance estimate, but it is only estimate we don't know real variance. I can't imagine how to definite TIE in this case. » Not sure what you mean here. Can you elaborate? Ofcource. It it my description of situation I think that "HARKing" come to bioequivalce because it is very expencive to make BE with big samplesize or make theraputic equivalence and it is compromise between regulators and industry. And from my side HARKing is a bad statistics (yes: consumer’s perspective this is not fortunate), but it is discussible ... some persons recommending that HARKing not be taught by educators, encouraged by reviewers or editors, or practiced by authors 
Helmut ★★★ Vienna, Austria, 20191108 20:26 (445 d 10:57 ago) @ PharmCat Posting: # 20770 Views: 1,846 

Hi PharmCat, » Sorry for my bad english! No worries, mine is hardly better. » Yes, we generate hypothesis, but we loss any link of TIE with reality. We form hypothesis from variance estimate, but it is only estimate we don't know real variance. Well, the expansion according to the guideline(s) uses the estimate as well. Try this one:
With a true CV of 30% we are not allowed to scale but the chance of passing with ABEL is higher than with ABE. In ~50% of studies we will observe a CV of >30% (\(s_{wR} >0.294\)) and expand the limits although the drug is not highly variable in the population (\(\sigma_{wR} \leq0.294\)). The fact that more than 5% pass at each of the borders of the acceptance range is a nasty side effect. » I can't imagine how to definite TIE in this case. In analogy to ABE (where the TIE is the chance of passing at the borders of the acceptance range) all authors (with one exception^{1}) of papers dealing with RSABE/ABEL employed the borders of expanded limits. IMHO, that’s a natural choice. Davit et al.^{1} distinguished between the ‘implied limits’ and the limits of the ‘desired consumer risk model’. The FDA assessed the TIE at the border of the latter, which decreases the TIE. I believe it that the FDA desires something but in actual studies one has to follow the guidance ending up with the former…
» […] because it is very expencive to make BE with big samplesize or make theraputic equivalence and it is compromise between regulators and industry. Yep. That was the original idea of SABE – avoiding extreme sample sizes whilst preserving power. Discussions started already at the first BioInternational conference.^{2} Heck, thirty years ago! » And from my side HARKing is a bad statistics (yes: consumer’s perspective this is not fortunate), … Agree. » … but it is discussible ... I’m not sure whether HARKing is the correct term. Given, the Null is constructed post hoc although at least according to a prespecified procedure. » … some persons recommending that HARKing not be taught by educators, encouraged by reviewers or editors, or practiced by authors Agree.
— Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 