Elena777 ☆ Belarus, 2020-01-28 08:02 (1715 d 04:15 ago) Posting: # 21090 Views: 25,051 |
|
Dear All, Is it OK to use the following approach of statistical evaluation in a BE study with full replicate design:
Edit: Category changed; see also this post #1. [Helmut] |
Helmut ★★★ Vienna, Austria, 2020-01-29 16:38 (1713 d 19:39 ago) @ Elena777 Posting: # 21095 Views: 22,350 |
|
Hi Elena, you are modifying the common decision scheme. Why? Since you are making essentially the same decisions though in a different order, you gain nothing. ❝ And another question: is this obligatory to include the detailed description of null and alternative hypotheses for Cmax (ABEL) and AUCt (ABE) together with description of conditions that lead to acceptance/rejection of a hypothesis in a protocol of a full replicate study? Well, that’s the problem with ABEL (and the FDA’s RSABE as well). There is no Null Hypothesis. Before you assess the data you have no clue which path(s) of the framework you will follow. You can – and should – only describe the procedure. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Elena777 ☆ Belarus, 2020-01-29 21:01 (1713 d 15:16 ago) @ Helmut Posting: # 21098 Views: 22,275 |
|
Dear Helmut, Let me describe the situation in detail. We recently conducted a replicative BS (EMA approach). We submitted the study to the Belarusian NCA and received comments on statistics. In the protocol, we stated that first we would try ABE, and if we failed then would go to scaling. We passed the BE criteria within 80-125% range for both Cmax and AUC (ABE approach). The expert demanded to calculate RR that we did not do but can do, but also to control TIE that we think is nonsense. As a result, I have a couple of additional questions.
I am not a statistician Best regards |
Helmut ★★★ Vienna, Austria, 2020-01-30 13:07 (1712 d 23:10 ago) @ Elena777 Posting: # 21101 Views: 22,345 |
|
Hi Elena, ❝ In the protocol, we stated that first we would try ABE, and if we failed then would go to scaling. So far, so good (though usually you would check for a CVwR >30% first). ❝ We passed the BE criteria within 80-125% range for both Cmax and AUC (ABE approach). OK. ❝ The expert demanded to calculate RR that we did not do but can do, … Likely he/she is interested whether the reference is a HVD(P). ❝ … but also to control TIE that we think is nonsense. Tricky. In your original post you stated also a decision tree. Any of these decisions made can be wrong, which will inflate the TIE. Imagine you failed ABE not because the reference is highly variable but by lacking power. Then you would continue to #2. The observed CVwR was >30% by chance and you expand the limits. However, the true CVwR (in the patient population) is ≤30%, and the decision wrong – inflated TIE (see there). More in the answer to #4. ❝ As a result, I have a couple of additional questions. ❝ ❝ 1. Is calculation RR for the reference drug mandatory … Quoting the EMA’s GL: For the acceptance interval to be widened the bioequivalence study must be of a replicate design where it has been demonstrated that the within-subject variability for Cmax of the reference compound in the study is >30%. Similar wording in other jurisdictions (ASEAN States, Australia, Canada, East African Community, Egypt, Eurasian Economic Union, New Zealand, Russian Federation, the WHO).❝ … and used somewhere else, except to widen the bioequivalence interval for Cmax (reference to my first questions)?
❝ 2. Is TIE nominal and cannot be controlled if we do not widen the bioequivalence interval in replicative studies and use ABEL? ❝ 3. Could you provide us with formula to calculate CI for fully replicative studies in math not computer format? It’s the same one like in ABE. ❝ 4. How mathematically (in formula) can the inflation of TIE take place in RBS if we nominally control t-parameter (choose it from tables, for example)? RBS = replicated biostudy? The problem with decision trees is that control of the TIE cannot be shown analytically (by a formula) – we need simulations. My gut feeling tells me that in your case (passing ABE in step #1) there should be no problem. But who knows? I try not to think with my gut. However, you could check the TIE for the conventional framework in PowerTOST . You need CVwT, CVwR, and the number of subjects / sequence. Example (one million simulated studies by default):
❝ 5. What role of fixed or random effects in RBS or ordinary BS? That’s almost a philosophical question. In the strict sense:
I performed large simulations and seemingly the EMA’s ‘Method A’ is slightly more conservative than ‘Method B’. However, both methods assume homoscedasticity (identical within-subject variabilities of T and R). A rather strong assumption which was demonstrated to be false is numerous studies… The FDA’s method (termed ‘Method C’ by the EMA) – which is a full-blown mixed-effects model – is generally more conservative (wider confidence interval). But that’s another story. ❝ I am not a statistician I tried to answer in a not too statistical way. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-30 14:08 (1712 d 22:10 ago) (edited on 2020-01-30 15:08) @ Helmut Posting: # 21102 Views: 22,277 |
|
Dear Helmut, ❝ Tricky. In your original post you stated also a decision tree. Any of these decisions made can be wrong, which will inflate the TIE. Imagine you failed ABE not because the reference is highly variable but by lacking power. Then you would continue to #2. The observed CVwR was >30% by chance and you expand the limits. However, the true CVwR (in the patient population) is ≤30%, and the decision wrong – inflated TIE (see there). More in the answer to #4. It is a bit confusing, but could you clarify a couple of points.
If I am correct, in my view it is a preferable solution to minimize the inflation TIE. Best regards |
Helmut ★★★ Vienna, Austria, 2020-01-30 16:09 (1712 d 20:08 ago) @ Mikalai Posting: # 21104 Views: 22,285 |
|
Hi Mikalai, ❝ ❝ […] see there ❝ ❝ 1. In the case of going by ABEL path (see the decision tree), I assume we do not use any population parameters and just compare concentration from samples. Nope. Scaled ABE (in both of its flavors ABEL and RSABE) is formulated in the unknown population parameters (see the linked slide above). For ABEL we have
❝ There is no inflation TIE? Is it correct? No it isn’t. On the contrary, the inflated TIE arises from a misspecification of CVwR. We think that the drug/drug product is highly variable, expand the limits and pass. But the true CVwR is <30%, i.e., not a HVD(P) and we would have failed ABE. That’s it. ❝ And we do not use RR for the ref drug in our calculation. Is it correct? If you mean by ‘we’ Elena’s approach, and stop since you showed ABE, no. ❝ 2. But if the study fails, we then calculate RR and a new bioequivalence interval if CV is higher than 30%. That is when we use RR and where TIE can be inflated. Is it correct? Yes. ❝ But we do not recalculate our new CI we just use that from ABEL to compare with a new or the old CI. In the latter case, we basically fail the study. Is it correct? I try to repeat Elena’s procedure. You make a lot of decisions (any of them can be wrong):
❝ The identical decision tree was used in the BS of rasagiline (registration required) and was accepted by EMA. https://clinicaldata.ema.europa.eu/web/cdp/home This site is a pain in the back (refuses my credentials, send as a reminder the user name I just typed in, etc.). If this procedure was really accepted, that’s even worse than the ‘usual’ inflation of up to ~0.09… Oh dear! Maybe I’ll set up simulations, time allowing. I expect the worst. ❝ If I am correct, in my view it is a preferable solution to minimize the inflation TIE. Sorry, not at all. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-30 17:02 (1712 d 19:16 ago) @ Helmut Posting: # 21106 Views: 22,150 |
|
Dear Helmut, I would like to apologize because I confused you in my previous post. I should have used ABE instead of ABEL as an abbreviation because I meant the average bioequivalence. I corrected it recently. Thus, we do not use RR for the ref drug in our calculations and should not control TIE if we initially go by the ABE approach and reach bioequivalence within 125-80% interval (alpha at 0.05). Also, it seems the only reason that we should calculate RR for the ref drug first is to calculate and control TIE latter if the CV is higher than 30%. If it turns out that it is lower than 30% then we simply use the ABE and no control TIE. However, if we do not plan to do this (control TIE), then it is not important to calculate the RR ref first. As you mentioned there Post and there post no regulatory agency issued a formal guideline on TIE in replicate studies, and FDA and EMA do not require and accept replicate studies without controlling TIE. In this case (no control TIE), we can calculate the RR only when we fail in Cmax. We do not recalculate our new CI we just use that from ABE to compare with a new or the standard CI if CV less than 30. In the latter case, we basically fail the study. I would not like to discuss whether we should or not control TIE in replicate studies in this branch. In my view, probably we should not, and it is also probably that leading agencies won't do anything with this issue unless a safety signal occurs. |
Helmut ★★★ Vienna, Austria, 2020-01-30 17:59 (1712 d 18:19 ago) @ Mikalai Posting: # 21108 Views: 22,207 |
|
HI Mikalai, ❝ […] we do not use RR for the ref drug in our calculations and should not control TIE if we initially go by the ABE approach and reach bioequivalence within 125-80% interval (alpha at 0.05). Read my last post again. And again. You do not know beforehand whether or not you will pass ABE. If not, you continue with the calculation CVwR and ABEL. If that happens, you will evaluate the data twice. That’s a textbook-example of multiplicity. Just by that the TIE will be \(1-(1-\alpha)^k=1-(1-0.05)^2=0.0975\). That’s not rocket science and already higher than with the usual framework, where we calculate the 90% CI only once. Add the potential bias of CVwR and you are in deep shit. Excuse my French. ❝ Also, it seems the only reason that we should calculate RR for the ref drug first is to calculate and control TIE latter if the CV is higher than 30%. No. It prevents us from potentially assessing the CI twice. See above. ❝ If it turns out that it is lower than 30% then we simply use the ABE and no control TIE. However, if we do not plan to do this (control TIE), then it is not important to calculate the RR ref first. See above. If you continue with this appraoch (IMHO, a bad idea), please use at least a 95% CI. ❝ As you mentioned there Post and there post no regulatory agency issued a formal guideline on TIE in replicate studies, and FDA and EMA do not require and accept replicate studies without controlling TIE. Unfortunately regulations ≠ science. In bioequivalence we must not forget the only important – the patient! He/she is a living person, not just α 0.05. Dirk Marteen Barends IMHO, it is a cowardly excuse to rely on an approach ‘because a guideline says so’ and the agencies accepts studies (quote of a European assessor: I don’t have enough time to read all those papers…). There are almost twenty published showing an increased patient’s risk by reference-scaling and not a single one (‼) showing the opposite. Even with the FDA’s ‘desired consumer risk model’ – called by some hocus pocus – there is a substantial inflation of the TIE. Your approach is worse. ❝ In this case (no control TIE), we can calculate the RR only when we fail in Cmax. We do not recalculate our new CI we just use that from ABE to compare with a new or the standard CI if CV less than 30. In the latter case, we basically fail the study. See above. ❝ I would not like to discuss whether we should or not control TIE in replicate studies in this branch. In my view, probably we should not, … Well, that’s like my niece being afraid of monsters in the nursery. Her solution: Close the eyes and cover the ears with her hands. ❝ … and it is also probably that leading agencies won't do anything with this issue unless a safety signal occurs. Forget pharmacovigilance. It is terribly insensitive. There are some HVD(P)s where the concentrations vary tenfold day-to-day. Seriously. Yep, even the maximum expansion is too conservative. Furthermore, we are not worried about drugs / drug products with a true CVwR of ~45% and higher. The troublesome are the borderline cases. PS: Waiting for a guideline takes quite some perseverance. Grizzle’s concept of testing for a sequence effect (unequal carry-over) of 1965 was proven false by Freeman in 1989. Took another 21 years to make it to the EMA’s GL. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-30 19:39 (1712 d 16:38 ago) @ Helmut Posting: # 21110 Views: 22,100 |
|
Dear Helmut, First of all, let's clarify a bit further. Why should we calculate CI twice if we do not plan control TIE. ❝ Read my last post again. And again. You do not know beforehand whether or not you will pass ABE. If not, you continue with the calculation CVwR and ABEL. If that happens, you will evaluate the data twice. That’s a textbook-example of multiplicity. Just by that the TIE will be \(1-(1-\alpha)^k=1-(1-0.05)^2=0.0975\). That’s not rocket science and already higher than with the usual framework, where we calculate the 90% CI only once. Add the the potential bias of CVwR and you are in deep shit. Excuse my French. As I understand the formula of CI for ABE and RABE/ABEL is the same (2 sequence 4-period study). Or I am wrong? We calculate ABE CI. If Cmax is out of range, we than calculate the RR for ref drug and recalculate a new bioequivalence interval if the CV is higher than 30. And then we compare whether our old ABE CI is within the new interval. In this case, we calculate CI once. Am I wrong? In terms of the decision tree, this approach is used. It is not unique. Multiple studies were accepted in Europe. It seems that there have been no complaints from European regulators. I would not like to discuss whether we should or not control TIE in ABEL/RABE in this branch, because it may divert from the main topic. But I can explain my position. It seems that to control TIE we may have to change the alpha after getting results from the study sample. It potentially can reduce power and the only way to balance this is to recruit more subjects. The question is how many? One should take into consideration that ABEL/RABE trials are more challenging: high dropout rates (longer than usual BE trials) and riskier (more AE because of more blood and more drugs). If we can find a solution when we can increase the sample size slightly, then it is OK; otherwise, we may approach the sample size of a 2-period trial that basically invalidates the whole concept. There is also another point. In Europe, we cannot expand limits for AUC and can for Cmax when it is safe. Why cannot we tolerate in this situation TIE that is higher than 5% (why 5%?), let's say 10%? Nothing is 'carved in the stone'. It appears that no drugs were withdrawn because of inflated TIE yet. Also whether we have reliable methods to control TIE statistically on which regulators agree? Finally, I disagree that pharmacovigilance is senseless. If it is done properly, it should and can pick up bad players (drugs and companies). Best regards |
Mikalai ★ Belarus, 2020-01-30 20:31 (1712 d 15:47 ago) @ Mikalai Posting: # 21113 Views: 22,159 |
|
I would like to include this as a quote. ❝ No. It prevents us from potentially assessing the CI twice. See above. Best regards |
Helmut ★★★ Vienna, Austria, 2020-01-30 21:07 (1712 d 15:10 ago) @ Mikalai Posting: # 21114 Views: 22,355 |
|
Hi Mikalai, ❝ […] Why should we calculate CI twice … You don’t calculate it twice, you possibly assess it twice. ❝ … if we do not plan control TIE. Cause you care about the patients? If you don’t give a shit about other people consider changing jobs and become a stock broker. ❝ As I understand the formula of CI for ABE and RABE/ABEL is the same (2 sequence 4-period study). Or I am wrong? We calculate ABE CI. If Cmax is out of range, we than calculate the RR for ref drug and recalculate a new bioequivalence interval if the CV is higher than 30. And then we compare whether our old ABE CI is within the new interval. In this case, we calculate CI once. Am I wrong? See above. You calculate the CI once. If you fail ABE (first assessment with a nominal α 0.05), you assess it for ABEL (second assessment with a nominal α 0.05). Is it clear now? Two tests, each performed at level 0.05. Inflated Type I Error. Full stop. ❝ In terms of the decision tree, this approach is used. It is not unique. Multiple studies were accepted in Europe. It seems that there have been no complaints from European regulators. Awful, just awful. ❝ […] It seems that to control TIE we may have to change the alpha after getting results from the study sample. It potentially can reduce power and the only way to balance this is to recruit more subjects. The question is how many? Easy to answer. ❝ One should take into consideration that ABEL/RABE trials are more challenging: high dropout rates (longer than usual BE trials) … Opt for one of the three period full replicates. Loss of power due to dropouts is overrated. ❝ … and riskier (more AE because of more blood and more drugs). Given. ❝ If we can find a solution when we may increase the sample size slightly, then it is OK; See the presentation linked above. Peanuts. ❝ … otherwise, we may approach the sample size of a 2-period trial that basically invalidates the whole concept. For your approach you possibly have to adjust α below Bonferroni’s 0.025. library(PowerTOST) ❝ There is also another point. In Europe, we cannot expand limits for AUC and can for Cmax when it is safe. Why cannot we tolerate in this situation TIE that is higher than 5% (why 5%?), let's say 10%? Nothing is 'carved in the stone'. Wow! Even for a hardened atheist that smells of heresy. ❝ It appears that no drugs were withdrawn because of inflated TIE yet. Duno. ❝ Also whether we have reliable methods to control TIE statistically on which regulators agree? As long as you use the same framework / method stated in the GL no regulator on this planet has any reason not to accept an approach using a lower α. ❝ Finally, I disagree that pharmacovigilance is senseless. If it is done properly, it should and can pick up bad players (drugs and companies). I don’t believe it. The only example I’m aware of was the formulation change of levothyroxine in France (passed ABE with narrower limits and the AEs went through the ceiling). — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-31 12:14 (1712 d 00:03 ago) @ Helmut Posting: # 21115 Views: 21,996 |
|
Dear Helmut, You seem to strengthen my doubts. ❝ See above. You calculate the CI once. If you fail ABE (first assessment with a nominal α 0.05), you assess it for ABEL (second assessment with a nominal α 0.05). Is it clear now? Two tests, each performed at level 0.05. Inflated Type I Error. Full stop. It has no practical sense for me. As I know trials frequently recalculated with some modifications, usually slight ones. I saw that the FDA did this, for example. I do not see that alpha levels were reduced in those recalculations. If we employ this approach (no recalculations or lower alpha level) we may paralyze the whole industry. There would be massive wastes of resources - a lot of studies would be rejected - and huge risks for patients - depriving, for example, them live-saving medications because trials should be repeated in some cases. Actually, we can ask people from the forum how many sponsors or CROs they know whose trials have never been recalculated? ❝ ❝ ❝ ❝ ❝ It seems that we have to multiple the ABE column by 2 to get full the sample size but not the ABEL column; otherwise, it has no much sense for me. I also wonder if the dropout rate has been considered in the calculations. Looking at your table and slides post, it is appears that in the region CV between 30 - 40% the sample size for ABEL and ABE trials may be very close to each other. If it is true, the ABEL trials (CV between 30-40) should not be allowed by ethic and regulatory bodies because of unnecessary risks for subjects. It invalidates the RABE/ABEL approach in this region. ❝ ❝ Finally, I disagree that pharmacovigilance is senseless. If it is done properly, it should and can pick up bad players (drugs and companies). ❝ ❝ I don’t believe it. ❝ The only example I’m aware of was the formulation change of levothyroxine in France (passed ABE with narrower limits and the AEs went through the ceiling). Actually, it works and even better the BE trials in some cases. Scotland is a good example. In 2015, it had a system, the GP system, that was well developed to spot safety signals. At that time it was not fully developed and tuned. It has some issues like privacy, it collects a lot of private data, complexity, not all doctors know how to operate the system to get full benefits. Also, it was not connected to the pharmacy and hospital systems yet. Best regards |
Helmut ★★★ Vienna, Austria, 2020-01-31 15:00 (1711 d 21:17 ago) @ Mikalai Posting: # 21119 Views: 21,908 |
|
Hi Mikalai, ❝ You seem to strengthen my doubts. Fine. ❝ ❝ If you fail ABE (first assessment with a nominal α 0.05), you assess it for ABEL (second assessment with a nominal α 0.05). […] Two tests, each performed at level 0.05. Inflated Type I Error. Full stop. ❝ It has no practical sense for me. As I know trials frequently recalculated with some modifications, usually slight ones. I saw that the FDA did this, for example. I do not see that alpha levels were reduced in those recalculations. Not limited to the FDA. We (CROs, applicants) could never, ever recalculate a study because that would be judged by assessors as cherry-picking. Of course, regulators play not in another league but in another sport, which can be expressed as Quod licet Iovi, non licet bovi The problem with a recalculation is – since the entire α was already spent in the original analysis – the TIE → ∞. Now what? If a passing study fails now, no problem; the risk is only a theoretical one since the product will not be marketed based on this study. Question: What is the TIE if the recalculated study passes as well?❝ If we employ this approach (no recalculations or lower alpha level) … Once you started with an α which controls the TIE in the original analysis, any lower α in the recalculation cannot work. OK, you could submit all studies with a 92% CI (α 0.04) and – if a recalculation is requested – perform it with a 98% CI (α 0.01). Power drops through the floor, good luck. ❝ … we may paralyze the whole industry … Agree, also with your examples. ❝ Actually, we can ask people from the forum how many sponsors or CROs they know whose trials have never been recalculated? Good idea. Only very few of mine. ❝ It seems that we have to multiple the ABE column by 2 to get full the sample size but not the ABEL column; otherwise, it has no much sense for me. Duno what you mean. Are you considering the sample size of a 2×2×2 crossover? The sample size of the 2-period 4-sequence full replicate is ~½ though the number of treatments / biosamples (driving the study cost) are essentially the same. CV (%) ABE.2x2x2 ABE.2x2x4 n.ABEL n.ABEL.Bonf n.ABEL.adj n.Molins ❝ I also wonder if the dropout rate has been considered in the calculations. No. That’s specific to the drug and I never “compensate for potential dropouts in order to maintain power” unless I expect a dropout rate of >15%. Waste of money because the impact of dropouts on power is generally small. Try the functions pa.ABE() and pa.scABE() of PowerTOST .❝ Looking at your table and slides post, it is appears that in the region CV between 30 - 40% the sample size for ABEL and ABE trials may be very close to each other. The plots in the slide are all for ABEL. A comparison of ABE and the EMA’s (unadjusted) ABEL: At 30% sample sizes are 40 and 34, at 40% 68 and 30. ❝ If it is true, the ABEL trials (CV between 30-40) should not be allowed by ethic and regulatory bodies because of unnecessary risks for subjects. It invalidates the RABE/ABEL approach in this region. Hhm, not sure what you mean. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-31 17:07 (1711 d 19:11 ago) @ Helmut Posting: # 21121 Views: 21,805 |
|
Dear Helmut ❝ Duno what you mean. Are you considering the sample size of a 2×2×2 crossover? The sample size of the 2-period 4-sequence full replicate is ~½ though the number of treatments / biosamples (driving the study cost) are essentially the same. ❝ ❝ ❝ ❝ ❝ ❝ Try the functions ❝ The plots in the slide are all for ABEL. A comparison of ABE and the EMA’s (unadjusted) ABEL: ❝ ❝ ❝ ❝ At 30% sample sizes are 40 and 34, at 40% 68 and 30. ❝ ❝ Hhm, not sure what you mean. I just would like to say that it has no sense to conduct ABEL/RABE trials if the CV is assumed around 30% because, in order to control TIE, we have to increase the sample size to around that of usual ABE trials. It violates GCP principles in this case and invalidates the ABEL/RABE approach (subjects are subjected to unnecessary risks in the ABEL/RABE trials). And it very complicates the matter. Basically, when should we consider an ABEL/RABE trial? Best regards It also seems that when we wish to control TIE the sample size will be around 50 (worst scenario + dropout rate) in almost every case (usually little known about reference withindividual variability) of the ABEL/RABE. But I have rarely seen such trials. Regards Edit: Merged with a later (now deleted) post. You can edit your posts for 24 hours. [Helmut] |
Helmut ★★★ Vienna, Austria, 2020-01-31 13:19 (1711 d 22:59 ago) @ Mikalai Posting: # 21116 Views: 21,926 |
|
Hi Mikalai, ❝ The identical decision tree was used in the BS of rasagiline (registration required) and was accepted by EMA. https://clinicaldata.ema.europa.eu/web/cdp/home I succeeded at the end and you are correct. Why am I not surprised that the study was performed in India? OK, the sponsor was European. One of my clients – should have asked me before… Clinical phase ended Jun 2009, study report v1 Oct 2009, v2 Mar 2011, v3 (final) Jul 2011. What happened in those 21 (!) months – waiting for the BE GL (published Jan 2010, effective Jul 2010)? The procedure for Cmax was almost like Elena described.
However, in the next section all of the above is ignored and the EMA’s \(\theta_s \approx k=0.760\) is given for ABEL. Statistical methods “The fixed effects sequence, subject nested within sequence, period and treatment were used in the ANOVA model…” Cough: Not an ANOVA with all effects fixed but a mixed-effects model with restricted maximum likelihood estimation, no nested structure. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-31 13:40 (1711 d 22:37 ago) @ Helmut Posting: # 21117 Views: 21,920 |
|
Dear Helmut, I would like to stress out again that this decision tree has been used in multiple accepted BE studies. The tree is used not only by Indian CROs but CROs from developed countries. Cannot say more because it is a bit confidential. And again no complaints from regulators at all; otherwise, it would not be used Regards Why is it flawed? They passed bioequivalence with the first step and did not go to the second one. It may be risky according to your approach but they were lucky enough. There is no TIE inflation in their study as I understand. What is wrong in relation to TIE inflation? Best regards Edit: Merged with a later (now deleted) post. You can edit your posts for 24 hours. [Helmut] |
Helmut ★★★ Vienna, Austria, 2020-01-31 15:17 (1711 d 21:00 ago) @ Mikalai Posting: # 21120 Views: 22,155 |
|
Hi Mikalai, ❝ I would like to stress out again that this decision tree has been used in multiple accepted BE studies. The tree is used not only by Indian CROs but CROs from developed countries. […] And again no complaints from regulators at all; otherwise, it would not be used Sad, so sad! (Mr Trump) ❝ Why is it flawed? It is not even wrong. (Wolfgang Pauli) ❝ They passed bioequivalence with the first step and did not go to the second one. By luck because the CV was lower than assumed (30% instead of 32%) and there were substantially fewer dropouts than anticipated. ❝ It may be risky according to your approach but they were lucky enough. There is no TIE inflation in their study as I understand. ❝ What is wrong in relation to TIE inflation? One should never design a study relying on luck! If with this approach (assessing ABE with a 90% CI first) one proceeds to ABEL because ABE failed, the patient’s risk is compromised and there is no bloody way to control the Type I Error. If regulators don’t give a shit about the patient’s risk, we should. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-01-31 17:41 (1711 d 18:36 ago) (edited on 2020-01-31 18:26) @ Helmut Posting: # 21122 Views: 21,759 |
|
Dear Helmut, It appears quite interesting. I again disagree with you. Your approach is very questionable. The sponsor risked and won. We are discussing a specific case. The decision tree may have been questionable, but the questionable part was never implemented. Now, let's imagine that regulators reject the study. But what reason? The reason would be that potentially, in another reality, the results could have been non-equivalent or TIE would be unacceptably high. Also one should take into consideration that at present there are no guidelines or other formal documents that prohibit the usage of this decision tree or require to control TIE and calculate the RR for the ref product first. The local NCA and ethics approved the study. In other words, the study should have been rejected because of an imaginary situation that never materialized in reality. But this approach can more or less be applied to any BE study. It is non-sense and not legally defendable. The only result would be money of taxpayers paid to the sponsor for delayed registration if this would take place in the developed world. Everything that may be a reason to reject studies should explicitly be stated in regulatory documents. Best regards |
Helmut ★★★ Vienna, Austria, 2020-01-31 21:28 (1711 d 14:50 ago) @ Mikalai Posting: # 21124 Views: 22,140 |
|
Hi Mikalai, ❝ It appears quite interesting. I again disagree with you. Why am I not surprised? Seems that your line of argument is a financial one (though veiled behind words like taxpayer’s money). I’m interested in the patient’s risk – possibly more than some regulators. I asked myself whether all is lost when ABE fails. I wouldn’t bet on a lucky punch. We know that TOST is not a uniformly most powerful test (“… the TOST procedure can be quite conservative, i.e., the achieved Type I error rate may be lower than the nominal 0.05 significance level.”)1 That means for nasty combinations of PE and CV with a given sample size the actual TIE might be <0.05. This opens a back door to recalculate the CI but with α = 1 – TIEact) and assess the study for ABEL whilst keeping the overall TIE controlled. 10,000 simulated studies, PE 0.85–0.95, CV 0.25–0.60, 2×2×4 design, powered (≥80%) for ABEL: • Studies which passed ABE (90% CI, CVwR not calculated). • Studies which failed ABE, CVWR >30%, and the wider CI passed ABEL. • Studies which failed (stopped in ABE because CVwR ≤30% or in ABEL). Nasty are the close misses in ABE. There is practically no α left and the study will likely fail ABEL as well because the new CI would be extremely wide. Example: CV 52.92%, PE 0.9118, n 26. With a 90% CI of 77.52–107.25% fails ABE and the actual TIE is 0.04809 (α for ABEL 0.00191). We continue to ABEL with the maximum expansion of 69.84–143.19%. The 99.618% CI is 68.15–121.98%, which fails ABEL as well. With the common approach we have a 81.2% chance of passing and with 0.0382 the TIE is easily controlled. Now for the money-wise argument: 45.9% passed ABE and only 7.7% ABEL (i.e., overall power 53.6%); that’s hardly better than tossing a coin. I checked a couple of other studies in the meantime. In all of them calculation of CVwR was the first step. In some the CVwR was <30% and the study assessed by ABE. I found two cases where CVwR was >30% and studies passed ABEL and the 90% was within 80.00–125.00%. The report correctly stated that ABEL (with expanded limits) was demonstrated and noted that conventional ABE would have passed as well. The latter was just a remark and not part of the procedure. Given all that, I consider the approach we discussed here as crap and IMHO, it should not be used for the following reasons:
EOD from my side unless we concentrate on science rather than profits.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-02-01 17:18 (1710 d 18:59 ago) (edited on 2020-02-01 17:33) @ Helmut Posting: # 21125 Views: 21,343 |
|
Dear Helmut ❝ Seems that your line of argument is a financial one (though veiled behind words like taxpayer’s money). I’m interested in the patient’s risk – possibly more than some regulators. I am not fighting for money but for democracy, the Republic... Your approach is nothing more than a road to hell lined with good intentions. Basically you advocate rejecting trials because of things that are neither legally prohibited nor have happened but are/were (when trials done) risky from one scientific angle. I guess this practice hardly only one that is so contentious in the industry. Your approach basically gives experts the power to arbitrally decide which studies to accept and which do not. It would end in mayhem. In Belarus, we often see this, and it usually ends that we have to prove that the globe is round not flat. If TIE must be controlled in ABEL/RABE, it should be ingrained in regulatory documents. Full stop. If somebody decided to risk and won (passing bioequivalence on ABE terms), these studies should be accepted. Otherwise, you would reject valid studies and would dictate how sponsors should spend their own money and how different countries have to vet BE trials. I am very skeptical that this decision tree can be outlawed (it is diffuclt to prosecute for things that did not happen), but legal experts may have other opinions. What is about ABEL/RABE trials when the CV is around 30% or uncertain but not small? Should we pump up the sample size in this case (I guess a majority of cases) to the level close to ABE trials which basically invalidates the whole concept of ABEL/RABE? Finally, I am not a statistician but wether your approach itself also might inflate TIE when we have to calculate RR (population parameter) and then CI which we compare with a standard or widened bioequivalence interval. We do two calculations in which alpha levels are involved. Best regards |
mittyri ★★ Russia, 2020-02-01 22:34 (1710 d 13:43 ago) @ Mikalai Posting: # 21127 Views: 21,300 |
|
Hi Mikalai, ❝ I am not fighting for money but for democracy, the Republic... Republic of what? ❝ Your approach is nothing more than a road to hell lined with good intentions. I think the reason of that words is the misunderstanding of arguments given above... ❝ Basically you advocate rejecting trials because of things that are neither legally prohibited nor have happened but are/were (when trials done) risky from one scientific angle. There's no science in p value = 0.05. Or could you do me a favour and give some link where the contrary arguments exist? That's just a convention (given in GLs too). Rules of game. If you go there, please use it. ❝ Your approach basically gives experts the power to arbitrally decide which studies to accept and which do not. where is it written? just to recalculate CIs using TIE correction gives the power? Is TIE some special value from expert mind? How do you think it is assessed? ❝ In Belarus, we often see this, and it usually ends that we have to prove that the globe is round not flat. Could you please give some example? From the comments on the forum I can conlude that experts in Belarus are rather progressive. ❝ If TIE must be controlled in ABEL/RABE, it should be ingrained in regulatory documents. Full stop. It should be controlled everywhere, not only in ABEL ❝ If somebody decided to risk and won (passing bioequivalence on ABE terms), these studies should be accepted. Please define 'win'. You didn't win until null hypothesis(es) is rejected. ❝ Otherwise, you would reject valid studies and would dictate how sponsors should spend their own money and how different countries have to vet BE trials. Oh my! pity Sponsors! I propose not to spend money at all. Just go with pharmaceutical assessment, no BE study. no cent for useless trials. Is that approach reasonable enough? ❝ I am very skeptical that this decision tree can be outlawed Are the arguments regarding TIE inflation not clear? WE are living in the real world where the precedence does not rule everywhere. And the experts also err. Maybe they didn't realize here the problem of TIE inflation. Remember that the problem is rather new. ❝ What is about ABEL/RABE trials when the CV is around 30% or uncertain but not small? what are the benefits of replicate design for that CV value (30%)? ❝ Should we pump up the sample size in this case (I guess a majority of cases) to the level close to ABE trials which basically invalidates the whole concept of ABEL/RABE? Have you read Helmut's calculations for the sample sizes? ❝ Finally, I am not a statistician I think it is a good reason not to read the end of that paragraph, sorry... — Kind regards, Mittyri |
Mikalai ★ Belarus, 2020-02-06 14:41 (1705 d 21:36 ago) (edited on 2020-02-06 16:18) @ mittyri Posting: # 21153 Views: 20,123 |
|
Hi Mitturi, My remarks ❝ Republic of what? This is from the Star Wars. ❝ I think the reason of that words is the misunderstanding of arguments given above... I think there is no misunderstanding. I never said that in ABEL/RABE trials TIE should not be controlled. What I worry is the control TIE may invalidate the whole method, because in a big number studies the reduction in sample size can be negligible but risks can double for subjects. I also afraid that this can be assesd only post-hoc. Do we have a decisicion tree for ethic committees? ❝ There's no science in p value = 0.05. Or could you do me a favour and give some link where the contrary arguments exist? That's just a convention (given in GLs too). Rules of game. If you go there, please use it. It probably deserves a separate topic. Who set up rules of the game in this case? Moreover TIE as consumer protection metric is very controversial. Have we asked people about this convention? Do we have good feedback mechanisms with consumers? Do we incorporate qulitative studies in our regulatory documetns. How can we represent consumers if we do not speak with them? It seems that people tolerate much higher risks than we use currently. Look at alternative medicine. They have millions customers and don't bother with any statistics. ❝ It should be controlled everywhere, not only in ABEL It is Utopia, and good luck with this. See my comments above. But I can give another argument. At the heart of bioequivalence lie the concept that products are different. But what does this mean. Is the fact that they are manufactured by different companies is enough to make this conclusion? Have manufacturing processes of different manufactures have ever been compared. And what about TIE deflating factors like industrial statisticians, industrial spies, angry sacked employees from original producers, the same equipment and the same supppliers. Might be that the TIE inflation risk is a bit overestimated This is why if something is widely used and is a reason to rejection, it should be mentioned in regulatory documents. ❝ Please define 'win'. You didn't win until null hypothesis(es) is rejected. If somebody shot all alpha level and passed the bioequivalence at the first stage. ❝ Are the arguments regarding TIE inflation not clear? WE are living in the real world where »the precedence does not rule everywhere. And the experts also err. Maybe they didn't »realize here the problem of TIE inflation. Remember that the problem is rather new. It seems that the issue is still not clear for most regualtors.It also seems that this issue is afloat for more 7 years. Also, where are results of the inflated TIE? Where are unsafe and ineffecient drugs. ❝ what are the benefits of replicate design for that CV value (30%)? We never know our final CV. See my comments below. ❝ Have you read Helmut's calucations for the sample sizes? ❝ I think it is a good reason not to read the end of that paragraph, sorry... There are no stupid questions there are either not good answers or ignorance. Sorry ... |
mittyri ★★ Russia, 2020-02-06 17:23 (1705 d 18:54 ago) @ Mikalai Posting: # 21154 Views: 19,903 |
|
hi Mikalai, ❝ Hi Mitturi, ^^^^is my nick so difficult to copy and paste? ❝ ❝ Republic of what? ❝ This is from the Star Wars. looks like rewording, cannot recall that ❝ I think there is no misunderstanding. I never said that in ABEL/RABE trials TIE should not be controlled. What I worry is the control TIE may invalidate the whole method, because in a big number studies the reduction in sample size can be negligible but risks can double for subjects. I cannot find the arguments for doubling risks for subjects. What risks are higher? ❝ I also afraid that this can be assesd only post-hoc. really? and what about the limits? do you know them before the study? ❝ Do we have a decisicion tree for ethic committees? see Helmut's lectures for schemes ❝ Who set up rules of games in this case? Moreover TIE as consumer protection metric is very controversial. Have we asked people about this convention? Do we have good feedback mechanisms with consumers? Do we incorporate qulitative studies in our regulatory documetns. How can we represent consumers if we do not speak with them? what answer do you want to get? "Yes, we are interested in TIE control" "No, 5% is too much for us, gimme 6%" ❝ It seems that people tolerate much higher risks than we use currently. Look at alternative medicine. They have millions customers and don't bother with any statistics. some people does. Believe or not but the people trust decisions of regulators. Placebos exist on the market, is that bad? Sometimes yes, sometimes not. ❝ ❝ It should be controlled everywhere, not only in ABEL ❝ It is Utopia, and good luck with this. See my comments above. But I can give another argument. so you blame ICH docs? ❝ At the heart of bioequivalence lie the concept that products are different. you are playing with words. different but still have comparable action ❝ Have manufacturing processes of different manufactures have ever been compared. how to assess the differences in overall and how are they inflate the bioequivalence? If you have an answer - go with an initiative to EMA, they will be happy to hear it I think. ❝ And what about TIE deflating factors like industrial statisticians, industrial spies, angry sacked employees from original producers, the same equipment and the same supppliers. sorry, who is industrial statistician? if you have any proposals regarding making the products more comparable - go to the initiative group and propose your solutions. I see only remarks. I don't understand why do you want to start a wholywar here. ❝ Might be that the TIE inflation risk is a bit overestimated a bit? what will trigger you to say it is not overestimated? There will always be sceptics/conservators, irrespective of initiatives. Especially on the slippery zone of risk/benefit of generic drugs. ❝ This is why if something is widely used and is a reason to rejection, it should be mentioned in regulatory documents. EMA and FDA mention 5%. Have you read the lectures? If we want to keep that value at the level of 5%, we should keep in mind TIE inflation. If you have another opinion - stop reading that ❝ ❝ Please define 'win'. You didn't win until null hypothesis(es) is rejected. ❝ ❝ If somebody shoot all alpha level and pass bioequivalence at the first stage. aha, stage! OK, so you are playing in the 2-stage game? Did you take into account the corrected alphas there? Do you remember 2-stage designs and corrected alphas irrespective of the results of 1st stage? ❝ It seems that the issue is still not clear for most regulators. It also seems that this issue is afloat for more 7 years. Also, where are results of the inflated TIE? Where are unsafe and inefficient drugs. where are the results of inefficient drugs started when Cmax had limits 75-133? If it is bad, why did the regulators change it? why that drugs are still on the market? ❝ There are no stupid questions there are either not good answers or ignorance. Sorry ... at least I did not asked something stupid! excellent! I am not surprised that you are not happy with my answers since even detailed Helmut's answers were rejected by you. For sure you can ignore TIE inflation for yourself until some request will knock your door. It is up to you to prepare yourself or just keep trying to neglect it — Kind regards, Mittyri |
d_labes ★★★ Berlin, Germany, 2020-02-05 20:16 (1706 d 16:01 ago) @ Mikalai Posting: # 21150 Views: 20,182 |
|
Dear Mikalai ❝ ... In Belarus, we often see this, and it usually ends that we have to prove that the globe is round not flat. Read the books of Terry Prattchets "Discworld" series and you will discover that the world is a flat disc resting on four elephants which are standing on a very big tortoise . I recommend especially the titles - The Science of Discworld (1999) - The Science of Discworld II: The Globe (2002) - The Science of Discworld III: Darwin's Watch (2005) - The Science of Discworld IV: Judgement Day (2013) They are full of wisdom and may furnish you to be reluctant in fighting for some ones alleged "truth" in science, may it be the own "truth" or the truth of others. — Regards, Detlew |
wienui ★ Germany/Oman, 2020-01-30 19:53 (1712 d 16:24 ago) @ Helmut Posting: # 21111 Views: 22,216 |
|
Hello Helmut, ❝ For the Gulf Cooperation Council (Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, United Arab Emirates) and South Africa you can use ABE for Cmax with fixed (!) limits of 75.00–133.33% if CVwR <30%. I think there is a mistake here, you are right according to the fixed limit of 75-133% in the GCC Guidelines in the case of HVD (HVDP) which is absolutely not correct and cause a lot of problems in the evaluation of such drug formulation's types, BUT widen to this range is not in the case of ABE and not at all if CVwR <30% . As according to GCC GL (citation down) a replicate design and prove of CVwR >30% ( not<) is required to wide only the Cmax. "Those HVDP for which a wider difference in Cmax is considered clinically irrelevant based on a sound clinical justification can be assessed with a widened acceptance range. If this is the case, a wider acceptance range (i.e. 75-133%) for Cmax can be used. For the acceptance interval to be widened the bioequivalence study must be of a replicate design where it has been demonstrated that the within subject variability for Cmax of the reference compound in the study is >30%." Best regards, — Cheers, Osama |
Helmut ★★★ Vienna, Austria, 2020-01-30 20:18 (1712 d 15:59 ago) @ wienui Posting: # 21112 Views: 22,319 |
|
Salam Osama, ❝ ❝ […] with fixed (!) limits of 75.00–133.33% if CVwR <30%. ❝ ❝ I think there is a mistake here, you are right according to the fixed limit of 75-133% in the GCC Guidelines in the case of HVD (HVDP) which is absolutely not correct and cause a lot of problems in the evaluation of such drug formulation's types, BUT widen to this range is not in the case of ABE and not at all if CVwR <30% . Of course, of course! Sorry, it was a stupid typing error. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
wienui ★ Germany/Oman, 2020-02-03 08:10 (1709 d 04:08 ago) (edited on 2020-02-03 17:16) @ Helmut Posting: # 21129 Views: 20,950 |
|
Dear Helmut, ❝ Of course, of course! Sorry, it was a stupid typing error. We all make such silly typing error. I would like only here really to express my sincere gratitude and thanks for the great excellent effort which you did and do. As a previous academic person who worked also in the industry and now in the regulatory, I still till the moment (I think better to say we still) learning a lot from you and other colleagues. For all of yours, I wish and hope the continuously for this Forum to illuminate our problems in the dark BE world. Best regards, — Cheers, Osama |
Helmut ★★★ Vienna, Austria, 2020-02-03 13:25 (1708 d 22:52 ago) @ wienui Posting: # 21130 Views: 20,922 |
|
Dear Osama, I think that the GCC’s guideline is not that bad (following what the EMA’s Q&A of July 2006 required). During the development of the EMA’s GL (and even in the June 2010 EGA/EMA joint workshop when the GL was already final) the industry lamented that for CVwR 30–40% one would need larger sample sizes than before. I kept my mouth shut because for years I was an advocate of reference-scaling. Although I was aware of the 2009 paper of the two Lászlós, it needed Detlew’s post of 2013 to comprehend the ugly problem of the inflated Type I Error. Example: 2×2×4 design, assumed T/R ratio 0.90, powered for ≥80%; in ABE at CVwR 30% switching the BE limits from 80.00–125.00% to 75.00–133.33%. For extreme CVs (say, 300% for some LALAs like mesalazine) sample sizes are nasty anyway (430 subject for the fixed wider limits and 224 for ABEL)… I designed such studies (reference-scaling for the EMA and the FDA) and the problem is less the CV but the uncertain T/R ratio driving the sample size. If the T/R ratio drops from 0.90 to 0.85, the sample size doubles to 448. ❝ I would like only here really to express my sincere gratitude and thanks for the great excellent effort which you did and do. As a previous academic person who worked also in the industry and now in one of the regulatory body I still till the moment (I think better to say we still) learning a lot from you and other colleagues. THX for the flowers! BTW, you are not the only regulator here. We have members of different European agencies, the Eurasian Economic Union, ASEAN states, ANVISA, and – believe it or not – the FDA. ❝ For all of yours, I wish and hope the continuously for this Forum to illuminate our problems in the dark BE world. You should have typeset “dark” also in bold. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2020-02-05 19:53 (1706 d 16:25 ago) @ Helmut Posting: # 21149 Views: 20,098 |
|
Dear Helmut, ❝ ... ❝ Example: 2×2×4 design, assumed T/R ratio 0.90, powered for ≥80%; in ABE at CVwR 30% switching the BE limits from 80.00–125.00% to 75.00–133.33%. ❝ ...What are these saw teeth pattern after CV=0.4 in the bottom picture arising from ? — Regards, Detlew |
Helmut ★★★ Vienna, Austria, 2020-02-05 20:46 (1706 d 15:32 ago) @ d_labes Posting: # 21151 Views: 20,321 |
|
Dear Detlew, ❝ What are these saw teeth pattern after CV=0.4 in the bottom picture arising from ? Who the heck wrote this package? Magnification: CV (%) n.ABE pwr.ABE TIE.ABE n.ABEL pwr.ABEL U TIE.ABEL — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-02-06 12:38 (1705 d 23:40 ago) @ Helmut Posting: # 21152 Views: 19,996 |
|
Ok, guys let discuss a bit further. I am gonna describe two bad scenarios. The first one, we assume that the CV of a drug is around 35%, so we are ready for the worst inflation TIE in our ABEL trial with 4-periods. Thus, we decided to increase our sample size to 42 subjects to compensate for the adjusted TIE and then added the 15% dropout rate that resulted in 49 subjects in our full sample size. We have done our trial. We have got the CV around 32% and big inflation TIE, but we passed because of safety measures. The second is a bit worse. We assume that the CV of a drug is around 40-42%. We decided to recruit 40 subjects - with full compensation - to conduct a 4 periods ABEL trial. Then have suddenly got the CV 32% and massive inflation TIE. We failed the trail because of the lack of power. We did the second trial with 49 subjects and passed. There are a couple of questions. Are these scenarios are plausible? How frequent can they be? Could three-periods trials reduce the number of subjects? In terms of the flat world and also as response to the remark from mittury. In Belarus, if you decided to conduct a crossover ABE trial in multiple groups and decided to drop the group factor and its interaction from the statistical model (3rd method of FDA) then you have to prove the groups are from the same population. You should statistically compare the demographics of groups. And you can only drop the group factor if there are no statistical differences between groups irrespectively of the fact that subjects have been randomized to the group before the first period of the first group. But how it is possible to have different populations if people have been recruited from the same places (Minsk and its surroundings for example) and according to the same inclusion and exclusion criteria. It is not like one group is from Belarus and the other is from China or Australia. Again, the design is the crossover. Also, mittury, you should take into account that not all sections of the Euroasian guideline are applied in Belarus at this time. It is left at the discretion of experts. One example. If they decide that a drug is risky (I do not know how they do this), then a sponsor should measure analytes - if the FDA says that - irrespectively of what is written in the guideline and the fact that the parent drug can be perfectly measured. No analytes and only parent drug, no registration. They do not disclose what other sections can be applied differently. Best regards |
Helmut ★★★ Vienna, Austria, 2020-02-06 21:12 (1705 d 15:06 ago) @ Mikalai Posting: # 21155 Views: 20,001 |
|
Hi Mikalai, ❝ I am gonna describe two bad scenarios. ❝ ❝ The first one, we assume that the CV of a drug is around 35%, so we are ready for the worst inflation TIE in our ABEL trial with 4-periods. Thus, we decided to increase our sample size to 42 subjects to compensate for the adjusted TIE and then added the 15% dropout rate that resulted in 49 subjects in our full sample size. We have done our trial. We have got the CV around 32% and big inflation TIE, but we passed because of safety measures. I cannot reproduce your numbers. Given. Though you could state in the SAP which adjusted α you expect (and that you increased the sample size accordingly), you have to state as well that you will adjust α based on the outcome of the study (observed CVwR, actual sample size). The TIE depends not only on CVwR but – to a minor extent – the sample size as well. CV alpha n TIE ← θ0 0.90, 80% power, naïve ABEL: inflated TIE0.35 0.036100 38 0.050000 ← larger sample size for adjusted α0.35 0.036100 44 0.050145 ← compensate for 15% dropouts, inflated TIE!0.35 0.035972 44 0.050000 ← requires more adjustmentIf the CVwR is higher than expected, open a bottle of Champagne; more scaling, less adjustment necessary and you gain power. You are right that you have to adjust more if the CVwR moves towards 30% but the loss in power in this example (35% → 32%) is negligible. CV alpha n do (%) TIE alpha.ad TIE.ad power 58 lines of R-code upon request. ❝ The second is a bit worse. We assume that the CV of a drug is around 40-42%. We decided to recruit 40 subjects - with full compensation - to conduct a 4 periods ABEL trial. Then have suddenly got the CV 32% and massive inflation TIE. We failed the trail because of the lack of power. Shit happens.
n alpha power ❝ We did the second trial with 49 subjects and passed. No dropouts this time? Risky; in such a case I would power the second study more. ❝ There are a couple of questions. ❝ Are these scenarios are plausible? The first one is common, though in my studies I try to educate my clients that this “increase the sample size based on anticipated dropout rate” is a waste of money. Try the function pa.scABE() of PowerTOST to see why.❝ How frequent can they be? No idea about the second one. I know just one case where the sponsor doubled the sample size… ❝ Could three-periods trials reduce the number of subjects? On the contrary, my dear Dr Watson! The power of study depends on the number of treatments. Compared to a 2×2×2 you get similar power with ~½ of the sample size in a 4-period and ~¾ of the sample size in a 3-period replicate or, if you prefer, you need ~50% more subjects in a 3-period than in a 4-period replicate. ABE Study costs are similar (since mainly driven by bioanalytics). However, the more periods, the more likely subjects drop out. One the other hand, the 3-period replicate requires more adjustment than the 4-period replicate. % sample size penalty of adjusted ABEL ❝ In Belarus, if you decided to conduct a crossover ABE trial in multiple groups and decided to drop the group factor and its interaction from the statistical model (3rd method of FDA) then you have to prove … We cannot prove anything in science. Proofs belong to the realms of logic and mathematics. ❝ … the groups are from the same population. You should statistically compare the demographics of groups. And you can only drop the group factor if there are no statistical differences between groups … That’s an extremely stupid approach. At which level will you test? 0.05, 0.10? What about false positives? BTW, any pre-test inflates the TIE and a post hoc test is nonsense.* We discussed that ad nauseam! Explore those. Don’t dive into such muddy waters. Include the group term in the model (i.e., specify the FDA’s Model II taking into account that groups were tested on different dates) but without a pre-test. The loss in power is very, very low compared to the pooled Model III. See the end of Example 1.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Mikalai ★ Belarus, 2020-02-10 17:10 (1701 d 19:08 ago) (edited on 2020-02-11 10:21) @ Helmut Posting: # 21157 Views: 18,967 |
|
Dear Helmut There are some additional questions ❝ I cannot reproduce your numbers. Given. My calculation was that I assumed that CV is 34 but we braced ourselves for the worst scenario. Am I correct that the worst scenario is around 42 subjects to compensate for TIE at CV 30% and 7(15%) subjects to compensate for a dropout rate? PS Total sample size 49 I assume that the TIE calculation is done after we have got our results from the sample. Is this calculation is straight forward with no other statistical approaches that may bring different results? I guess that the TIE calculation and compensation for TIE correction were realized in PowerTost. What about validation issues? Any feedback from regulators on software? How should we proceed if CV from our sample is 30.00%? Should we calculate TIE if CV is less than 30% (I guess no)? Has anyone submitted an ABEL trial with TIE correction and has got no objections from European regulators or FDA (Is this applicable for FDA?). ❝ Include the group term in the model (i.e., specify the FDA’s Model II taking into account that groups were tested on different dates) but without a pre-test. The loss in power is very, very low compared to the pooled Model III. Can the use of Model 2 (FDA), basically group and its interactions, be a reason to reject study in Europe? Or it is a more conservative model and if we pass with this model we always pass with the standard ABE model? Best regards |
zizou ★ Plzeň, Czech Republic, 2020-02-01 18:00 (1710 d 18:18 ago) @ Helmut Posting: # 21126 Views: 21,340 |
|
Hello everybody and nobody. ❝ you are modifying the common decision scheme. Why? ❝ Since you are making essentially the same decisions though in a different order, you gain nothing. Exactly... ❝ Well, that’s the problem with ABEL (and the FDA’s RSABE as well). There is no Null Hypothesis. and exactly again! In standard ABE, null hypothesis (bioinequivalence) and alternative hypothesis (bioequivalence) are known. The BE is defined using the acceptance limits 80-125% for 90% CI of GMR. The TIE (Type I Error) is the probability (alpha) of rejection of true null hypothesis by test. (It's clear but problematic when null hypothesis is unknown and even more problematic in mentioned different order - see below.) TIE inflation for ABEL/RSABE with widening the acceptance limits (because of floating acceptance limits) is described in different topics many times. The suggested different order of analysis in ABEL/RSABE is only a bad attempt to hide the increased patient's risk. When we apply the standard limits of 80-125% in the first step and conclude BE using 90% CI - end of analysis ... hey, it looks like standard study with TIE <=5%. But it is only the part of the decision tree. It reminded me the Monty Hall problem with intuitive 50–50 chance. When we look only at a part of the game, it seems really as 50-50. Back to the bioequivalence: Can we say after the conclusion of BE in the first step that there was no option to wide the acceptance limits? If yes - Ok. If no - There was additional probability to conclude BE. TIE inflation is obvious when you look at the whole plan. The simulations would certainly confirm TIE inflation. Keep in mind that the observed GMR and intra-subject CV of reference are only estimates, we don't know the true (population) values. Imagine the border case example with true GMR 80% (and true intra-subject CV of R 35%), 5% of studies will pass the acceptance criteria 80-125% in the first step and another few % of studies in the second step (after widening of limits). With true GMR 80%, study was performed and we were lucky in observed values, so we concluded BE in the first step. But we can't say that TIE was <=5%! This different order is more controversial than the standard order according the guidelines. Null hypothesis is set in the first step of different order. BE is evaluated - concluded or not. When the test failed to conclude bioequivalence, the new null hypothesis will be tested with different definition of BE. BE is evaluated again - concluded or not. Standard order is clear, you construct the null hypothesis and test it. With different order we can fail to demonstrate BE from the first test, oh no..., so we redefine bioequivalence, i.e. new hypothesis, new test (comparison), we have the second chance. Btw. When I was starting to write the post I thought that order doesn't change the TIE, but now... I think, the different order has higher TIE than the standard order. (Helmut was faster with similar example.) Example 2x2x4 replicate design with the same sample size n=24 for both orders: assumed GMR 90%, assumed intra-subject CV of R 50% (for higher intra-subject CVs there will be higher probability of two tests in different order).
Moreover the first test is totally irrelevant if assessed as the whole. If the first test is BE (the second test would always confirm that), if not, only the second test would be decisive. So equal result of study you receive by testing only the second test, so I don't see any reason for using the different order - no pros, possible TIE inflation as cons. Best regards, zizou |
nobody nothing 2020-02-02 00:30 (1710 d 11:47 ago) @ zizou Posting: # 21128 Views: 21,355 |
|
❝ Hello everybody and nobody. Lately more often than not I sit here can't stop shaking my head for hours, sometimes I think it must be Parkinsons' ***sigh*** — Kindest regards, nobody |
Elena777 ☆ Belarus, 2020-03-10 20:28 (1672 d 15:50 ago) @ Helmut Posting: # 21235 Views: 16,723 |
|
Dear Helmut, still have some doubts regarding decision scheme. As per you comment: ❝ you are modifying the common decision scheme. Why? It seems not only I would like to modify a decision scheme. I saw the following conclusion in the PAR (https://db.cbg-meb.nl/Pars/h116289.pdf). Conclusion on bioequivalence study The reference product was found to be highly variable for Cmax with an intra-subject variability greater than 30%: the intra-individual coefficient of variation was 41.62%. Widening of the acceptance intervals for Cmax was allowed but, based on the study results, proved not to be necessary. The 90% confidence intervals calculated for AUC0-t and Cmax are within the bioequivalence acceptance range of 0.80 – 1.25. Based on the submitted bioequivalence study Atorvastatine STADA is considered bioequivalent with Lipitor. As I understand, they did not use the decision scheme described in the lecture. Otherwise it was obligatory for them to modify CI for Cmax. But they just stated that "widening of the acceptance intervals for Cmax was allowed but, based on the study results, proved not to be necessary". So, how do you think what decision tree was used in that study? |
Helmut ★★★ Vienna, Austria, 2020-03-10 22:06 (1672 d 14:12 ago) @ Elena777 Posting: # 21236 Views: 16,958 |
|
Hi Elena, ❝ So, how do you think what decision tree was used in that study? No idea. In a PAR we see always the wording of the agency. Without the study protocol & report we simply don’t know what exactly was planned & performed. I have an CDA with the sponsor in place and a friend there owes me a favor. No guarantees. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
nobody nothing 2020-02-03 16:07 (1708 d 20:11 ago) @ Elena777 Posting: # 21131 Views: 20,938 |
|
— Kindest regards, nobody |
Astea ★★ Russia, 2020-04-02 14:41 (1649 d 22:36 ago) @ nobody Posting: # 21313 Views: 14,085 |
|
Dear Friends! I'm very sorry for digging this theme once more, but my female logic suffers from trying to get how could the sequence of statements influence the TIE. I suppose the TIE in alternative Elena's approach to be the same as in the standard scheme. Would you be so kind to explain it for one thousand and first time again? Let A be the statement that CI is in [80,125], B - the statement that CV<30% and C - the statement that CI is [L,U], where L and U are the scaled limits. Then BE statement is (B∧A)∨(¬B∧C) for standard approach and (A∨(¬A∧(¬B∧C))) for the alternative approach. The non BE statements are (¬A∧B)∨(¬A∧¬C)∨(¬B∧¬C) and (¬A∧B)∨(¬A∧¬C) correspondingly. The only difference is (¬B∧¬C) which is logically equals to (¬A∧¬C) when ¬B is TRUE and zero if ¬B is FALSE. Testing the data twice is obvious in the alternative sequence, but the second test doesn't suffer from the error on CV_R because the first test A has already done. And in the standard approach we also assess the data twice: first to get CV_R, then to calculate CI. I'm totally lost: my gut feeling says that there should not be difference, TIE is the same in both cases. I read the thread once and once again and still have internal contradictions that I couldn't prove. Practically: while dealing with scaled approach I program to get CI as well as CV_R in the output at the same time. What I should do - close the left eye not to see that CI is in the [80-125] and only looking for the CV_R by my right eye? — "Being in minority, even a minority of one, did not make you mad" |