Elena777
☆    

Belarus,
2020-01-28 08:02
(1521 d 07:02 ago)

Posting: # 21090
Views: 22,211
 

 Statistical evaluation and BE hypo­theses in full replicate design [RSABE / ABEL]

Dear All,

Is it OK to use the following approach of statistical evaluation in a BE study with full replicate design:
  1. Average bioequivalence is used to determine bioequivalence first.
  2. If bioequivalence is not met for Cmax, scaled-average-bioequivalence can be used if both of the following conditions are met:
    • the point estimate of the geometric mean ratio lies within the 80.00-125.00% range;
    • the R-R within-subject CV is > 30%, and is not the result of outliers?
And another question: is this obligatory to include the detailed description of null and alternative hypotheses for Cmax (ABEL) and AUCt (ABE) together with description of conditions that lead to acceptance/rejection of a hypothesis in a protocol of a full replicate study?


Edit: Category changed; see also this post #1[Helmut]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-29 16:38
(1519 d 22:26 ago)

@ Elena777
Posting: # 21095
Views: 19,713
 

 Inflation of the TIE as well

Hi Elena,

you are modifying the common decision scheme. Why?
Since you are making essentially the same decisions though in a different order, you gain nothing.

❝ And another question: is this obligatory to include the detailed description of null and alternative hypotheses for Cmax (ABEL) and AUCt (ABE) together with description of conditions that lead to acceptance/rejection of a hypothesis in a protocol of a full replicate study?


Well, that’s the problem with ABEL (and the FDA’s RSABE as well). There is no Null Hypothesis. Before you assess the data you have no clue which path(s) of the framework you will follow. You can – and should – only describe the procedure.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Elena777
☆    

Belarus,
2020-01-29 21:01
(1519 d 18:03 ago)

@ Helmut
Posting: # 21098
Views: 19,646
 

 Inflation of the TIE as well

Dear Helmut,
Let me describe the situation in detail.
We recently conducted a replicative BS (EMA approach). We submitted the study to the Belarusian NCA and received comments on statistics. In the protocol, we stated that first we would try ABE, and if we failed then would go to scaling. We passed the BE criteria within 80-125% range for both Cmax and AUC (ABE approach). The expert demanded to calculate RR that we did not do but can do, but also to control TIE that we think is nonsense. As a result, I have a couple of additional questions.
  1. Is calculation RR for the reference drug mandatory and used somewhere else, except to widen the bioequivalence interval for Cmax (reference to my first questions)?
  2. Is TIE nominal and cannot be controlled if we do not widen the bioequivalence interval in replicative studies and use ABEL?
  3. Could you provide us with formula to calculate CI for fully replicative studies in math not computer format?
  4. How mathematically (in formula) can the inflation of TIE take place in RBS if we nominally control t-parameter (choose it from tables, for example)?
  5. What role of fixed or random effects in RBS or ordinary BS?

I am not a statistician
Best regards
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-30 13:07
(1519 d 01:58 ago)

@ Elena777
Posting: # 21101
Views: 19,707
 

 Tricky…

Hi Elena,

❝ In the protocol, we stated that first we would try ABE, and if we failed then would go to scaling.


So far, so good (though usually you would check for a CVwR >30% first).

❝ We passed the BE criteria within 80-125% range for both Cmax and AUC (ABE approach).


OK.

❝ The expert demanded to calculate RR that we did not do but can do, …


Likely he/she is interested whether the reference is a HVD(P).

❝ … but also to control TIE that we think is nonsense.


Tricky. In your original post you stated also a decision tree. Any of these decisions made can be wrong, which will inflate the TIE. Imagine you failed ABE not because the reference is highly variable but by lacking power. Then you would continue to #2. The observed CVwR was >30% by chance and you expand the limits. However, the true CVwR (in the patient population) is ≤30%, and the decision wrong – inflated TIE (see there). More in the answer to #4.

❝ As a result, I have a couple of additional questions.


❝ 1. Is calculation RR for the reference drug mandatory …


Quoting the EMA’s GL:

For the acceptance interval to be widened the bioequivalence study must be of a replicate design where it has been demonstrated that the within-subject variability for Cmax of the reference compound in the study is >30%.

Similar wording in other jurisdictions (ASEAN States, Australia, Canada, East African Community, Egypt, Eurasian Economic Union, New Zealand, Russian Federation, the WHO).

❝ … and used somewhere else, except to widen the bioequivalence interval for Cmax (reference to my first questions)?

  • For the Gulf Cooperation Council (Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, United Arab Emirates) and South Africa you can use ABE for Cmax with fixed (!) limits of 75.00–133.33% if CVwR >30%.*
  • AUC additionally for the WHO, where you also have to compare within-subject variabilities of T and R.
  • For Health Canada AUC only.
  • Any PK metric for the FDA and China’s CDE (though by another method called RSABE). In the linked guidance you see that the assessment whether the reference is highly variable sits on top of the decision tree. Only if swR <0.294 (CVwR ~30%), you would assess the study by ABE – not the other way ’round.

❝ 2. Is TIE nominal and cannot be controlled if we do not widen the bioequivalence interval in replicative studies and use ABEL?

❝ 3. Could you provide us with formula to calculate CI for fully replicative studies in math not computer format?


It’s the same one like in ABE.

❝ 4. How mathematically (in formula) can the inflation of TIE take place in RBS if we nominally control t-parameter (choose it from tables, for example)?


RBS = replicated biostudy?
The problem with decision trees is that control of the TIE cannot be shown analytically (by a formula) – we need simulations. My gut feeling tells me that in your case (passing ABE in step #1) there should be no problem. But who knows?

     I try not to think with my gut.
If I’m serious about understanding the world,
thinking with anything besides my brain, as tempting as that might be,
is likely to get me into trouble.
    Carl Sagan

I’m too busy to set up simulations. Ask a statistician (hint: R is much faster than SAS, MATLAB, or GNU Octave).

However, you could check the TIE for the conventional framework in PowerTOST. You need CVwT, CVwR, and the number of subjects / sequence. Example (one million simulated studies by default):

library(PowerTOST)
CVwT   <- 0.20
CVwR   <- 0.20
CV     <- c(CVwT, CVwR)
n      <- c(12, 12) # subjects / sequence (arbitrary order)
theta0 <- scABEL(CV = CVwR)[["upper"]]
# simulations via the key statistics
power.scABEL(CV = CV, n = n, theta0 = theta0, design = "2x2x4")
# simulations based on subject data (slower)
power.scABEL.sdsims(CV = CV, n = n, theta0 = theta0, design = "2x2x4")

If the empiric TIE is ≤0.05, fine with the usual approach. Whether this is sufficient to convince the expert, duno. Not what you planned & did.

❝ 5. What role of fixed or random effects in RBS or ordinary BS?


That’s almost a philosophical question. In the strict sense:
  1. If you treat subjects as a fixed effect, you make a statement about the subjects in the study.
  2. If you treat them as a random effect, you make a statement about the population of other subjects.
At the end of the day you extrapolate the results of the study to the population of patients. Some statisticians (including ones of the FDA, Health Canada, China’s CDE, and myself) think that #2 is the correct way. Others (of the EMA, …) prefer #1. If the study is balanced and complete (i.e., no missing periods) the outcome is identical.
I performed large simulations and seemingly the EMA’s ‘Method A’ is slightly more conservative than ‘Method B’. However, both methods assume homoscedasticity (identical within-subject variabilities of T and R). A rather strong assumption which was demonstrated to be false is numerous studies… The FDA’s method (termed ‘Method C’ by the EMA) – which is a full-blown mixed-effects model – is generally more conservative (wider confidence interval). But that’s another story.

❝ I am not a statistician


I tried to answer in a not too statistical way.



Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-30 14:08
(1519 d 00:57 ago)

(edited by Mikalai on 2020-01-30 15:08)
@ Helmut
Posting: # 21102
Views: 19,633
 

 Tricky…

Dear Helmut,

❝ Tricky. In your original post you stated also a decision tree. Any of these decisions made can be wrong, which will inflate the TIE. Imagine you failed ABE not because the reference is highly variable but by lacking power. Then you would continue to #2. The observed CVwR was >30% by chance and you expand the limits. However, the true CVwR (in the patient population) is ≤30%, and the decision wrong – inflated TIE (see there). More in the answer to #4.


It is a bit confusing, but could you clarify a couple of points.
  1. In the case of going by the ABE path (see the decision tree), I assume we do not use any population parameters and just compare concentrations from samples. There is no inflation TIE? Is it correct? And we do not use RR for the ref drug in our calculation. Is it correct?
  2. But if the study fails, we then calculate RR and a new bioequivalence interval if CV is higher than 30%. That is when we use RR and where TIE can be inflated. Is it correct? But we do not recalculate our new CI we just use that from ABE to compare with a new or the old CI. In the latter case, we basically fail the study. Is it correct?
The identical decision tree was used in the BS of rasagiline (registration required) and was accepted by EMA. https://clinicaldata.ema.europa.eu/web/cdp/home
If I am correct, in my view it is a preferable solution to minimize the inflation TIE.
Best regards
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-30 16:09
(1518 d 22:55 ago)

@ Mikalai
Posting: # 21104
Views: 19,626
 

 Terrible…

Hi Mikalai,

❝ ❝ […] see there


❝ 1. In the case of going by ABEL path (see the decision tree), I assume we do not use any population parameters and just compare concentration from samples.


Nope. Scaled ABE (in both of its flavors ABEL and RSABE) is formulated in the unknown population parameters (see the linked slide above). For ABEL we have
  • the fixed regulatory standardized variation \(\sigma_0=\sqrt{\log_{e}(0.30^2+1)}=0.2935604\ldots\)
  • which leads to the switching condition \(\theta_s=\frac{\log_{e}(1.25)}{\sigma_0}\small{=0.7601283\ldots}\) (given in the guideline as k and for reasons beyond my intellectual reach rounded to 0.760).
  • The expanded limits are \([L,U]=\text{e}^{\mp \theta_s\cdot \sigma_{wR}}\).
  • At the end we assess whether the 90% CI of \(\mu_T-\mu_R\) lies entirely within \([L,U]\).
Note the Greek letters all over the place. What we get from the study are only estimates of the parameters.

❝ There is no inflation TIE? Is it correct?


No it isn’t. On the contrary, the inflated TIE arises from a misspecification of CVwR. We think that the drug/drug product is highly variable, expand the limits and pass. But the true CVwR is <30%, i.e., not a HVD(P) and we would have failed ABE. That’s it.

❝ And we do not use RR for the ref drug in our calculation. Is it correct?


If you mean by ‘we’ Elena’s approach, and stop since you showed ABE, no.

❝ 2. But if the study fails, we then calculate RR and a new bioequivalence interval if CV is higher than 30%. That is when we use RR and where TIE can be inflated. Is it correct?


Yes.

❝ But we do not recalculate our new CI we just use that from ABEL to compare with a new or the old CI. In the latter case, we basically fail the study. Is it correct?


I try to repeat Elena’s procedure. You make a lot of decisions (any of them can be wrong):
  1. Assess ABE with α 0.05.
    • If the 90% CI entirely outside 80.00–125.00%, stop (bioinequivalent).
    • If the 90% CI within 80.00–125.00%, stop (pass).
    • If not, calculate CVwR (and the stupid outlier check)
  2. ABEL branch
    • ≤30% → stop (fail).
    • >30% → expand the limits.
      • If the 90% CI is not entirely within [L,U] → stop (fail).
      • Otherwise, check additionally whether the PE is within 80.00–125.00%.
        If no, fail.
        If yes, pass.
Apart from the issues with the misspecification of CVwR, if you proceed with this approach to the second step the TIE will be inflated for sure. The entire nominal α is already ‘consumed’ in step #1 – there is simply nothing ‘left’ for step #2. I would never use such a framework. Even if you use Bonferroni’s 0.025 in the first step, there might still an inflation due to CVwR. Check out again slide 15. In this approach the CI is only calculated once.

❝ The identical decision tree was used in the BS of rasagiline (registration required) and was accepted by EMA. https://clinicaldata.ema.europa.eu/web/cdp/home


This site is a pain in the back (refuses my credentials, send as a reminder the user name I just typed in, etc.). If this procedure was really accepted, that’s even worse than the ‘usual’ inflation of up to ~0.09… Oh dear!
Maybe I’ll set up simulations, time allowing. I expect the worst.

❝ If I am correct, in my view it is a preferable solution to minimize the inflation TIE.


Sorry, not at all.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-30 17:02
(1518 d 22:03 ago)

@ Helmut
Posting: # 21106
Views: 19,518
 

 Tricky…

Dear Helmut,
I would like to apologize because I confused you in my previous post. I should have used ABE instead of ABEL as an abbreviation because I meant the average bioequivalence. I corrected it recently.
Thus, we do not use RR for the ref drug in our calculations and should not control TIE if we initially go by the ABE approach and reach bioequivalence within 125-80% interval (alpha at 0.05).

Also, it seems the only reason that we should calculate RR for the ref drug first is to calculate and control TIE latter if the CV is higher than 30%. If it turns out that it is lower than 30% then we simply use the ABE and no control TIE. However, if we do not plan to do this (control TIE), then it is not important to calculate the RR ref first. As you mentioned there Post and there post no regulatory agency issued a formal guideline on TIE in replicate studies, and FDA and EMA do not require and accept replicate studies without controlling TIE. In this case (no control TIE), we can calculate the RR only when we fail in Cmax. We do not recalculate our new CI we just use that from ABE to compare with a new or the standard CI if CV less than 30. In the latter case, we basically fail the study.

I would not like to discuss whether we should or not control TIE in replicate studies in this branch. In my view, probably we should not, and it is also probably that leading agencies won't do anything with this issue unless a safety signal occurs.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-30 17:59
(1518 d 21:06 ago)

@ Mikalai
Posting: # 21108
Views: 19,552
 

 Deep shit

HI Mikalai,

❝ […] we do not use RR for the ref drug in our calculations and should not control TIE if we initially go by the ABE approach and reach bioequivalence within 125-80% interval (alpha at 0.05).


Read my last post again. And again. You do not know beforehand whether or not you will pass ABE. If not, you continue with the calculation CVwR and ABEL. If that happens, you will evaluate the data twice. That’s a textbook-example of multiplicity. Just by that the TIE will be \(1-(1-\alpha)^k=1-(1-0.05)^2=0.0975\). That’s not rocket science and already higher than with the usual framework, where we calculate the 90% CI only once. Add the potential bias of CVwR and you are in deep shit. Excuse my French.

❝ Also, it seems the only reason that we should calculate RR for the ref drug first is to calculate and control TIE latter if the CV is higher than 30%.


No. It prevents us from potentially assessing the CI twice. See above.

❝ If it turns out that it is lower than 30% then we simply use the ABE and no control TIE. However, if we do not plan to do this (control TIE), then it is not important to calculate the RR ref first.


See above. If you continue with this appraoch (IMHO, a bad idea), please use at least a 95% CI.

❝ As you mentioned there Post and there post no regulatory agency issued a formal guideline on TIE in replicate studies, and FDA and EMA do not require and accept replicate studies without controlling TIE.


Unfortunately regulations science.

[image]     In bioequivalence we must not forget the
only important – the patient! He/she is a living
person, not just α 0.05.
     Dirk Marteen Barends


IMHO, it is a cowardly excuse to rely on an approach ‘because a guideline says so’ and the agencies accepts studies (quote of a European assessor: I don’t have enough time to read all those papers…).
There are almost twenty published showing an increased patient’s risk by re­fe­rence-scaling and not a single one (‼) showing the opposite. Even with the FDA’s ‘desired consumer risk model’ – called by some hocus pocus – there is a substantial inflation of the TIE. Your approach is worse.

❝ In this case (no control TIE), we can calculate the RR only when we fail in Cmax. We do not recalculate our new CI we just use that from ABE to compare with a new or the standard CI if CV less than 30. In the latter case, we basically fail the study.


See above.

❝ I would not like to discuss whether we should or not control TIE in replicate studies in this branch. [image]In my view, probably we should not, …


Well, that’s like my niece being afraid of monsters in the nursery. Her solution: Close the eyes and cover the ears with her hands.

❝ … and it is also probably that leading agencies won't do anything with this issue unless a safety signal occurs.


Forget pharmacovigilance. It is terribly insensitive. There are some HVD(P)s where the concentrations vary tenfold day-to-day. Seriously. Yep, even the maximum expansion is too conservative. Furthermore, we are not worried about drugs / drug products with a true CVwR of ~45% and higher. The troublesome are the borderline cases. :thumb down:


PS: Waiting for a guideline takes quite some perseverance. Grizzle’s concept of testing for a sequence effect (unequal carry-over) of 1965 was proven false by Freeman in 1989. Took another 21 years to make it to the EMA’s GL.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-30 19:39
(1518 d 19:26 ago)

@ Helmut
Posting: # 21110
Views: 19,454
 

 Deep shit

Dear Helmut,

First of all, let's clarify a bit further. Why should we calculate CI twice if we do not plan control TIE.

❝ Read my last post again. And again. You do not know beforehand whether or not you will pass ABE. If not, you continue with the calculation CVwR and ABEL. If that happens, you will evaluate the data twice. That’s a textbook-example of multiplicity. Just by that the TIE will be \(1-(1-\alpha)^k=1-(1-0.05)^2=0.0975\). That’s not rocket science and already higher than with the usual framework, where we calculate the 90% CI only once. Add the the potential bias of CVwR and you are in deep shit. Excuse my French.


As I understand the formula of CI for ABE and RABE/ABEL is the same (2 sequence 4-period study). Or I am wrong? We calculate ABE CI. If Cmax is out of range, we than calculate the RR for ref drug and recalculate a new bioequivalence interval if the CV is higher than 30. And then we compare whether our old ABE CI is within the new interval. In this case, we calculate CI once. Am I wrong?

In terms of the decision tree, this approach is used. It is not unique. Multiple studies were accepted in Europe. It seems that there have been no complaints from European regulators.

I would not like to discuss whether we should or not control TIE in ABEL/RABE in this branch, because it may divert from the main topic. But I can explain my position. It seems that to control TIE we may have to change the alpha after getting results from the study sample. It potentially can reduce power and the only way to balance this is to recruit more subjects. The question is how many? One should take into consideration that ABEL/RABE trials are more challenging: high dropout rates (longer than usual BE trials) and riskier (more AE because of more blood and more drugs). If we can find a solution when we can increase the sample size slightly, then it is OK; otherwise, we may approach the sample size of a 2-period trial that basically invalidates the whole concept.

There is also another point. In Europe, we cannot expand limits for AUC and can for Cmax when it is safe. Why cannot we tolerate in this situation TIE that is higher than 5% (why 5%?), let's say 10%? Nothing is 'carved in the stone'. It appears that no drugs were withdrawn because of inflated TIE yet.

Also whether we have reliable methods to control TIE statistically on which regulators agree?

Finally, I disagree that pharmacovigilance is senseless. If it is done properly, it should and can pick up bad players (drugs and companies).

Best regards
Mikalai
★    

Belarus,
2020-01-30 20:31
(1518 d 18:34 ago)

@ Mikalai
Posting: # 21113
Views: 19,534
 

 Deep shit

I would like to include this as a quote.

❝ No. It prevents us from potentially assessing the CI twice. See above.


Best regards
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-30 21:07
(1518 d 17:57 ago)

@ Mikalai
Posting: # 21114
Views: 19,729
 

 Deep shit

Hi Mikalai,

❝ […] Why should we calculate CI twice …


You don’t calculate it twice, you possibly assess it twice.

❝ … if we do not plan control TIE.


Cause you care about the patients? If you don’t give a shit about other people consider changing jobs and become a stock broker.

❝ As I understand the formula of CI for ABE and RABE/ABEL is the same (2 sequence 4-period study). Or I am wrong? We calculate ABE CI. If Cmax is out of range, we than calculate the RR for ref drug and recalculate a new bioequivalence interval if the CV is higher than 30. And then we compare whether our old ABE CI is within the new interval. In this case, we calculate CI once. Am I wrong?


See above. You calculate the CI once. If you fail ABE (first assessment with a nominal α 0.05), you assess it for ABEL (second assessment with a nominal α 0.05). Is it clear now? Two tests, each performed at level 0.05. Inflated Type I Error. Full stop.

❝ In terms of the decision tree, this approach is used. It is not unique. Multiple studies were accepted in Europe. It seems that there have been no complaints from European regulators.


Awful, just awful.

❝ […] It seems that to control TIE we may have to change the alpha after getting results from the study sample. It potentially can reduce power and the only way to balance this is to recruit more subjects. The question is how many?


Easy to answer.

❝ One should take into consideration that ABEL/RABE trials are more challenging: high dropout rates (longer than usual BE trials) …


Opt for one of the three period full replicates. Loss of power due to dropouts is overrated.

❝ … and riskier (more AE because of more blood and more drugs).


Given.

❝ If we can find a solution when we may increase the sample size slightly, then it is OK;


See the presentation linked above. Peanuts.

❝ … otherwise, we may approach the sample size of a 2-period trial that basically invalidates the whole concept.


For your approach you possibly have to adjust α below Bonferroni’s 0.025.

library(PowerTOST)
CV     <- seq(0.2, 0.5, 0.1)
design <- "2x2x4"
# defaults: theta0 0.90, targetpower 0.80
res <- data.frame(CV = CV, n.ABE = NA, n.ABEL = NA,
                  n.ABEL.Bonf = NA, n.ABEL.adj = NA, n.Molins = NA)
for (j in 1:nrow(res)) {
  res[j, 2]       <- sampleN.TOST(CV = CV[j], design = design,
                                  details = FALSE,
                                  print = FALSE)[["Sample size"]]
  res[j, 3] <- sampleN.scABEL(CV = CV[j], design = design, details = FALSE,
                              print = FALSE)[["Sample size"]]
  res[j, 4] <- sampleN.scABEL(alpha = 0.025, CV = CV[j], design = design,
                              details = FALSE, print = FALSE)[["Sample size"]]
  res[j, 5] <- sampleN.scABEL.ad(CV = CV[j], design = design, details = FALSE,
                                 print = FALSE)[["Sample size"]]
  alpha.adj <- scABEL.ad(CV = 0.3, design = design, n =  res$n.ABEL[j],
                         details = FALSE, print = FALSE)[["alpha.adj"]]
  res[j, 6] <- sampleN.scABEL(alpha = alpha.adj, CV = CV[j],
                             design = design, details = FALSE,
                             print = FALSE)[["Sample size"]]
}
res$CV <- 100*res$CV; names(res)[1] <- "CV (%)"
print(res, row.names = FALSE)

CV (%) n.ABE n.ABEL n.ABEL.Bonf n.ABEL.adj n.Molins
    20    10     18          24         18       22
    30    20     34          44         42       42
    40    34     30          38         32       36
    50    50     28          34         28       32


❝ There is also another point. In Europe, we cannot expand limits for AUC and can for Cmax when it is safe. Why cannot we tolerate in this situation TIE that is higher than 5% (why 5%?), let's say 10%? Nothing is 'carved in the stone'.


Wow! Even for a hardened atheist that smells of heresy. :not really:

❝ It appears that no drugs were withdrawn because of inflated TIE yet.


Duno.

❝ Also whether we have reliable methods to control TIE statistically on which regulators agree?


As long as you use the same framework / method stated in the GL no regulator on this planet has any reason not to accept an approach using a lower α.

❝ Finally, I disagree that pharmacovigilance is senseless. If it is done properly, it should and can pick up bad players (drugs and companies).


I don’t believe it.
The only example I’m aware of was the formulation change of levothyroxine in France (passed ABE with narrower limits and the AEs went through the ceiling).

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-31 12:14
(1518 d 02:50 ago)

@ Helmut
Posting: # 21115
Views: 19,365
 

 Deep shit

Dear Helmut,
You seem to strengthen my doubts.

❝ See above. You calculate the CI once. If you fail ABE (first assessment with a nominal α 0.05), you assess it for ABEL (second assessment with a nominal α 0.05). Is it clear now? Two tests, each performed at level 0.05. Inflated Type I Error. Full stop.


It has no practical sense for me. As I know trials frequently recalculated with some modifications, usually slight ones. I saw that the FDA did this, for example. I do not see that alpha levels were reduced in those recalculations. If we employ this approach (no recalculations or lower alpha level) we may paralyze the whole industry. There would be massive wastes of resources - a lot of studies would be rejected - and huge risks for patients - depriving, for example, them live-saving medications because trials should be repeated in some cases. Actually, we can ask people from the forum how many sponsors or CROs they know whose trials have never been recalculated?

CV (%) n.ABE n.ABEL n.ABEL.Bonf n.ABEL.adj n.Molins

    20    10     18          24         18       22

    30    20     34          44         42       42

    40    34     30          38         32       36

    50    50     28          34         28       32


It seems that we have to multiple the ABE column by 2 to get full the sample size but not the ABEL column; otherwise, it has no much sense for me. I also wonder if the dropout rate has been considered in the calculations. Looking at your table and slides post, it is appears that in the region CV between 30 - 40% the sample size for ABEL and ABE trials may be very close to each other. If it is true, the ABEL trials (CV between 30-40) should not be allowed by ethic and regulatory bodies because of unnecessary risks for subjects. It invalidates the RABE/ABEL approach in this region.

❝ ❝ Finally, I disagree that pharmacovigilance is senseless. If it is done properly, it should and can pick up bad players (drugs and companies).


❝ I don’t believe it.

❝ The only example I’m aware of was the formulation change of levothyroxine in France (passed ABE with narrower limits and the AEs went through the ceiling).


Actually, it works and even better the BE trials in some cases. Scotland is a good example. In 2015, it had a system, the GP system, that was well developed to spot safety signals. At that time it was not fully developed and tuned. It has some issues like privacy, it collects a lot of private data, complexity, not all doctors know how to operate the system to get full benefits. Also, it was not connected to the pharmacy and hospital systems yet.

Best regards
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-31 15:00
(1518 d 00:04 ago)

@ Mikalai
Posting: # 21119
Views: 19,269
 

 Deep shit

Hi Mikalai,

❝ You seem to strengthen my doubts.


Fine.

❝ ❝ If you fail ABE (first assessment with a nominal α 0.05), you assess it for ABEL (second assessment with a nominal α 0.05). […] Two tests, each performed at level 0.05. Inflated Type I Error. Full stop.


❝ It has no practical sense for me. As I know trials frequently recalculated with some modifications, usually slight ones. I saw that the FDA did this, for example. I do not see that alpha levels were reduced in those recalculations.


Not limited to the FDA. We (CROs, applicants) could never, ever recalculate a study because that would be judged by assessors as cherry-picking. :cherry picking:
Of course, regulators play not in another league but in another sport, which can be expressed as

Quod licet Iovi, non licet bovi

The problem with a recalculation is – since the entire α was already spent in the original analysis – the TIE → ∞. Now what? If a passing study fails now, no problem; the risk is only a theoretical one since the product will not be marketed based on this study. Question: What is the TIE if the recalculated study passes as well?

❝ If we employ this approach (no recalculations or lower alpha level) …


Once you started with an α which controls the TIE in the original analysis, any lower α in the recalculation cannot work. OK, you could submit all studies with a 92% CI (α 0.04) and – if a recalculation is requested – perform it with a 98% CI (α 0.01). Power drops through the floor, good luck.

❝ … we may paralyze the whole industry …


Agree, also with your examples.

❝ Actually, we can ask people from the forum how many sponsors or CROs they know whose trials have never been recalculated?


Good idea. Only very few of mine. ;-)

❝ It seems that we have to multiple the ABE column by 2 to get full the sample size but not the ABEL column; otherwise, it has no much sense for me.


Duno what you mean. Are you considering the sample size of a 2×2×2 crossover? The sample size of the 2-period 4-sequence full replicate is ~½ though the number of treatments / biosamples (driving the study cost) are essentially the same.
CV (%) ABE.2x2x2 ABE.2x2x4 n.ABEL n.ABEL.Bonf n.ABEL.adj n.Molins
    20        20        10     18          24         18       22
    30        40        20     34          44         42       42
    40        66        34     30          38         32       36
    50        98        50     28          34         28       32


❝ I also wonder if the dropout rate has been considered in the calculations.


No. That’s specific to the drug and I never “compensate for potential dropouts in order to maintain power” unless I expect a dropout rate of >15%. Waste of money because the impact of dropouts on power is generally small. Try the functions pa.ABE() and pa.scABE() of PowerTOST.

❝ Looking at your table and slides post, it is appears that in the region CV between 30 - 40% the sample size for ABEL and ABE trials may be very close to each other.


The plots in the slide are all for ABEL. A comparison of ABE and the EMA’s (unadjusted) ABEL:

[image]

At 30% sample sizes are 40 and 34, at 40% 68 and 30.

❝ If it is true, the ABEL trials (CV between 30-40) should not be allowed by ethic and regulatory bodies because of unnecessary risks for subjects. It invalidates the RABE/ABEL approach in this region.


Hhm, not sure what you mean.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-31 17:07
(1517 d 21:58 ago)

@ Helmut
Posting: # 21121
Views: 19,163
 

 Deep shit

Dear Helmut

❝ Duno what you mean. Are you considering the sample size of a 2×2×2 crossover? The sample size of the 2-period 4-sequence full replicate is ~½ though the number of treatments / biosamples (driving the study cost) are essentially the same.

CV (%) ABE.2x2x2 ABE.2x2x4 n.ABEL n.ABEL.Bonf n.ABEL.adj n.Molins

    20        20        10     18          24         18       22

    30        40        20     34          44         42       42

    40        66        34     30          38         32       36

    50        98        50     28          34         28       32

❝ Try the functions pa.ABE() and pa.scABE() of PowerTOST.


❝ The plots in the slide are all for ABEL. A comparison of ABE and the EMA’s (unadjusted) ABEL:


[image]


❝ At 30% sample sizes are 40 and 34, at 40% 68 and 30.


❝ Hhm, not sure what you mean.


I just would like to say that it has no sense to conduct ABEL/RABE trials if the CV is assumed around 30% because, in order to control TIE, we have to increase the sample size to around that of usual ABE trials. It violates GCP principles in this case and invalidates the ABEL/RABE approach (subjects are subjected to unnecessary risks in the ABEL/RABE trials). And it very complicates the matter. Basically, when should we consider an ABEL/RABE trial?

Best regards


It also seems that when we wish to control TIE the sample size will be around 50 (worst scenario + dropout rate) in almost every case (usually little known about reference withindividual variability) of the ABEL/RABE. But I have rarely seen such trials.

Regards


Edit: Merged with a later (now deleted) post. You can edit your posts for 24 hours. [Helmut]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-31 13:19
(1518 d 01:46 ago)

@ Mikalai
Posting: # 21116
Views: 19,289
 

 Flawed evaluation accepted

Hi Mikalai,

❝ The identical decision tree was used in the BS of rasagiline (registration required) and was accepted by EMA. https://clinicaldata.ema.europa.eu/web/cdp/home


I succeeded at the end and you are correct. Why am I not surprised that the study was performed in India? OK, the sponsor was European. One of my clients – should have asked me before… :-(
Clinical phase ended Jun 2009, study report v1 Oct 2009, v2 Mar 2011, v3 (final) Jul 2011. What happened in those 21 (!) months – waiting for the BE GL (published Jan 2010, effective Jul 2010)?

The procedure for Cmax was almost like Elena described.
  • Assess for ABE. If it fails, ABEL will be assessed if both conditions are met:
    • PE within 80.00–125.00%,
    • CVwR >30% and not the result of outliers.
Interesting:
  • PROC MIXED was used with sequence, period, and treatment as fixed effects and subjects as a random effect.
    I like that the stupid nested structure subject(sequence) was not used. However, when it comes to ABE, according to the GL all effects fixed (PROC GLM) are preferred.
  • Post hoc power to detect a 20% difference under the Null of no difference. Jesusfuckingchrist!
  • Lengthy discussion of significant period effect (almost one page).
  • Lund’s test of studentized residuals instead of box plots. Given, though see this post why Lund’s test should not be used – except in 2×2×2 crossovers.
  • The formula for the CI is stated as \(CI_{1,2}=(\bar{x}_{test}-\bar{x}_{ref})\cdot t_{\alpha/2,N-2}\cdot \mp \sigma \sqrt{2/N}\), where \(DF=N-2\).
    Oops!
    • The error term is only correct if sequences are balanced, i.e., \(n_1=\ldots=n_i\) or in case of this 2-sequence 4-period replicate study \(n_1=n_2=N/2\). In general it is \(\sqrt{1/s\sum_{i=1}^{i=s}1/n_i}\), where \(s\) is the number of sequences. Here it did not hurt because the study was balanced (13 subjects in each sequence).
    • What hurts are the degrees of freedom; \(N-2\) is only correct for a balanced 2×2×2 crossover. For a balanced 2-sequence 4-period replicate the degrees of freedom are \(3N-4\). In this case it means the CI would be wider than necessary (24 dfs instead of 74).
      Of course, SAS didn’t give a shit on what is written in the protocol and PROC MIXED came up with 69.5 Satterthwaite’s degrees of freedom and calculated the CI correctly. ;-)
  • The regulatory standardized variation \(\sigma_0\) is given as \(0.25\) (that’s the one of the FDA) and not as the EMA’s \(\sigma_0=\sqrt{\log_{e}(0.30^2+1)}=0.2935604\ldots\) and consequently the switching condition \(\theta_s=\frac{\log_{e}(1.25)}{\sigma_0}=0.8925742\ldots\) which lead to the FDA’s ‘implied limits’ (which are wider than the EMA’s).
    However, in the next section all of the above is ignored and the EMA’s \(\theta_s \approx k=0.760\) is given for ABEL.
  • No upper cap of scaling (according to the GL at CVwR 50%) is given.
In the corresponding EPAR we read:

Statistical methods
Analysis of variance (ANOVA) was performed on the ln-transformed Cmax, AUC0–t and AUC0–∞. The fixed effects sequence, subject nested within sequence, period and treatment were used in the ANOVA model to calculate the pharmacokinetic parameters. The test to reference ratio of geometric LSmeans and the corresponding 90% confidence interval based on the ln-transformed Cmax and AUC0–t data were calculated. The parameter Tmax was analyzed using a non-parametric approach.

Criteria for conclusion of bioequivalence:
Bioequivalence was concluded if the test to reference ratio of geometric LSmeans and the corresponding 90% confidence interval for the Cmax and AUC0–t fell within the acceptance limits of 80.00 to 125.00%. Widening of the acceptance criteria for Cmax was proposed for conclusion of bioequivalence. However, since the results of the bioequivalence study for the pharmacokinetic parameter Cmax were within the normal acceptance criteria of 80–125% (as shown below) widening of the criteria for Cmax was not necessary.

The 90% confidence interval of the test/reference ratio (difference in least square means) derived from the ANOVA of the log-transformed pharmacokinetic parameters AUC0–t and Cmax for rasagiline in plasma was within the 80.00% – 125.00% acceptance range. Therefore the test formulation (███████) is judged to be bioequivalent to the reference product (███████) in healthy adult volunteers under fasting conditions.


“The fixed effects sequence, subject nested within sequence, period and treatment were used in the ANOVA model…”
Cough: Not an ANOVA with all effects fixed but a mixed-effects model with restricted maximum likelihood estimation, no nested structure.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-31 13:40
(1518 d 01:24 ago)

@ Helmut
Posting: # 21117
Views: 19,287
 

 Flawed evaluation accepted

Dear Helmut,

I would like to stress out again that this decision tree has been used in multiple accepted BE studies. The tree is used not only by Indian CROs but CROs from developed countries. Cannot say more because it is a bit confidential. And again no complaints from regulators at all; otherwise, it would not be used

Regards


Why is it flawed?
They passed bioequivalence with the first step and did not go to the second one. It may be risky according to your approach but they were lucky enough. There is no TIE inflation in their study as I understand.
What is wrong in relation to TIE inflation?

Best regards


Edit: Merged with a later (now deleted) post. You can edit your posts for 24 hours. [Helmut]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-31 15:17
(1517 d 23:47 ago)

@ Mikalai
Posting: # 21120
Views: 19,529
 

 Flawed evaluation accepted

Hi Mikalai,

❝ I would like to stress out again that this decision tree has been used in multiple accepted BE studies. The tree is used not only by Indian CROs but CROs from developed countries. […] And again no complaints from regulators at all; otherwise, it would not be used


Sad, so sad! (Mr Trump)

❝ Why is it flawed?


It is not even wrong. (Wolfgang Pauli)

❝ They passed bioequivalence with the first step and did not go to the second one.


By luck because the CV was lower than assumed (30% instead of 32%) and there were substantially fewer dropouts than anticipated.

❝ It may be risky according to your approach but they were lucky enough. There is no TIE inflation in their study as I understand.

❝ What is wrong in relation to TIE inflation?


One should never design a study relying on luck!
If with this approach (assessing ABE with a 90% CI first) one proceeds to ABEL because ABE failed, the patient’s risk is compromised and there is no bloody way to control the Type I Error.
If regulators don’t give a shit about the patient’s risk, we should.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-01-31 17:41
(1517 d 21:23 ago)

(edited by Mikalai on 2020-01-31 18:26)
@ Helmut
Posting: # 21122
Views: 19,123
 

 Flawed evaluation accepted

Dear Helmut,

It appears quite interesting. I again disagree with you.

Your approach is very questionable.

The sponsor risked and won. We are discussing a specific case. The decision tree may have been questionable, but the questionable part was never implemented. Now, let's imagine that regulators reject the study. But what reason? The reason would be that potentially, in another reality, the results could have been non-equivalent or TIE would be unacceptably high. Also one should take into consideration that at present there are no guidelines or other formal documents that prohibit the usage of this decision tree or require to control TIE and calculate the RR for the ref product first. The local NCA and ethics approved the study. In other words, the study should have been rejected because of an imaginary situation that never materialized in reality. But this approach can more or less be applied to any BE study. It is non-sense and not legally defendable. The only result would be money of taxpayers paid to the sponsor for delayed registration if this would take place in the developed world. Everything that may be a reason to reject studies should explicitly be stated in regulatory documents.

Best regards
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-31 21:28
(1517 d 17:37 ago)

@ Mikalai
Posting: # 21124
Views: 19,495
 

 Flawed approach even if accepted ?

Hi Mikalai,

❝ It appears quite interesting. I again disagree with you.


Why am I not surprised? ;-)

Seems that your line of argument is a financial one (though veiled behind words like taxpayer’s money). I’m interested in the patient’s risk – possibly more than some regulators.

I asked myself whether all is lost when ABE fails. I wouldn’t bet on a lucky punch.
We know that TOST is not a uniformly most powerful test (“… the TOST procedure can be quite conservative, i.e., the achieved Type I error rate may be lower than the nominal 0.05 significance level.”)1 That means for nasty combinations of PE and CV with a given sample size the actual TIE might be <0.05. This opens a back door to recalculate the CI but with α = 1 – TIEact) and assess the study for ABEL whilst keeping the overall TIE controlled.
10,000 simulated studies, PE 0.85–0.95, CV 0.25–0.60, 2×2×4 design, powered (≥80%) for ABEL:

[image]

Studies which passed ABE (90% CI, CVwR not calculated).
Studies which failed ABE, CVWR >30%, and the wider CI passed ABEL.
Studies which failed (stopped in ABE because CVwR ≤30% or in ABEL).

Nasty are the close misses in ABE. There is practically no α left and the study will likely fail ABEL as well because the new CI would be extremely wide.
Example: CV 52.92%, PE 0.9118, n 26. With a 90% CI of 77.52–107.25% fails ABE and the actual TIE is 0.04809 (α for ABEL 0.00191). We continue to ABEL with the maximum expansion of 69.84–143.19%. The 99.618% CI is 68.15–121.98%, which fails ABEL as well.
With the common approach we have a 81.2% chance of passing and with 0.0382 the TIE is easily controlled.

Now for the money-wise argument: 45.9% passed ABE and only 7.7% ABEL (i.e., overall power 53.6%); that’s hardly better than tossing a coin.

I checked a couple of other studies in the meantime. In all of them calculation of CVwR was the first step. In some the CVwR was <30% and the study assessed by ABE. I found two cases where CVwR was >30% and studies passed ABEL and the 90% was within 80.00–125.00%. The report correctly stated that ABEL (with expanded limits) was demonstrated and noted that conventional ABE would have passed as well. The latter was just a remark and not part of the procedure.

[image]Given all that, I consider the approach we discussed here as crap and IMHO, it should not be used for the following reasons:
  • [image]One has to hope [sic] that the study passes ABE. If the study is powered for ABEL, chance to demonstrate ABE <50%.
  • If the possibility of a second evaluation is foreseen, Bon­ferroni’s 95% CI requires an increases of the sample size by at least 30%.
  • If ABE fails with the 90% CI, CVwR >30%, and ABEL is assessed (a second time by the 90% CI), massive inflation of the TIE.
  • If a wider CI is possible for ABEL, extremely low power.
It amazes me that nobody (on both the CRO/industry and regulatory side) seemingly bothers to read the relevant papers. In all it is clear that for ABEL one has to first assess whether the drug / drug product is highly variable.2–5 BTW, a coauthor of the last one is a member of the EMA’s PK Working Party.

EOD from my side unless we concentrate on science rather than profits.


  1. Jones B, Kenward MG. Design and Analysis of Cross-Over Trials. Boca Raton: Chapman & Hall/CRC; 3rd ed. 2014. p. 371.
  2. Boddy AW, Snikeris FC, Kringle RO, Wei GCG, Oppermann JA, Midha KK. An Approach for Widening the Bioequivalence Acceptance Limits in the Case of Highly Variable Drugs. Pharm Res. 1995; 12(12): 1865–8. doi:10.1023/A:1016219317744.
  3. Tóthfalusi L, Endrényi L, Midha KK. Scaling or wider bioequivalence limits for highly variable drugs and for the special case of Cmax. Int J Clin Pharmacol Ther. 2003; 41(5): 217–25. doi:10.5414/cpp41217.
  4. Endrényi L, Tóthfalusi L. Regulatory and Study Conditions for the Determination of Bioequivalence of Highly Variable Drugs. J Pharm Pharmaceut Sci. 2009; 12(1):138–49. doi:10.18433/j3zw2c.
  5. Tothfálusi L, Endrényi L, García Arieta A. Evaluation of Bioequivalence for Highly Variable Drugs with Scaled Average Bioequivalence. Clin Pharmaco­kinet. 2009; 48(11): 725–43. doi:10.2165/11318040-000000000-00000.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-02-01 17:18
(1516 d 21:46 ago)

(edited by Mikalai on 2020-02-01 17:33)
@ Helmut
Posting: # 21125
Views: 18,712
 

 Flawed approach even if accepted ?

Dear Helmut

❝ Seems that your line of argument is a financial one (though veiled behind words like taxpayer’s money). I’m interested in the patient’s risk – possibly more than some regulators.


I am not fighting for money but for democracy, the Republic...

Your approach is nothing more than a road to hell lined with good intentions.

Basically you advocate rejecting trials because of things that are neither legally prohibited nor have happened but are/were (when trials done) risky from one scientific angle. I guess this practice hardly only one that is so contentious in the industry. Your approach basically gives experts the power to arbitrally decide which studies to accept and which do not. It would end in mayhem. In Belarus, we often see this, and it usually ends that we have to prove that the globe is round not flat.

If TIE must be controlled in ABEL/RABE, it should be ingrained in regulatory documents. Full stop.

If somebody decided to risk and won (passing bioequivalence on ABE terms), these studies should be accepted. Otherwise, you would reject valid studies and would dictate how sponsors should spend their own money and how different countries have to vet BE trials. I am very skeptical that this decision tree can be outlawed (it is diffuclt to prosecute for things that did not happen), but legal experts may have other opinions.

What is about ABEL/RABE trials when the CV is around 30% or uncertain but not small? Should we pump up the sample size in this case (I guess a majority of cases) to the level close to ABE trials which basically invalidates the whole concept of ABEL/RABE?

Finally, I am not a statistician but wether your approach itself also might inflate TIE when we have to calculate RR (population parameter) and then CI which we compare with a standard or widened bioequivalence interval. We do two calculations in which alpha levels are involved.

Best regards
mittyri
★★  

Russia,
2020-02-01 22:34
(1516 d 16:30 ago)

@ Mikalai
Posting: # 21127
Views: 18,667
 

 misunderstanding

Hi Mikalai,

❝ I am not fighting for money but for democracy, the Republic...


Republic of what?

❝ Your approach is nothing more than a road to hell lined with good intentions.


I think the reason of that words is the misunderstanding of arguments given above...

❝ Basically you advocate rejecting trials because of things that are neither legally prohibited nor have happened but are/were (when trials done) risky from one scientific angle.


There's no science in p value = 0.05. Or could you do me a favour and give some link where the contrary arguments exist? That's just a convention (given in GLs too). Rules of game. If you go there, please use it.

❝ Your approach basically gives experts the power to arbitrally decide which studies to accept and which do not.


:confused: where is it written? just to recalculate CIs using TIE correction gives the power? Is TIE some special value from expert mind? How do you think it is assessed?

❝ In Belarus, we often see this, and it usually ends that we have to prove that the globe is round not flat.


Could you please give some example? From the comments on the forum I can conlude that experts in Belarus are rather progressive.

❝ If TIE must be controlled in ABEL/RABE, it should be ingrained in regulatory documents. Full stop.


It should be controlled everywhere, not only in ABEL

❝ If somebody decided to risk and won (passing bioequivalence on ABE terms), these studies should be accepted.


Please define 'win'. You didn't win until null hypothesis(es) is rejected.

Otherwise, you would reject valid studies and would dictate how sponsors should spend their own money and how different countries have to vet BE trials.


Oh my! pity Sponsors!:-D

I propose not to spend money at all. Just go with pharmaceutical assessment, no BE study. no cent for useless trials. Is that approach reasonable enough?

❝ I am very skeptical that this decision tree can be outlawed


Are the arguments regarding TIE inflation not clear? WE are living in the real world where the precedence does not rule everywhere. And the experts also err. Maybe they didn't realize here the problem of TIE inflation. Remember that the problem is rather new.

❝ What is about ABEL/RABE trials when the CV is around 30% or uncertain but not small?


what are the benefits of replicate design for that CV value (30%)?

❝ Should we pump up the sample size in this case (I guess a majority of cases) to the level close to ABE trials which basically invalidates the whole concept of ABEL/RABE?


Have you read Helmut's calculations for the sample sizes?

❝ Finally, I am not a statistician


I think it is a good reason not to read the end of that paragraph, sorry...

Kind regards,
Mittyri
Mikalai
★    

Belarus,
2020-02-06 14:41
(1512 d 00:23 ago)

(edited by Mikalai on 2020-02-06 16:18)
@ mittyri
Posting: # 21153
Views: 17,493
 

 misunderstanding

Hi Mitturi,
My remarks

❝ Republic of what?


This is from the Star Wars.

❝ I think the reason of that words is the misunderstanding of arguments given above...


I think there is no misunderstanding. I never said that in ABEL/RABE trials TIE should not be controlled. What I worry is the control TIE may invalidate the whole method, because in a big number studies the reduction in sample size can be negligible but risks can double for subjects. I also afraid that this can be assesd only post-hoc. Do we have a decisicion tree for ethic committees?

❝ There's no science in p value = 0.05. Or could you do me a favour and give some link where the contrary arguments exist? That's just a convention (given in GLs too). Rules of game. If you go there, please use it.


It probably deserves a separate topic. Who set up rules of the game in this case? Moreover TIE as consumer protection metric is very controversial. Have we asked people about this convention? Do we have good feedback mechanisms with consumers? Do we incorporate qulitative studies in our regulatory documetns. How can we represent consumers if we do not speak with them? It seems that people tolerate much higher risks than we use currently. Look at alternative medicine. They have millions customers and don't bother with any statistics.

❝ It should be controlled everywhere, not only in ABEL


It is Utopia, and good luck with this. See my comments above. But I can give another argument.

At the heart of bioequivalence lie the concept that products are different. But what does this mean. Is the fact that they are manufactured by different companies is enough to make this conclusion? Have manufacturing processes of different manufactures have ever been compared. And what about TIE deflating factors like industrial statisticians, industrial spies, angry sacked employees from original producers, the same equipment and the same supppliers. Might be that the TIE inflation risk is a bit overestimated This is why if something is widely used and is a reason to rejection, it should be mentioned in regulatory documents.

❝ Please define 'win'. You didn't win until null hypothesis(es) is rejected.


If somebody shot all alpha level and passed the bioequivalence at the first stage.

❝ Are the arguments regarding TIE inflation not clear? WE are living in the real world where »the precedence does not rule everywhere. And the experts also err. Maybe they didn't »realize here the problem of TIE inflation. Remember that the problem is rather new.


It seems that the issue is still not clear for most regualtors.It also seems that this issue is afloat for more 7 years. Also, where are results of the inflated TIE? Where are unsafe and ineffecient drugs.

❝ what are the benefits of replicate design for that CV value (30%)?


We never know our final CV. See my comments below.

❝ Have you read Helmut's calucations for the sample sizes?

See my comments below

❝ I think it is a good reason not to read the end of that paragraph, sorry...


There are no stupid questions there are either not good answers or ignorance. Sorry ...
mittyri
★★  

Russia,
2020-02-06 17:23
(1511 d 21:41 ago)

@ Mikalai
Posting: # 21154
Views: 17,274
 

 misunderstanding

hi Mikalai,

❝ Hi Mitturi,

       ^^^^
is my nick so difficult to copy and paste?

❝ ❝ Republic of what?

❝ This is from the Star Wars.


looks like rewording, cannot recall that

❝ I think there is no misunderstanding. I never said that in ABEL/RABE trials TIE should not be controlled. What I worry is the control TIE may invalidate the whole method, because in a big number studies the reduction in sample size can be negligible but risks can double for subjects.


I cannot find the arguments for doubling risks for subjects. What risks are higher?

❝ I also afraid that this can be assesd only post-hoc.


really? and what about the limits? do you know them before the study?

❝ Do we have a decisicion tree for ethic committees?


see Helmut's lectures for schemes

❝ Who set up rules of games in this case? Moreover TIE as consumer protection metric is very controversial. Have we asked people about this convention? Do we have good feedback mechanisms with consumers? Do we incorporate qulitative studies in our regulatory documetns. How can we represent consumers if we do not speak with them?


what answer do you want to get? "Yes, we are interested in TIE control"
"No, 5% is too much for us, gimme 6%"

❝ It seems that people tolerate much higher risks than we use currently. Look at alternative medicine. They have millions customers and don't bother with any statistics.


some people does. Believe or not but the people trust decisions of regulators.
Placebos exist on the market, is that bad? Sometimes yes, sometimes not.

❝ ❝ It should be controlled everywhere, not only in ABEL

❝ It is Utopia, and good luck with this. See my comments above. But I can give another argument.


so you blame ICH docs? :-D

❝ At the heart of bioequivalence lie the concept that products are different.


you are playing with words. different but still have comparable action

❝ Have manufacturing processes of different manufactures have ever been compared.


how to assess the differences in overall and how are they inflate the bioequivalence? If you have an answer - go with an initiative to EMA, they will be happy to hear it I think.

❝ And what about TIE deflating factors like industrial statisticians, industrial spies, angry sacked employees from original producers, the same equipment and the same supppliers.


sorry, who is industrial statistician? :confused:
if you have any proposals regarding making the products more comparable - go to the initiative group and propose your solutions. I see only remarks. I don't understand why do you want to start a wholywar here.

❝ Might be that the TIE inflation risk is a bit overestimated


a bit? what will trigger you to say it is not overestimated? There will always be sceptics/conservators, irrespective of initiatives. Especially on the slippery zone of risk/benefit of generic drugs.

❝ This is why if something is widely used and is a reason to rejection, it should be mentioned in regulatory documents.


EMA and FDA mention 5%. Have you read the lectures? If we want to keep that value at the level of 5%, we should keep in mind TIE inflation. If you have another opinion - stop reading that

❝ ❝ Please define 'win'. You didn't win until null hypothesis(es) is rejected.


❝ If somebody shoot all alpha level and pass bioequivalence at the first stage.


aha, stage! OK, so you are playing in the 2-stage game? Did you take into account the corrected alphas there? Do you remember 2-stage designs and corrected alphas irrespective of the results of 1st stage?

❝ It seems that the issue is still not clear for most regulators. It also seems that this issue is afloat for more 7 years. Also, where are results of the inflated TIE? Where are unsafe and inefficient drugs.


where are the results of inefficient drugs started when Cmax had limits 75-133? If it is bad, why did the regulators change it? why that drugs are still on the market?

❝ There are no stupid questions there are either not good answers or ignorance. Sorry ...


at least I did not asked something stupid! excellent! I am not surprised that you are not happy with my answers since even detailed Helmut's answers were rejected by you.
For sure you can ignore TIE inflation for yourself until some request will knock your door. It is up to you to prepare yourself or just keep trying to neglect it

Kind regards,
Mittyri
d_labes
★★★

Berlin, Germany,
2020-02-05 20:16
(1512 d 18:48 ago)

@ Mikalai
Posting: # 21150
Views: 17,537
 

 The globe is flat!

Dear Mikalai

❝ ... In Belarus, we often see this, and it usually ends that we have to prove that the globe is round not flat.


Read the books of Terry Prattchets "Discworld" series and you will discover that the world is a flat disc resting on four elephants which are standing on a very big tortoise :cool:.
I recommend especially the titles
- The Science of Discworld (1999)
- The Science of Discworld II: The Globe (2002)
- The Science of Discworld III: Darwin's Watch (2005)
- The Science of Discworld IV: Judgement Day (2013)
They are full of wisdom and may furnish you to be reluctant in fighting for some ones alleged "truth" in science, may it be the own "truth" or the truth of others.

Regards,

Detlew
wienui
★    

Germany/Oman,
2020-01-30 19:53
(1518 d 19:11 ago)

@ Helmut
Posting: # 21111
Views: 19,558
 

 Tricky…

Hello Helmut,

❝ For the Gulf Cooperation Council (Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, United Arab Emirates) and South Africa you can use ABE for Cmax with fixed (!) limits of 75.00–133.33% if CVwR <30%.


I think there is a mistake here, you are right according to the fixed limit of 75-133% in the GCC Guidelines in the case of HVD (HVDP) which is absolutely not correct and cause a lot of problems in the evaluation of such drug formulation's types, BUT widen to this range is not in the case of ABE and not at all if CVwR <30% .

As according to GCC GL (citation down) a replicate design and prove of CVwR >30% ( not<) is required to wide only the Cmax.

"Those HVDP for which a wider difference in Cmax is considered clinically irrelevant based on a sound clinical justification can be assessed with a widened acceptance range. If this is the case, a wider acceptance range (i.e. 75-133%) for Cmax can be used. For the acceptance interval to be widened the bioequivalence study must be of a replicate design where it has been demonstrated that the within subject variability for Cmax of the reference compound in the study is >30%."


Best regards,

Cheers,
Osama
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-30 20:18
(1518 d 18:46 ago)

@ wienui
Posting: # 21112
Views: 19,697
 

 Tricky…

Salam Osama,

❝ ❝ […] with fixed (!) limits of 75.00–133.33% if CVwR <30%.


❝ I think there is a mistake here, you are right according to the fixed limit of 75-133% in the GCC Guidelines in the case of HVD (HVDP) which is absolutely not correct and cause a lot of problems in the evaluation of such drug formulation's types, BUT widen to this range is not in the case of ABE and not at all if CVwR <30% .


Of course, of course! Sorry, it was a stupid typing error. :angry:

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
wienui
★    

Germany/Oman,
2020-02-03 08:10
(1515 d 06:55 ago)

(edited by wienui on 2020-02-03 17:16)
@ Helmut
Posting: # 21129
Views: 18,333
 

 Tricky…

Dear Helmut,

❝ Of course, of course! Sorry, it was a stupid typing error. :angry:


We all make such silly typing error.

I would like only here really to express my sincere gratitude and thanks for the great excellent effort which you did and do. As a previous academic person who worked also in the industry and now in the regulatory, I still till the moment (I think better to say we still) learning a lot from you and other colleagues.

For all of yours, I wish and hope the continuously for this Forum to illuminate our problems in the dark BE world.

Best regards,

Cheers,
Osama
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-02-03 13:25
(1515 d 01:40 ago)

@ wienui
Posting: # 21130
Views: 18,288
 

 ABE vs. ABEL

Dear Osama,

I think that the GCC’s guideline is not that bad (following what the EMA’s Q&A of July 2006 required).
During the development of the EMA’s GL (and even in the June 2010 EGA/EMA joint workshop when the GL was already final) the industry lamented that for CVwR 30–40% one would need larger sample sizes than before. I kept my mouth shut because for years I was an advocate of reference-scaling. Although I was aware of the 2009 paper of the two Lászlós, it needed Detlew’s post of 2013 to comprehend the ugly problem of the inflated Type I Error.
Example: 2×2×4 design, assumed T/R ratio 0.90, powered for ≥80%; in ABE at CVwR 30% switching the BE limits from 80.00–125.00% to 75.00–133.33%.

[image]

For extreme CVs (say, 300% for some LALAs like mesalazine) sample sizes are nasty anyway (430 subject for the fixed wider limits and 224 for ABEL)…
I designed such studies (reference-scaling for the EMA and the FDA) and the problem is less the CV but the uncertain T/R ratio driving the sample size. If the T/R ratio drops from 0.90 to 0.85, the sample size doubles to 448.

❝ I would like only here really to express my sincere gratitude and thanks for the great excellent effort which you did and do. As a previous academic person who worked also in the industry and now in one of the regulatory body I still till the moment (I think better to say we still) learning a lot from you and other colleagues.


THX for the flowers!
BTW, you are not the only regulator here. We have members of different European agencies, the Eurasian Economic Union, ASEAN states, ANVISA, and – believe it or not – the FDA. :-D

❝ For all of yours, I wish and hope the continuously for this Forum to illuminate our problems in the dark BE world.


You should have typeset “dark” also in bold. ;-)

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2020-02-05 19:53
(1512 d 19:12 ago)

@ Helmut
Posting: # 21149
Views: 17,485
 

 zigzag

Dear Helmut,

❝ ...

❝ Example: 2×2×4 design, assumed T/R ratio 0.90, powered for ≥80%; in ABE at CVwR 30% switching the BE limits from 80.00–125.00% to 75.00–133.33%.

[image]

...


What are these saw teeth pattern after CV=0.4 in the bottom picture arising from :confused:?

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-02-05 20:46
(1512 d 18:19 ago)

@ d_labes
Posting: # 21151
Views: 17,695
 

 zigzag

Dear Detlew,

❝ What are these saw teeth pattern after CV=0.4 in the bottom picture arising from :confused:?


Who the heck wrote this package? Magnification:

[image]

CV (%) n.ABE pwr.ABE TIE.ABE n.ABEL pwr.ABEL      U TIE.ABEL
  40.0    30  82.293    0.05     30   80.656 1.3402 0.059119
  40.2    30  81.982    0.05     30   80.789 1.3420 0.058830
  40.4    30  81.671    0.05     30   80.928 1.3438 0.058529
  40.6    30  81.360    0.05     30   81.052 1.3456 0.058155
  40.8    30  81.048    0.05     30   81.216 1.3475 0.057810
  41.0    30  80.737    0.05     30   81.345 1.3493 0.057426
  41.2    30  80.425    0.05     30   81.442 1.3511 0.057055
  41.4    30  80.114    0.05     30   81.566 1.3529 0.056674
  41.6    32  82.054    0.05     30   81.697 1.3548 0.055961
  41.8    32  81.755    0.05     30   81.817 1.3566 0.055543
  42.0    32  81.457    0.05     30   81.934 1.3584 0.055135
  42.2    32  81.158    0.05     30   82.032 1.3603 0.054688
  42.4    32  80.858    0.05     30   82.140 1.3621 0.054249
  42.6    32  80.559    0.05     28   80.013 1.3639 0.053814
  42.8    32  80.260    0.05     28   80.125 1.3657 0.053351
  43.0    34  82.071    0.05     28   80.235 1.3676 0.051870
  43.2    34  81.783    0.05     28   80.342 1.3694 0.051397
  43.4    34  81.496    0.05     28   80.446 1.3712 0.050891
  43.6    34  81.208    0.05     28   80.548 1.3731 0.050351
  43.8    34  80.920    0.05     28   80.624 1.3749 0.049815
  44.0    34  80.633    0.05     28   80.735 1.3767 0.049276
  44.2    34  80.345    0.05     28   80.838 1.3786 0.048778
  44.4    34  80.057    0.05     28   80.901 1.3804 0.048260
  44.6    36  81.766    0.05     28   80.970 1.3822 0.046397
  44.8    36  81.489    0.05     28   81.047 1.3841 0.045844
  45.0    36  81.211    0.05     28   81.116 1.3859 0.045311

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-02-06 12:38
(1512 d 02:27 ago)

@ Helmut
Posting: # 21152
Views: 17,361
 

 zigzag

Ok, guys let discuss a bit further. I am gonna describe two bad scenarios.

The first one, we assume that the CV of a drug is around 35%, so we are ready for the worst inflation TIE in our ABEL trial with 4-periods. Thus, we decided to increase our sample size to 42 subjects to compensate for the adjusted TIE and then added the 15% dropout rate that resulted in 49 subjects in our full sample size. We have done our trial. We have got the CV around 32% and big inflation TIE, but we passed because of safety measures.


The second is a bit worse. We assume that the CV of a drug is around 40-42%. We decided to recruit 40 subjects - with full compensation - to conduct a 4 periods ABEL trial. Then have suddenly got the CV 32% and massive inflation TIE. We failed the trail because of the lack of power. We did the second trial with 49 subjects and passed.


There are a couple of questions.
Are these scenarios are plausible?
How frequent can they be?
Could three-periods trials reduce the number of subjects?


In terms of the flat world and also as response to the remark from mittury. In Belarus, if you decided to conduct a crossover ABE trial in multiple groups and decided to drop the group factor and its interaction from the statistical model (3rd method of FDA) then you have to prove the groups are from the same population. You should statistically compare the demographics of groups. And you can only drop the group factor if there are no statistical differences between groups irrespectively of the fact that subjects have been randomized to the group before the first period of the first group. But how it is possible to have different populations if people have been recruited from the same places (Minsk and its surroundings for example) and according to the same inclusion and exclusion criteria. It is not like one group is from Belarus and the other is from China or Australia. Again, the design is the crossover.

Also, mittury, you should take into account that not all sections of the Euroasian guideline are applied in Belarus at this time. It is left at the discretion of experts. One example. If they decide that a drug is risky (I do not know how they do this), then a sponsor should measure analytes - if the FDA says that - irrespectively of what is written in the guideline and the fact that the parent drug can be perfectly measured. No analytes and only parent drug, no registration. They do not disclose what other sections can be applied differently.

Best regards
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-02-06 21:12
(1511 d 17:53 ago)

@ Mikalai
Posting: # 21155
Views: 17,343
 

 helter-skelter

Hi Mikalai,

❝ I am gonna describe two bad scenarios.


❝ The first one, we assume that the CV of a drug is around 35%, so we are ready for the worst inflation TIE in our ABEL trial with 4-periods. Thus, we decided to increase our sample size to 42 subjects to compensate for the adjusted TIE and then added the 15% dropout rate that resulted in 49 subjects in our full sample size. We have done our trial. We have got the CV around 32% and big inflation TIE, but we passed because of safety measures.


I cannot reproduce your numbers. Given. Though you could state in the SAP which adjusted α you expect (and that you increased the sample size accordingly), you have to state as well that you will adjust α based on the outcome of the study (observed CVwR, actual sample size).
The TIE depends not only on CVwR but – to a minor extent – the sample size as well.

  CV    alpha  n      TIE
0.35 0.050000 34 0.065566
θ0 0.90, 80% power, naïve ABEL: inflated TIE
0.35 0.036100 38 0.050000 ← larger sample size for adjusted α
0.35 0.036100 44 0.050145 ← compensate for 15% dropouts, inflated TIE!
0.35 0.035972 44 0.050000 ← requires more adjustment

If the CVwR is higher than expected, open a bottle of Champagne; more scaling, less adjustment necessary and you gain power. You are right that you have to adjust more if the CVwR moves towards 30% but the loss in power in this example (35% → 32%) is negligible.

  CV    alpha  n do (%)      TIE alpha.ad TIE.ad   power
0.32 0.035972 44   0.00 0.071249 0.033167   0.05 0.83263
0.32 0.035972 43   2.63 0.070963 0.033393   0.05 0.82458
0.32 0.035972 42   5.26 0.070994 0.033460   0.05 0.81875
0.32 0.035972 41   7.89 0.071328 0.033307   0.05 0.80961
0.32 0.035972 40  10.53 0.071046 0.033356   0.05 0.80196
0.32 0.035972 39  13.16 0.070948 0.033430   0.05 0.79346
0.32 0.035972 38  15.79 0.071003 0.033342   0.05 0.78409


58 lines of R-code upon request.

❝ The second is a bit worse. We assume that the CV of a drug is around 40-42%. We decided to recruit 40 subjects - with full compensation - to conduct a 4 periods ABEL trial. Then have suddenly got the CV 32% and massive inflation TIE. We failed the trail because of the lack of power.


Shit happens.

library(PowerTOST)
CV.exp <- 0.40
n.adj  <- sampleN.scABEL.ad(CV = CV.exp, design="2x2x4",
                            print = FALSE)[["Sample size"]]
n.act  <- seq(40, n.adj, -1)
CV.act <- 0.32
res    <- data.frame(n = n.act, alpha = NA, power = NA)
for (j in seq_along(n.act)) {
  x           <- scABEL.ad(CV = CV.act, design="2x2x4",
                           n = n.act[j], theta0 = 0.90,
                           print = FALSE)
  res[j, 2:3] <- as.numeric(unlist(x)[c(15, 17)])
}
print(res, row.names = FALSE)


 n    alpha   power
40 0.033355 0.80195
39 0.033430 0.79346
38 0.033342 0.78409
37 0.033587 0.77602
36 0.033399 0.76411
35 0.033224 0.75303
34 0.033527 0.74509
33 0.033422 0.73438
32 0.033414 0.72151


❝ We did the second trial with 49 subjects and passed.


No dropouts this time? Risky; in such a case I would power the second study more.

❝ There are a couple of questions.

❝ Are these scenarios are plausible?


The first one is common, though in my studies I try to educate my clients that this “increase the sample size based on anticipated dropout rate” is a waste of money. Try the function pa.scABE() of PowerTOST to see why.

❝ How frequent can they be?


No idea about the second one. I know just one case where the sponsor doubled the sample size…

❝ Could three-periods trials reduce the number of subjects?


On the contrary, my dear Dr Watson! The power of study depends on the number of treatments. Compared to a 2×2×2 you get similar power with ~½ of the sample size in a 4-period and ~¾ of the sample size in a 3-period replicate or, if you prefer, you need ~50% more subjects in a 3-period than in a 4-period replicate.

ABE
  CV 2x2x2 2x2x4 2x2x3  4/2  3/2
0.30    40    20    30 0.50 0.75
0.35    52    26    38 0.50 0.73
0.40    66    34    50 0.52 0.76
0.45    82    42    62 0.51 0.76
0.50    98    50    74 0.51 0.76
0.55   116    58    86 0.50 0.74
0.60   134    68   100 0.51 0.75
ABEL
  CV 2x2x4 2x2x3  3/4
0.30    34    50 1.47
0.35    34    50 1.47
0.40    30    46 1.53
0.45    28    42 1.50
0.50    28    42 1.50
0.55    30    44 1.47
0.60    32    48 1.50
ABEL (adjusted)
  CV 2x2x4 2x2x3  3/4
0.30    42    66 1.57
0.35    38    58 1.53
0.40    32    50 1.56
0.45    28    42 1.50
0.50    28    42 1.50
0.55    30    44 1.47
0.60    32    48 1.50


Study costs are similar (since mainly driven by bioanalytics). However, the more periods, the more likely subjects drop out. One the other hand, the 3-period replicate requires more adjustment than the 4-period replicate.

% sample size penalty of adjusted ABEL
  CV 2x2x4 2x2x3
0.30 23.53 32.00
0.35 11.76 16.00
0.40  6.67  8.70
0.45   –     – 
0.50   –     – 
0.55   –     – 
0.60   –     – 


❝ In Belarus, if you decided to conduct a crossover ABE trial in multiple groups and decided to drop the group factor and its interaction from the statistical model (3rd method of FDA) then you have to prove …


We cannot prove anything in science. Proofs belong to the realms of logic and mathematics.

❝ … the groups are from the same population. You should statistically compare the demographics of groups. And you can only drop the group factor if there are no statistical differences between groups …


That’s an extremely stupid approach. At which level will you test? 0.05, 0.10? What about false positives? BTW, any pre-test inflates the TIE and a post hoc test is nonsense.* We discussed that ad nauseam! Explore those.
Don’t dive into such muddy waters. Include the group term in the model (i.e., specify the FDA’s Model II taking into account that groups were tested on different dates) but without a pre-test. The loss in power is very, very low compared to the pooled Model III. See the end of Example 1.


  • Writing a paper about that is on my todo-list since May 2017… Some ideas in this article.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Mikalai
★    

Belarus,
2020-02-10 17:10
(1507 d 21:55 ago)

(edited by Mikalai on 2020-02-11 10:21)
@ Helmut
Posting: # 21157
Views: 16,332
 

 helter-skelter

Dear Helmut
There are some additional questions

❝ I cannot reproduce your numbers. Given.


My calculation was that I assumed that CV is 34 but we braced ourselves for the worst scenario. Am I correct that the worst scenario is around 42 subjects to compensate for TIE at CV 30% and 7(15%) subjects to compensate for a dropout rate?

PS
Total sample size 49

I assume that the TIE calculation is done after we have got our results from the sample. Is this calculation is straight forward with no other statistical approaches that may bring different results?

I guess that the TIE calculation and compensation for TIE correction were realized in PowerTost. What about validation issues? Any feedback from regulators on software?

How should we proceed if CV from our sample is 30.00%?

Should we calculate TIE if CV is less than 30% (I guess no)?

Has anyone submitted an ABEL trial with TIE correction and has got no objections from European regulators or FDA (Is this applicable for FDA?).

❝ Include the group term in the model (i.e., specify the FDA’s Model II taking into account that groups were tested on different dates) but without a pre-test. The loss in power is very, very low compared to the pooled Model III.


Can the use of Model 2 (FDA), basically group and its interactions, be a reason to reject study in Europe? Or it is a more conservative model and if we pass with this model we always pass with the standard ABE model?

Best regards
zizou
★    

Plzeň, Czech Republic,
2020-02-01 18:00
(1516 d 21:05 ago)

@ Helmut
Posting: # 21126
Views: 18,708
 

 Inflation of the TIE as well

Hello everybody and nobody.

❝ you are modifying the common decision scheme. Why?

❝ Since you are making essentially the same decisions though in a different order, you gain nothing.


Exactly...

❝ Well, that’s the problem with ABEL (and the FDA’s RSABE as well). There is no Null Hypothesis.


and exactly again!

In standard ABE, null hypothesis (bioinequivalence) and alternative hypothesis (bioequivalence) are known. The BE is defined using the acceptance limits 80-125% for 90% CI of GMR.
The TIE (Type I Error) is the probability (alpha) of rejection of true null hypothesis by test. (It's clear but problematic when null hypothesis is unknown and even more problematic in mentioned different order - see below.)

TIE inflation for ABEL/RSABE with widening the acceptance limits (because of floating acceptance limits) is described in different topics many times.

The suggested different order of analysis in ABEL/RSABE is only a bad attempt to hide the increased patient's risk.

When we apply the standard limits of 80-125% in the first step and conclude BE using 90% CI - end of analysis ... hey, it looks like standard study with TIE <=5%.
But it is only the part of the decision tree.

It reminded me the Monty Hall problem with intuitive 50–50 chance. When we look only at a part of the game, it seems really as 50-50.

Back to the bioequivalence:
Can we say after the conclusion of BE in the first step that there was no option to wide the acceptance limits?
If yes - Ok.
If no - There was additional probability to conclude BE. TIE inflation is obvious when you look at the whole plan.
The simulations would certainly confirm TIE inflation. Keep in mind that the observed GMR and intra-subject CV of reference are only estimates, we don't know the true (population) values. Imagine the border case example with true GMR 80% (and true intra-subject CV of R 35%), 5% of studies will pass the acceptance criteria 80-125% in the first step and another few % of studies in the second step (after widening of limits). With true GMR 80%, study was performed and we were lucky in observed values, so we concluded BE in the first step. But we can't say that TIE was <=5%!

This different order is more controversial than the standard order according the guidelines. Null hypothesis is set in the first step of different order. BE is evaluated - concluded or not. When the test failed to conclude bioequivalence, the new null hypothesis will be tested with different definition of BE. BE is evaluated again - concluded or not.
Standard order is clear, you construct the null hypothesis and test it. With different order we can fail to demonstrate BE from the first test, oh no..., so we redefine bioequivalence, i.e. new hypothesis, new test (comparison), we have the second chance.

Btw. When I was starting to write the post I thought that order doesn't change the TIE, but now...
I think, the different order has higher TIE than the standard order.
(Helmut was faster with similar example.)
Example 2x2x4 replicate design with the same sample size n=24 for both orders: assumed GMR 90%, assumed intra-subject CV of R 50% (for higher intra-subject CVs there will be higher probability of two tests in different order).
  • standard order
    • One test with acceptance limits 69.84-143.19% or narrower (it depends on observed value), TIE <5% (due to scaling cap and GMR restriction) (let's denote TIE as TIE1)
  • different order
    • The first test with acceptance limits 80-125%, lower power than in standard order (with the same number of subjets) "partial" TIE 4.8%
      The probability of the end after the first test is only 29% in this example, i.e. 71% probability that the second test will be done.
    • The second test with acceptance limits 69.84-143.19% or narrower (it depends on observed value), TIE <5% (due to scaling cap and GMR restriction) (the same as TIE1)
      In total, aggregate TIE is 4.8% + 0.71*TIE1 (aggregate TIE ?, probably higher then 5%).
It really looks like as the order changes the TIE - the reason is of course in the change of BE definition (i.e. null hypothesis) for the first test.

Moreover the first test is totally irrelevant if assessed as the whole. If the first test is BE (the second test would always confirm that), if not, only the second test would be decisive. So equal result of study you receive by testing only the second test, so I don't see any reason for using the different order - no pros, possible TIE inflation as cons.

Best regards,
zizou
nobody
nothing

2020-02-02 00:30
(1516 d 14:34 ago)

@ zizou
Posting: # 21128
Views: 18,718
 

 Inflation of the TIE as well

❝ Hello everybody and nobody.


Lately more often than not I sit here can't stop shaking my head for hours, sometimes I think it must be Parkinsons'

***sigh***

Kindest regards, nobody
Elena777
☆    

Belarus,
2020-03-10 20:28
(1478 d 18:37 ago)

@ Helmut
Posting: # 21235
Views: 14,097
 

 Inflation of the TIE as well

Dear Helmut, still have some doubts regarding decision scheme.

As per you comment:

❝ you are modifying the common decision scheme. Why?


It seems not only I would like to modify a decision scheme. I saw the following conclusion in the PAR (https://db.cbg-meb.nl/Pars/h116289.pdf).

Conclusion on bioequivalence study
The reference product was found to be highly variable for Cmax with an intra-subject variability greater than 30%: the intra-individual coefficient of variation was 41.62%. Widening of the acceptance intervals for Cmax was allowed but, based on the study results, proved not to be necessary. The 90% confidence intervals calculated for AUC0-t and Cmax are within the bioequivalence acceptance range of 0.80 – 1.25. Based on the submitted bioequivalence study Atorvastatine STADA is considered bioequivalent with Lipitor.


As I understand, they did not use the decision scheme described in the lecture. Otherwise it was obligatory for them to modify CI for Cmax. But they just stated that "widening of the acceptance intervals for Cmax was allowed but, based on the study results, proved not to be necessary".

So, how do you think what decision tree was used in that study?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-03-10 22:06
(1478 d 16:59 ago)

@ Elena777
Posting: # 21236
Views: 14,336
 

 Fishing in the dark

Hi Elena,

❝ So, how do you think what decision tree was used in that study?


No idea. In a PAR we see always the wording of the agency. Without the study protocol & report we simply don’t know what exactly was planned & performed. I have an CDA with the sponsor in place and a friend there owes me a favor. No guarantees.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
nobody
nothing

2020-02-03 16:07
(1514 d 22:58 ago)

@ Elena777
Posting: # 21131
Views: 18,294
 

 Statistical evaluation and BE hypo­theses in full replicate design


Kindest regards, nobody
Astea
★★  

Russia,
2020-04-02 14:41
(1456 d 01:23 ago)

@ nobody
Posting: # 21313
Views: 11,429
 

 TIE, repeat once more please...

Dear Friends!

I'm very sorry for digging this theme once more, but my female logic suffers from trying to get how could the sequence of statements influence the TIE. I suppose the TIE in alternative Elena's approach to be the same as in the standard scheme. Would you be so kind to explain it for one thousand and first time again?

Let A be the statement that CI is in [80,125], B - the statement that CV<30% and C - the statement that CI is [L,U], where L and U are the scaled limits. Then BE statement is
(B∧A)∨(¬B∧C) for standard approach and (A∨(¬A∧(¬B∧C))) for the alternative approach. The non BE statements are (¬A∧B)∨(¬A∧¬C)∨(¬B∧¬C) and (¬A∧B)∨(¬A∧¬C) correspondingly. The only difference is (¬B∧¬C) which is logically equals to (¬A∧¬C) when ¬B is TRUE and zero if ¬B is FALSE.

Testing the data twice is obvious in the alternative sequence, but the second test doesn't suffer from the error on CV_R because the first test A has already done. And in the standard approach we also assess the data twice: first to get CV_R, then to calculate CI. I'm totally lost: my gut feeling says that there should not be difference, TIE is the same in both cases. I read the thread once and once again and still have internal contradictions that I couldn't prove.

Practically: while dealing with scaled approach I program to get CI as well as CV_R in the output at the same time. What I should do - close the left eye not to see that CI is in the [80-125] and only looking for the CV_R by my right eye?

"Being in minority, even a minority of one, did not make you mad"
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,638 registered users;
80 visitors (0 registered, 80 guests [including 8 identified bots]).
Forum time: 15:05 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5