Bioequivalence and Bioavailability Forum

Dear Dan!

❝ an assessor realized that the number of AEs differ between the treatment groups and he/she concluded that the tolerability of one product is worse than the other.

Oh no. I got the same question once – many, many years ago.

❝ I would reply that bioavailability in terms of peak and total exposure is taken as a surrogate parameter for efficacy and safety and I would point out that bioequivalence studies are not designed to investigate the safety profile in terms of AEs of one formulation in comparison to another.

Correct.

❝ Due to the small numbers of subjects and small number of AEs any differences become significant but statistically not relevant.

❝ Is this correct or do you have other suggestions?

First I would not accept the idea to simply compare the number of AEs. IMHO only a comparison of ADRs (relationship to IMP rated certainly or probably/likely) makes sense. Or would you include headache, which is known to occur in up to 30 % of subjects due to the setting of the study (less sleep, no breakfast until four hours, no coffee, ). Furthermore I would not pool ADRs (apples with oranges), but evaluate them separately. If done that way I would expect differences between treatments to diminish. Only if you still get significant differences, I would continue with your argument.

Dear Dr_Dan

Sorry to hear this.
If the CIs show BE and the assessor can raise a PSRtPH on basis of count data, then the BE guideline apparently needs to be re-written as the traditional metrics do not assure BE for drug products like yours.
Less clear of course if your CI goes from 1.18-1.24 etc, when the assessor can definitely argue the exposure is higher. But if the CI spans over 1.0 this should -at least in theory- be a piece of cake.

Dear Dr_Dan,

the table you gave is a nice but sparse contingency table.
When traditional chi-square is calculated the resulting p-value is often imprecise if the table is sparse (low numbers and/or zeros being frequent) which is arguably the case for your data.
For this reason these tables are typically evaluated with resampling.


With R:
DrDansTable= as.table(rbind(c(1,3,2,3,0), c(0,2,0,3,1)))
chisq.test(DrDansTable, simulate.p.value = TRUE, B = 100000)

        Pearson's Chi-squared test with simulated p-value (based on 1e+05
        replicates)

data:  DrDansTable
X-squared = 3.75, df = NA, p-value = 0.6612


If you do the table with ordinary calculation of chi square then you end up at p=0.44.

Thus:

"The applicant acknowledges the concern raised by the assessor and has undertaken a further analysis of the occurrence of adverse events for the two treatments in trial XV63-GFR-Q-61 as follows.
The p-value for the contigency table (insert it around here) as evaluated by resampling using 100.000 random replicates is approx. 0.66; for a significant difference to be present the p-values should be 0.05. The assessor's concern is therefore not supported by the numbers presented in the applicant's study report.
Evaluation of the table with ordinary chi-square gives a p-value of 0.44 which likewise is very much above the significance limit. Because the table is sparse, the applicant contends that preference should be given to the evaluation with resampling. It will always converge towards the true p-value with increasing sampling number.
Regardless of the evaluation method, the result does not in any way suggest the presence of a difference, which is in perfect accordance with the outcome of the trial in terms of the primary parameters Cmax and AUCt. Should the two types of outcomes differ in their interpretations, the primary parameters should prevail anyway, as the trial was dimensioned to compare these rather than the secondary parameters.
All in all, the primary trial outcomes as well as further analysis of the AEs observed in the study do not support the notion of the test product having a safety profile that differs from that of the reference product."

Someone should check my calculations, though, as I frequently err.

Dear Dr_Dan!

This is an interesting question but IMHO this is a non-inferiority hypothesis, i.e. the sponsor would like to show that the occurrence of AEs with the test treatment is non-inferior to the reference treatment. However, I think a consensus on a-priori definition of relevance threshold is difficult or hopeless. According to my opinion it’s very similar to statistical problems occurring in non-clinical toxicological studies. Hothorn (2010) suggests estimation of confidence intervals and their post-hoc interpretation in terms of tolerable thresholds when no a-priori definition of relevance thresholds is available.

What do you think by providing a confidence interval plot? The y-axis depicts the different type of AEs (e.g. according to MedDRA classification) and the x-axis shows the difference in proportion between test and reference and the corresponding confidence interval. A similar plot - for statistical analysis of organ weights - can be found in Hothorn (2010, slide 30) for illustration of this idea.

Please let me know what you think about this approach!

Best regards

Martin

Hothorn LA (2010). Statistical analysis of short-term studies in regulatory toxicology using R. Webinar organized by the American Statistician Association, Biopharmaceutical Section on Thursday, May 20.
http://www.biopharmnet.com/doc/doc03002-07.html or http://www.biopharmnet.com/doc/2010_05_20_webinar.pdf

PS.: when you would like to use the McNemar test you can calculate the odds ratio and the corresponding confidence interval instead of differences in proportions using the R function mcnemar.exact of R package exact2x2.