Bioequivalence and Bioavailability Forum • Recipe for ABEL and adjusted α

Helmut
★★★

Vienna, Austria,
2017-03-23 20:10
(2582 d 19:43 ago)

Posting: # 17181
Views: 4,510

Recipe for ABEL and adjusted α [RSABE / ABEL]

Dear all,

since we published our paper¹ about the Type I Error, I received questions about its application, regulatory acceptability, etc. In the following I summarize my answers.

Q Is the method accepted by regulatory agencies?
A I don’t know. Since the paper was published quite recently I don’t think that – even if in the meantime studies were performed accordingly – any made it through regulatory assessment so far.
Q Is there a reason why an agency should not accept the method?
A Not at all! Even if an agency (following the BSWP’s opinion of “We don’t like simulations!”) believes [sic] that the entire approach is nonsense, this is no sufficient reason to reject it. The adjusted α is always smaller than the nominal α 0.05. In other words, it means that you care more about the patient’s risk than they do (the method is more conservative than the one recommended in the guideline). You can’t be punished for being cautious (i.e., using a wider CI). Nobody can reasonably argue against an α <0.05. Remember that in the beginning of BE we routinely used a 95% CI and not a 90% CI!
Q How to deal with the sample size penalty?
A First of all one has to increase the sample size (in order to maintain the desired power) only in the critical region around CV_wR 30%. The sample size penalty drops quickly and vanishes for “true” HVD(P)s (say CV_wR >40%) completely (see Fig. 3b of the paper).
Example: RTRT|TRTR, CV_wR 35%, assumed T/R-ratio 0.90, target power 80% which calls for a study in 34 subjects (81.2% power). With the nominal α 0.05 the Type I Error would be 0.0656. If one evaluates the study with the adjusted α 0.03630 (92.74% CI) power will drop to 77.3%. Up to the sponsor to decide whether this is still acceptable. Of course one could increase the sample size to 38 (81.0% power). Note that in this case another (slightly smaller) adjusted α is calculated.
Q Should I specify the adjusted α in the protocol?
A No. The actual study will require an adjusted α which likely is different from what was planned (different CV_wR, smaller sample size due to dropouts, unbalanced sequences). If the CV_wR turns out to be closer to 30% than expected and one will evaluate the study with what was stated in the protocol, the TIE would be inflated. On the other hand if the CV_wR would be more far away from the critical region, one would have to adjust less and therefore, gain power. To quote the conclusions of the paper:
It should be sufficient for regulatory acceptance to unambiguously specify the method in the study protocol, which was shown to be more conservative than the current recommendations in any case.
Hence, one should state in the protocol what was assumed (CV_wR, n), expected based on it (adjusted α), and that the actual adjusted α will be derived from the study’s data. It will not hurt to make clear that the adjusted α will be ≤0.05.
Q I do not understand how the underlying statistical distributions are used in the simulations. Is that correct at all?
A Good point. A similar question was raised by one of the reviewers – and it took us two months* to compare the method introduced by Zheng et al.² with simulations of subject data. For the results see the Supplementary material 1 and 2. In short: The agreement is fine for the full replicate designs (RTRT|TRTR and RTR|TRT) and sufficient for the partial replicate (RRT|RTR|TRR). If you don’t trust in it, PowerTOST contains since v1.4-4 the new function power.scABEL.sdsims(). A comparison:
power.scABEL(CV=0.3, design="2x2x4", n=34, theta0=1.25, nsims=1e6) # [1] 0.081626 power.scABEL.sdsims(CV=0.3, design="2x2x4", n=34, theta0=1.25, nsims=1e6) # [1] 0.081602
You have to endure runtimes which are 50–300× slower than the ones of power.scABEL(). On GitHub I updated the functions scABEL.ad() and sampleN.scABEL.ad() accordingly. Expect them on CRAN in April.
Q The functions in PowerTOST cover balanced and unbalanced studies. How to deal with incomplete data (e.g., data set I of the Q&A)?
A That’s a valid point. First of all we could only use the slow subject data simulations. We would have to eliminate missing periods from the simulated data sets as in the study. There are some problems: How to unambiguously (read: foolproof) specify in the function calls which data are missing? The degrees of freedom have to be modified which would require a major re-write of the code. Doable but cumbersome. We are thinking about; don’t expect anything in the near future.

Answers to questions not asked:

If the study was performed in a full replicate design you could (or better: should) take the additional information about CV_wT into account. Example: RTRT|TRTR, CV_wR = CV_wT 35%, T/R-ratio 0.90, 34 subjects. Adjusted α 0.03630 (post hoc power 77.3%). Now: CV_wR 35%, CV_wT 30%. Adjusted α 0.03512 (post hoc power 81.4%). Although you adjust more (due to the lower CV_wT the chance of passing BE increases and hence, the TIE) power increases as well – despite a wider CI.
The partial replicate design, again. The ‘crippled model’ (© Jiří Hofmann) recommended by the EMA in the Q&A prevents the disaster we sometimes observe with the FDA’s over-specified mixed-effects model for ABE (i.e., if σ_wR <0.294). However, assuming equal variances of T and R might be (and likely is) simply wrong! Essentially we are fishing in the dark. Hence, all simulations should be taken with a grain of salt. If ever possible, avoid this design.

@Detlew: Something wrong or did I forget anything?

Labes D, Schütz H. Inflation of Type I Error in the Evaluation of Scaled Average Bioequivalence, and a Method for its Control. Pharm Res. 2016;33(11):2805–14. doi:10.1007/s11095-016-2006-1. free full-text view-only.
Zheng C, Wang J, Zhao L. Testing bioequivalence for multiple formulations with power and sample size calculations. Pharm Stat. 2012;11(4):334–41. doi:10.1002/pst.1522

Up to eight simultaneous R-sessions on two machines running 24/7. Detlew and Ben did a hell of a job to tweak the code. Simulations of 34 subjects RTRT|TRTR took originally more than two hours to complete. Now we are down at 30 seconds!

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2017-03-27 13:28
(2579 d 03:25 ago)

@ Helmut
Posting: # 17187
Views: 3,247

Recipe for ABEL and adjusted α

Post reply

Dear Helmut,

❝ @Detlew: Something wrong or did I forget anything?

Totally Ok!

For installing PowerTOST development versions from GitHub see this thread. After installing Hadley Wickham's package devtools a simple call

devtools::install_github("Detlew/PowerTOST")

is only necessary. Contrary to what was written there it is not necessary to have RTools installed as I just have checked after deinstalling mine Rtools.

—
Regards,

Detlew

d_labes
★★★

Berlin, Germany,
2017-05-23 12:25
(2522 d 04:28 ago)

@ Helmut
Posting: # 17391
Views: 3,047

PowerTOST V1.4-5 - scABEL functions

Post reply

Dear All!

The (official) version on CRAN, promised by Helmut for April, is out now.

Beside the functions scABEL.ad() and sampleN.scABEL.ad() now allowing simulations via subject data there is another goody:
power.scABEL.sdsims() has an argument design_dta which you may use to evaluate designs with missing data. specify here a data.frame with columns subject, sequence, period and tmt without the rows were missings occured. Optional specify an additional column logval (log-transformed PK metric) with NA indicating the missings.
This feature is experimental! Only use it if you know what you do.

Example:

# full data partial replicate design (CVwT==CVwR), n=36 (12 in each sequence group)

# power at theta0=0.9 

power.scABEL.sdsims(CV=0.3, n=36)

[1] 0.67794

# TIE

power.scABEL.sdsims(CV=0.3, n=36, theta0=1.25, nsims=1e6)

[1] 0.071743



# create the data.frame (misuse a hidden function in PowerTOST)

dta <- PowerTOST:::prep_data2(seqs=c("TRR", "RTR", "RRT"), nseq=c(12, 12, 12), muR=log(10), 

                              ldiff=log(0.9), s2wT=CV2mse(0.3), s2wR=CV2mse(0.3))

# missings for subject 1 and 2 in period 3, sequence group TRR

dta$logval[dta$subject==1 & dta$period==3] <- NA

dta$logval[dta$subject==2 & dta$period==3] <- NA

# power at theta0=0.9 

power.scABEL.sdsims(CV=0.3, design_dta=dta)

[1] 0.67038

# TIE

power.scABEL.sdsims(CV=0.3, design_dta=dta, theta0=1.25, nsims=1e6)

[1] 0.072705



# Evaluation of subjects with full data (excl. subject 1,2)

dta0 <- dta[!(dta$subject %in% c(1,2)), ]


# power at theta0=0.9 

power.scABEL.sdsims(CV=0.3, design_dta=dta0)

[1] 0.65273

# or equivalently

power.scABEL.sdsims(CV=0.3, n=c(10, 12, 12))

[1] 0.65273

# TIE

power.scABEL.sdsims(CV=0.3, design_dta=dta0, theta0=1.25, nsims=1e6)

[1] 0.071462

# or equivalently

power.scABEL.sdsims(CV=0.3, n=c(10, 12, 12), theta0=1.25, nsims=1e6)

[1] 0.071462

—
Regards,

Detlew