EU: simulation-based methods in agony [Two-Stage / GS Designs]
Hi ElMaestro & all,
an update: Yesterday at the 4th Biosimilars Forum a participant asked whether simulation-based methods (control of the type I error in a sufficiently high number of simulations, i.e., 1 mio in each cell of a narrow grid of n1/CV combinations) are acceptable. Andreas Brandt (BfArM and observer at the BSWP) answered “No.” Was also the opinion of Stephan Lehr (Austrian agency AGES). Later Andreas said that such methods might be acceptable if no alternative which shows analytically control of the TIE is available. That means for crossover designs all simulation-based methods are essentially dead – go with Ref.#3 of this post (which is implemented in
Then I summarized my experiences in three scientific advices about parallel designs. Tricky cause (a) exact methods don’t exist and (b) simulations have to cover a wide range of unequal group sizes and unequal variances. My simulations (‘Type 1’ TSD) covered nG1=nG2=124–250 (step size 2), CV 0.24–1.0 (step size 0.02), T/R-ratio 0.90, target power 80% = 1 mio sim’s in each of the 2,496 cells. On top of that extreme scenarios with heteroscedasticity (CV-ratios 1:4 to 4:1), each for equal and unequal group sizes (increasing dropout-rates up to ~50% in one group). Overall ~2.82 billion (!) simulations. With an adjusted α 0.0274 the maximum TIE was 0.04987. In the ‘scientific’ advices regulatory statisticians told me that it is not acceptable and claimed that exact methods exist. I asked them for publications but never received an answer.
Was also the opinion of Andreas, Stephan, and Júlia Singer. Sorry folks, mixed up non-inferiority (where repeated confidence intervals are available indeed) with equivalence (nada). Júlia promised to send me one. IMHO, would be a big surprise.* Then Andreas meant – smiling – that if nothing is published, there might still exist ones (absence of evidence is not evidence of absence). Splendid, very helpful.
A mathematician is a blind man
in a dark room looking for a black cat
which isn’t there. attributed to Charles Darwin
an update: Yesterday at the 4th Biosimilars Forum a participant asked whether simulation-based methods (control of the type I error in a sufficiently high number of simulations, i.e., 1 mio in each cell of a narrow grid of n1/CV combinations) are acceptable. Andreas Brandt (BfArM and observer at the BSWP) answered “No.” Was also the opinion of Stephan Lehr (Austrian agency AGES). Later Andreas said that such methods might be acceptable if no alternative which shows analytically control of the TIE is available. That means for crossover designs all simulation-based methods are essentially dead – go with Ref.#3 of this post (which is implemented in
Power2Stage
) instead.Then I summarized my experiences in three scientific advices about parallel designs. Tricky cause (a) exact methods don’t exist and (b) simulations have to cover a wide range of unequal group sizes and unequal variances. My simulations (‘Type 1’ TSD) covered nG1=nG2=124–250 (step size 2), CV 0.24–1.0 (step size 0.02), T/R-ratio 0.90, target power 80% = 1 mio sim’s in each of the 2,496 cells. On top of that extreme scenarios with heteroscedasticity (CV-ratios 1:4 to 4:1), each for equal and unequal group sizes (increasing dropout-rates up to ~50% in one group). Overall ~2.82 billion (!) simulations. With an adjusted α 0.0274 the maximum TIE was 0.04987. In the ‘scientific’ advices regulatory statisticians told me that it is not acceptable and claimed that exact methods exist. I asked them for publications but never received an answer.
Was also the opinion of Andreas, Stephan, and Júlia Singer. Sorry folks, mixed up non-inferiority (where repeated confidence intervals are available indeed) with equivalence (nada). Júlia promised to send me one. IMHO, would be a big surprise.* Then Andreas meant – smiling – that if nothing is published, there might still exist ones (absence of evidence is not evidence of absence). Splendid, very helpful.
A mathematician is a blind man
in a dark room looking for a black cat
which isn’t there. attributed to Charles Darwin
- She sent me the papers of Anders1 and Zheng et al.2 Both preserve the TIE – based on simulations.
Only the former is for parallel designs. The latter is for asymmetric alphas (0.01, 0.04) in crossovers.
- Fuglsang A. Sequential Bioequivalence Approaches for Parallel Designs. AAPS J. 2014;16(3):373–8.
doi:10.1208/s12248-014-9571-1. Free Full text.
- Zheng C, Zhao L, Wang J. Modifications of sequential designs in bioequivalence trials. Pharm Stat. 2015;14(3):180–8. doi:10.1002/pst.1672.
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- EU, method C preference ElMaestro 2019-10-11 10:25 [Two-Stage / GS Designs]
- On the contrary, my dear Dr. Watson! Helmut 2019-10-11 11:52
- On the contrary, my dear Dr. Watson! nobody 2019-10-11 17:39
- On the contrary, my dear Dr. Watson! Helmut 2019-10-11 17:47
- On the contrary, my dear Dr. Watson! ElMaestro 2019-10-11 18:01
- On the contrary, my dear Dr. Watson! nobody 2019-10-11 18:08
- On the contrary, my dear Dr. Watson! Helmut 2019-10-11 18:11
- On the contrary, my dear Dr. Watson! ElMaestro 2019-10-11 18:01
- On the contrary, my dear Dr. Watson! Helmut 2019-10-11 17:47
- On the contrary, my dear Dr. Watson! nobody 2019-10-11 17:39
- EU: simulation-based methods in agonyHelmut 2019-10-19 12:38
- EU: ABEL TIE?? mittyri 2019-10-19 23:24
- Reference-scaling: only simulations possible Helmut 2019-10-20 13:58
- EU: ABEL TIE?? mittyri 2019-10-19 23:24
- On the contrary, my dear Dr. Watson! Helmut 2019-10-11 11:52