ElMaestro ★★★ Belgium?, 20191011 10:25 Posting: # 20683 Views: 851 

Hi all, I remember having heard EU regulators mention preference for method C out of consideration for the type I error. But I can't seem to find a presentation from anyone saying so. Do you know, do one of you experts have a link or a presentation by a regulator where this was stated? Many thanks. — I could be wrong, but... Best regards, ElMaestro 
Helmut ★★★ Vienna, Austria, 20191011 11:52 @ ElMaestro Posting: # 20684 Views: 813 

Hi ElMaestro, » I remember having heard EU regulators mention preference for method C out of consideration for the type I error. What? Where? » But I can't seem to find a presentation from anyone saying so. Would surprise me if there is any. » Do you […] have a link or a presentation by a regulator where this was stated? Nope. The collaborative work about the type I error was removed from the work plan last year (Paola Coppola’s presentation at BioBridges 2018): No work plans published this year for both parties due to Brexit. However, there is an unequivocal preference towards methods which show analytically strict control of the TIE.^{1,2,3} In my experience European regulatory statisticians hate simulationbased methods. On Wednesday’s workshop I endured a frustrating chat with a statistician of the Austrian agency AGES. Collection of errors and misconceptions:
Yesterday I sent a clarification
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
nobody nothing 20191011 17:39 @ Helmut Posting: # 20685 Views: 717 

Welcome to the wonderful world of alternative facts eeerh... scientific discussion, I meant. Just a matter of days a we start discussions on whether it's raining outside or not. btw. any slide show of this bio19 event for nonparticipants? — Kindest regards, nobody 
Helmut ★★★ Vienna, Austria, 20191011 17:47 @ nobody Posting: # 20686 Views: 721 

Hi nobody, » Just a matter of days a we start discussions on whether it's raining outside or not. Did you need an umbrella afterwards? » btw. any slide show of this bio19 event for nonparticipants? They will, once we get all permissions (almost ready). Archive of last year’s presentations here. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro ★★★ Belgium?, 20191011 18:01 @ Helmut Posting: # 20687 Views: 711 

Hi both, » They will, once we get all permissions (almost ready). Archive of last year’s presentations here. Note Paola Coppola's presentation, slide 32 I had a wtf moment of sorts when she presented that. Suspended does not mean the parties who suspended it do not think it is important, don't let that detail fool you. — I could be wrong, but... Best regards, ElMaestro 
nobody nothing 20191011 18:08 @ ElMaestro Posting: # 20688 Views: 708 

...last year stuff I bingewatched one evening last year — Kindest regards, nobody 
Helmut ★★★ Vienna, Austria, 20191011 18:11 @ ElMaestro Posting: # 20689 Views: 704 

Hi ElMaestro, » Suspended does not mean the parties who suspended it do not think it is important, don't let that detail fool you. Old believes die hard. It was on work plan since 2015. In order to seriously assess the methods they would have to run own simulations which makes them . I don’t expect that we will ever see sumfink. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Helmut ★★★ Vienna, Austria, 20191019 12:38 @ ElMaestro Posting: # 20705 Views: 480 

Hi ElMaestro & all, an update: Yesterday at the 4^{th} Biosimilars Forum a participant asked whether simulationbased methods (control of the type I error in a sufficiently high number of simulations, i.e., 1 mio in each cell of a narrow grid of n_{1}/CV combinations) are acceptable. Andreas Brandt (BfArM and observer at the BSWP) answered “No.” Was also the opinion of Stephan Lehr (Austrian agency AGES). Later Andreas said that such methods might be acceptable if no alternative which shows analytically control of the TIE is available. That means for crossover designs all simulationbased methods are essentially dead – go with Ref.#3 of this post (which is implemented in Power2Stage ) instead.Then I summarized my experiences in three scientific advices about parallel designs. Tricky cause (a) exact methods don’t exist and (b) simulations have to cover a wide range of unequal group sizes and unequal variances. My simulations (‘Type 1’ TSD) covered n_{G1}=n_{G2}=124–250 (step size 2), CV 0.24–1.0 (step size 0.02), T/Rratio 0.90, target power 80% = 1 mio sim’s in each of the 2,496 cells. On top of that extreme scenarios with heteroscedasticity (CVratios 1:4 to 4:1), each for equal and unequal group sizes (increasing dropoutrates up to ~50% in one group). Overall ~2.82 billion (!) simulations. With an adjusted α 0.0274 the max. TIE was 0.04987. In the scientific advices regulatory statisticians told me that it is not acceptable and claimed that exact methods exist. I asked them for publications but never received an answer. Was also the opinion of Andreas, Stephan, and Júlia Singer. Sorry folks, mixed up noninferiority (where repeated confidence intervals are available indeed) with equivalence (nada). Júlia promised to send me one. IMHO, would be a big surprise.* Then Andreas meant – smiling – that if nothing is published, there might still exist ones (absence of evidence is not evidence of absence). Splendid, very helpful. A mathematician is a blind man in a dark room looking for a black cat which isn’t there. attributed to Charles Darwin
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
mittyri ★★ Russia, 20191019 23:24 @ Helmut Posting: # 20707 Views: 431 

Hi Helmut, » an update: Yesterday at the 4^{th} Biosimilars Forum a participant asked whether simulationbased methods (control of the type I error in a sufficiently high number of simulations, i.e., 1 mio in each cell of a narrow grid of n_{1}/CV combinations) are acceptable. Andreas Brandt (BfArM and observer at the BSWP) answered “No.” sad story BTW: in your lecture you suggested to adjust the CIs for ABEL due to TIE inflation. Is that also dead? Does it mean that you cannot prove anything using sims?? — Kind regards, Mittyri 
Helmut ★★★ Vienna, Austria, 20191020 13:58 @ mittyri Posting: # 20708 Views: 418 

Hi mittyri, » sad story “So sad!” (© Mr Trump) » BTW: in your lecture you suggested to adjust the CIs for ABEL due to TIE inflation. Is that also dead? » Does it mean that you cannot prove anything using sims?? I hope not. For all referencescaling methods we need already simulations to estimate the sample size. It would be strange to allow them here but not for the TIE. RSABE/ABEL is tricky anyhow. The model is based on (population) parameters $$\theta_s\leq\tfrac{\mu_T\mu_R}{\sigma_{wR}}\leq+\theta_s$$ which are unknown. We have only their estimates. Exactly this misspecification (apply scaling although the drug is not highly variable) leads to the inflated TIE. As I wrote above, Andreas Brandt said that “[simulationbased] methods might [sic] be acceptable if no alternative which shows analytically control of the TIE is available”. Clearly the case here. In the current implementation referencescaling is a framework with two (RSABE) and three (ABEL) decisions. No way to solve that analytically (given, at least the GMRrestriction could be implemented by setting α 0.5).Our method^{2} follows the ‘spirit’ of the GL, i.e., we assume that \(s_{wR}=\sigma_{wR}\). Molins et al.^{2} proposed to assume the worst, i.e., regardless of \(s_{wR}\) adjust α as if \(CV_{wR}=0.30\). Conservative but it has a substantial negative impact on power (esp. for really high variability where an inflated TIE is unlikely). For examples see the RSABE vignette of the working version of the next release of PowerTOST and Rcode at the end.What is better? Rely on the ad hoc solutions (which are not perfect) or follow the book, ignore the inflation and put the patients in jeopardy?
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 