Helmut ★★★ Vienna, Austria, 20160304 15:37 (2062 d 13:09 ago) Posting: # 16051 Views: 10,414 

Dear all, last month I had the displeasure to attend a “scientific” advisory meeting at a Scandivian agency. Background:
The agency’s statistician said (my comments in blue):
The work plan 2016 of the BSWP contains this: Type I error control in twostage designs in bioequivalence studies
— Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
ElMaestro ★★★ Denmark, 20160304 23:16 (2062 d 05:29 ago) @ Helmut Posting: # 16052 Views: 8,654 

Hi Hötzi, clearly there must be some kind of misunderstanding. Some of the Spandinavians are not easy to understand as recent linguistic research has proven. I have ok experience with scientific advices at EU agencies; as far as I recall I did not once run into someone telling me that an approach wasn't acceptable, as long as it was backed well up with a simulation to control type I errors. Regulators have for several years now accepted twostage approaches (see e.g. the Q&A you linked to) and as far as I know Potvin's papers and its descendants are the only ones containing anything that just remotely resembles a proof of control over the type I error. But ok, perhaps regulators have a secret method which is acceptable (as long as there is a period in stage term ) and which they forgot to share with the rest of the world. I can't tell if this is the case, but I somehow doubt it. Is the problem simulation rather than math proof? It isn't my impression that simulations are outright banned or discouraged and some members of the PKWP or BSWP have published enjoyable and useful science based on simulation. Some of them have even been active in research funded under EU's 7th frame programme  it was called "Biosim" and the whole idea of simulation was enthusiastically promoted as being scientifically sound and necessary from many angles and it even enjoyed regulatory support from regulators in AT, NL, ES, DE, FR, NO and possibly UK not sure?!. The countries are just from the top of my head. »
Yah I read that one too. I have a copy in the loo, might come in handy if I one day run out of roll. — Pass or fail! ElMaestro 
Helmut ★★★ Vienna, Austria, 20160305 15:17 (2061 d 13:28 ago) @ ElMaestro Posting: # 16054 Views: 8,531 

Hi ElMaestro, » Some of the Spandinavians are not easy to understand as recent linguistic research has proven. Interesting! Given these problems did you ever consider to opt for an easier language? Most Irish abandoned “Sweet Gaelic” for the far more simple English. What ’bout Gibberish? » […] as far as I know Potvin's papers and its descendants are the only ones containing anything that just remotely resembles a proof of control over the type I error. Not even remotely. You cannot plug the decision tree of the frameworks into a formula to estimate power. Hence, a mathematical proof is not possible. The control of the TIE shown over the assessed combos of n_{1}/CV is purely empirical. That’s why hardcore statisticians without further considerations will tell you that these methods are crap. » But ok, perhaps regulators have a secret method which is acceptable (as long as there is a period in stage term ) and which they forgot to share with the rest of the world. I can't tell if this is the case, but I somehow doubt it. So do I. The only one which claims to contain a proof is Kieser & Rauch.* IMHO, the two lines in the article are actually no more than a claim… I asked the agency’s statistician whether she means this paper and she replied “No.” » Is the problem simulation rather than math proof? Yes. As said above this gives statistician hiccups. The ideal situation would be a proof for the type I error. Most statisticians accept simulations only for the type II error. IMHO, a proof for frameworks is not possible. » It isn't my impression that simulations are outright banned or discouraged and some members of the PKWP or BSWP have published enjoyable and useful science based on simulation. Well, the entire referencescaling stuff (might it be the FDA’s RSABE for HVDs or NTIDS and the EMA’s ABEL) is entirely based on simulations. These methods are frameworks as well. Proof of the control of the TIE impossible. Do these expert statisticians don’t know that or just ignore it? Case 1: Bad. Case 2: Double moral standards. » […] research funded under EU's 7th frame programme  it was called "Biosim"… Ended March 2010. Different cup of tea. Toys for boys. No confirmatory statistics like in BE.
— Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
ElMaestro ★★★ Denmark, 20160306 22:59 (2060 d 05:47 ago) (edited by ElMaestro on 20160306 23:49) @ Helmut Posting: # 16060 Views: 8,292 

To whom it may concern: » (...)Yes. As said above this gives statistician hiccups. The ideal situation would be a proof for the type I error. Most statisticians accept simulations only for the type II error.(...) Power is the chance of showing BE for some given model at a given CV and GMR. The type I error is by definition the power when the GMR is chosen to be exactly on the acceptance border (either high or low, there is no difference). Thus power and type I error is for all practical purposes one and the same thing, all that differs is the applied GMR. Why on earth would you trust anyone's results after he/she types 0.95 on a keyboard and presses Enter, but not trust the same person typing 0.8 or 1.25 and hitting Enter? There is a not a single equation or iteration that is not operating according to the same rule set when we simulate for power vs. when we simulate for type I error. The difference is solely in the value of a single variable. And I was always taught that cherrypicking is not an option. Consider two BE plasma samples that are being analysed. They are the input into the exact same process. Whatever comes out of that process has to be trusted and it isn't an option for me to prefer one output over the one as long as the process they were subjected to was the same and valid. If I try cherrypicking I will be subjected to inspection and questioning, and that is really fair enough. If we subject "0.95" and "1.25" (or "0.80") to the exact same process then we do not consider the output arising from one of them to be more valid than the output arising from the other. So can you please tell me that either both results can be trusted or that neither of the two results can? Many, many thanks. I really mean it. — Pass or fail! ElMaestro 
d_labes ★★★ Berlin, Germany, 20160307 11:41 (2059 d 17:04 ago) @ ElMaestro Posting: # 16064 Views: 8,240 

Dear ElMaestro, you are totally right IMHO. The reason why statisticians (some of) are accepting simulations only for type II error are the different roles of power and type I error. Ideal statistical tests meet the rule that type I error is <=0.05 or some other threshold agreed upon. Using simulations to determine this has the drawback that you only can show this for the scenarios you have simulated. Not for the general case. Therefore a proof assuring no alphainflation is preferred. Power on the other hand is useful for planning purposes (only). No strict criterion has to be applied to this term in general. — Regards, Detlew 
d_labes ★★★ Berlin, Germany, 20160306 13:31 (2060 d 15:14 ago) @ Helmut Posting: # 16057 Views: 8,270 

Dear Helmut, for me that's not an opinion but rather total ignorance of the scientific work on this field . — Regards, Detlew 
Helmut ★★★ Vienna, Austria, 20160306 14:45 (2060 d 14:01 ago) @ d_labes Posting: # 16058 Views: 8,351 

Dear Detlew, I’m sick of some statisticians’ obsession with mathematical proofs. The threebody problem in physics can’t be solved analytically. However, man landed on the moon and the New Horizons space probe reached Pluto. Same goes with the NavierStokes equations of fluid dynamics. Aircraft are designed by the finite element method (fortunately resulting in a much, much lower risk* than 0.05). If statisticians are so worried about numeric methods they should stop traveling – except by foot. Biostatistician. One who has neither the intellect for mathematics nor the commitment for medicine but likes to dabble in both. Stephen Senn — Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
DavidManteigas ★ Portugal, 20160602 12:57 (1972 d 16:49 ago) @ Helmut Posting: # 16378 Views: 7,162 

» Biostatistician. One who has neither the intellect for mathematics » nor the commitment for medicine » but likes to dabble in both. Stephen Senn Nice one Helmut, never heard! I believe that is true for some old statisticians not used to computers and with a strong mathematics background. I'm a junior statistician and my background is in life sciences. Simulation is my favourite tool for everything, both because I'm not as good as that in mathematical theory and because simulation allows us to produce a lot of useful output to learn and prove our points (and it is much more simple to explain that mathematical equations :D). As far as I saw, new statisticians joining the pharmaceutical industry are much more computer sciencelovers than mathematics nerds. So it will be a matter of time until Stephen's definition will be outdated :D 
Helmut ★★★ Vienna, Austria, 20160602 13:30 (1972 d 16:15 ago) @ DavidManteigas Posting: # 16379 Views: 7,179 

Hi David, welcome to the forum! » Nice one Helmut, never heard! From Stephen’s Statistical Issues in Drug Development, p.457. I fully agree with your point of view. » So it will be a matter of time until Stephen's definition will be outdated I’m not sure if I’ll live long enough. What is the average age of members of the EMA’s Biostatistical Working Party? A member of the PKWP officially asked the BWSP whether a TIE of 0.0501 [sic] is acceptable and the answer was “No!”… — Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
d_labes ★★★ Berlin, Germany, 20160602 14:44 (1972 d 15:02 ago) @ Helmut Posting: # 16383 Views: 7,242 

Hi David and Helmut, » From Stephen’s Statistical Issues in Drug Development, p.457. An other source of similar pieces of wisdom is "The Devil's Drug Development Dictionaries" . Here a taster (one of my favorites): Statistician.
— Regards, Detlew 
Helmut ★★★ Vienna, Austria, 20160602 15:59 (1972 d 13:46 ago) @ d_labes Posting: # 16386 Views: 7,190 

Dear Detlew, Martin told me another one: Stephen was asked by a colleague “How is your wife?” and he replied “Compared to what?” Professional deformation? — Diftor heh smusma 🖖 Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes 
d_labes ★★★ Berlin, Germany, 20160602 16:42 (1972 d 13:03 ago) @ Helmut Posting: # 16388 Views: 7,125 

Dear Helmut, » Stephen was asked by a colleague “How is your wife?” and he replied “Compared to what?” » Professional deformation? Missing Null/Alternative hypo — Regards, Detlew 