Helmut ★★★ ![]() ![]() Vienna, Austria, 2016-03-04 16:37 (3324 d 00:10 ago) Posting: # 16051 Views: 12,823 |
|
Dear all, last month I had the displeasure to attend a “scientific” advisory meeting at a Scandivian agency. Background:
The agency’s statistician said (my comments in blue):
The work plan 2016 of the BSWP contains this: Type I error control in two-stage designs in bioequivalence studies
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2016-03-05 00:16 (3323 d 16:30 ago) @ Helmut Posting: # 16052 Views: 10,569 |
|
Hi Hötzi, clearly there must be some kind of misunderstanding. Some of the Spandinavians are not easy to understand as recent linguistic research has proven. I have ok experience with scientific advices at EU agencies; as far as I recall I did not once run into someone telling me that an approach wasn't acceptable, as long as it was backed well up with a simulation to control type I errors. Regulators have for several years now accepted two-stage approaches (see e.g. the Q&A you linked to) and as far as I know Potvin's papers and its descendants are the only ones containing anything that just remotely resembles a proof of control over the type I error. But ok, perhaps regulators have a secret method which is acceptable (as long as there is a period in stage term ![]() ![]() ![]() ❝
Yah I read that one too. I have a copy in the loo, might come in handy if I one day run out of roll. — Pass or fail! ElMaestro |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2016-03-05 16:17 (3323 d 00:29 ago) @ ElMaestro Posting: # 16054 Views: 10,497 |
|
Hi ElMaestro, ❝ Some of the Spandinavians are not easy to understand as recent linguistic research has proven. Interesting! Given these problems did you ever consider to opt for an easier language? Most Irish abandoned “Sweet Gaelic” for the far more simple English. What ’bout ![]() ❝ […] as far as I know Potvin's papers and its descendants are the only ones containing anything that just remotely resembles a proof of control over the type I error. Not even remotely. You cannot plug the decision tree of the frameworks into a formula to estimate power. Hence, a mathematical proof is not possible. The control of the TIE shown over the assessed combos of n1/CV is purely empirical. That’s why hard-core statisticians without further considerations will tell you that these methods are crap. ❝ But ok, perhaps regulators have a secret method which is acceptable (as long as there is a period in stage term So do I. The only one which claims to contain a proof is Kieser & Rauch.* IMHO, the two lines in the article are actually no more than a claim… I asked the agency’s statistician whether she means this paper and she replied “No.” ❝ Is the problem simulation rather than math proof? Yes. As said above this gives statistician hiccups. The ideal situation would be a proof for the type I error. Most statisticians accept simulations only for the type II error. IMHO, a proof for frameworks is not possible. ❝ It isn't my impression that simulations are outright banned or discouraged and some members of the PKWP or BSWP have published enjoyable and useful science based on simulation. Well, the entire reference-scaling stuff (might it be the FDA’s RSABE for HVDs or NTIDS and the EMA’s ABEL) is entirely based on simulations. These methods are frameworks as well. Proof of the control of the TIE impossible. Do these expert statisticians don’t know that or just ignore it? Case 1: Bad. Case 2: Double moral standards. ![]() ❝ […] research funded under EU's 7th frame programme - it was called "Biosim"… Ended March 2010. Different cup of tea. Toys for boys. No confirmatory statistics like in BE.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2016-03-06 23:59 (3321 d 16:48 ago) @ Helmut Posting: # 16060 Views: 10,254 |
|
To whom it may concern: ❝ (...)Yes. As said above this gives statistician hiccups. The ideal situation would be a proof for the type I error. Most statisticians accept simulations only for the type II error.(...) Power is the chance of showing BE for some given model at a given CV and GMR. The type I error is by definition the power when the GMR is chosen to be exactly on the acceptance border (either high or low, there is no difference). Thus power and type I error is for all practical purposes one and the same thing, all that differs is the applied GMR. Why on earth would you trust anyone's results after he/she types 0.95 on a keyboard and presses Enter, but not trust the same person typing 0.8 or 1.25 and hitting Enter? There is a not a single equation or iteration that is not operating according to the same rule set when we simulate for power vs. when we simulate for type I error. The difference is solely in the value of a single variable. And I was always taught that cherry-picking is not an option. Consider two BE plasma samples that are being analysed. They are the input into the exact same process. Whatever comes out of that process has to be trusted and it isn't an option for me to prefer one output over the one as long as the process they were subjected to was the same and valid. If I try cherry-picking I will be subjected to inspection and questioning, and that is really fair enough. If we subject "0.95" and "1.25" (or "0.80") to the exact same process then we do not consider the output arising from one of them to be more valid than the output arising from the other. So can you please tell me that either both results can be trusted or that neither of the two results can? Many, many thanks. I really mean it. — Pass or fail! ElMaestro |
d_labes ★★★ Berlin, Germany, 2016-03-07 12:41 (3321 d 04:05 ago) @ ElMaestro Posting: # 16064 Views: 10,167 |
|
Dear ElMaestro, you are totally right IMHO. The reason why statisticians (some of) are accepting simulations only for type II error are the different roles of power and type I error. Ideal statistical tests meet the rule that type I error is <=0.05 or some other threshold agreed upon. Using simulations to determine this has the drawback that you only can show this for the scenarios you have simulated. Not for the general case. Therefore a proof assuring no alpha-inflation is preferred. Power on the other hand is useful for planning purposes (only). No strict criterion has to be applied to this term in general. — Regards, Detlew |
d_labes ★★★ Berlin, Germany, 2016-03-06 14:31 (3322 d 02:15 ago) @ Helmut Posting: # 16057 Views: 10,184 |
|
Dear Helmut, for me that's not an opinion but rather total ignorance of the scientific work on this field ![]() — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2016-03-06 15:45 (3322 d 01:02 ago) @ d_labes Posting: # 16058 Views: 10,307 |
|
Dear Detlew, I’m sick of some statisticians’ obsession with mathematical proofs. The three-body problem in physics can’t be solved analytically. However, man landed on the moon and the New Horizons space probe reached Pluto. Same goes with the Navier-Stokes equations of fluid dynamics. Aircraft are designed by the finite element method (fortunately resulting in a much, much lower risk* than 0.05). If statisticians are so worried about numeric methods they should stop traveling – except by foot. Biostatistician. One who has neither the intellect for mathematics nor the commitment for medicine but likes to dabble in both. Stephen Senn — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
DavidManteigas ★ Portugal, 2016-06-02 14:57 (3234 d 02:50 ago) @ Helmut Posting: # 16378 Views: 9,081 |
|
❝ Biostatistician. One who has neither the intellect for mathematics ❝ nor the commitment for medicine ❝ but likes to dabble in both. Stephen Senn Nice one Helmut, never heard! I believe that is true for some old statisticians not used to computers and with a strong mathematics background. I'm a junior statistician and my background is in life sciences. Simulation is my favourite tool for everything, both because I'm not as good as that in mathematical theory and because simulation allows us to produce a lot of useful output to learn and prove our points (and it is much more simple to explain that mathematical equations :D). As far as I saw, new statisticians joining the pharmaceutical industry are much more computer science-lovers than mathematics nerds. So it will be a matter of time until Stephen's definition will be outdated :D |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2016-06-02 15:30 (3234 d 02:16 ago) @ DavidManteigas Posting: # 16379 Views: 9,131 |
|
Hi David, welcome to the forum! ❝ Nice one Helmut, never heard! From Stephen’s Statistical Issues in Drug Development, p.457. I fully agree with your point of view. ❝ So it will be a matter of time until Stephen's definition will be outdated I’m not sure if I’ll live long enough. What is the average age of members of the EMA’s Biostatistical Working Party? A member of the PKWP officially asked the BWSP whether a TIE of 0.0501 [sic] is acceptable and the answer was “No!”… — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2016-06-02 16:44 (3234 d 01:03 ago) @ Helmut Posting: # 16383 Views: 9,257 |
|
Hi David and Helmut, ❝ From Stephen’s Statistical Issues in Drug Development, p.457. An other source of similar pieces of wisdom is "The Devil's Drug Development Dictionaries" ![]() Here a taster (one of my favorites): Statistician.
— Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2016-06-02 17:59 (3233 d 23:47 ago) @ d_labes Posting: # 16386 Views: 9,126 |
|
Dear Detlew, Martin told me another one: Stephen was asked by a colleague “How is your wife?” and he replied “Compared to what?” Professional deformation? ![]() — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2016-06-02 18:42 (3233 d 23:04 ago) @ Helmut Posting: # 16388 Views: 9,043 |
|
Dear Helmut, ❝ Stephen was asked by a colleague “How is your wife?” and he replied “Compared to what?” ![]() ❝ Professional deformation? ![]() — Regards, Detlew |