statistically significant ≠ clinically relevant [General Statistics]
Hi Siva Krishna,
I guess 100% was not contained in the 90% CI, right?
If yes, you have a statistically significant difference which is clinically not relevant.1 We abandoned testing for a statistically significant difference (see ElMaestro’s post) 33 (‼) years ago with Schuirmann’s TOST.2 To quote Wasserstein et al.3
For which power did you plan the study? It might well be that
See also the second part of this post. If you are in the lower right quadrants, you have high power and a statistically significant treatment effect is likely.
See also this article.
I guess 100% was not contained in the 90% CI, right?
If yes, you have a statistically significant difference which is clinically not relevant.1 We abandoned testing for a statistically significant difference (see ElMaestro’s post) 33 (‼) years ago with Schuirmann’s TOST.2 To quote Wasserstein et al.3
Don’t Say “Statistically Significant”
For which power did you plan the study? It might well be that
- the T/R-ratio was closer to 100% than assumed and/or
- the CV was lower than assumed and/or
- the dropout-rate was lower than anticipated.
See also the second part of this post. If you are in the lower right quadrants, you have high power and a statistically significant treatment effect is likely.
See also this article.
- \(\small{\Delta}\) = clinically relevant difference. Commonly 0.20 (20%). For NTIDs (EMA and other jurisdicions) \(\small{\Delta}\) 0.10, for Cmax (Russian Federation, EEU, GCC) \(\small{\Delta}\) 0.25. For HVD(P)s, where CVwR >30%, \(\small{\Delta}\) >0.30 (scaled to the variability of the reference). The acceptance range for bioequivalence \(\small{\left \{\theta_1,\theta_2\right \}}\) is calculated by \(\small{\theta_1=1-\Delta}\), \(\small{\theta_2=(1-\Delta)^{-1}}\). If the 90% CI lies entirely within \(\small{\left \{\theta_1,\theta_2\right \}}\), the observed difference of the treatment effect is considered clinically not relevant – irrespective how wide the CI is or where the point estimate lies. A formulation with a PE of 100% (CI 80.00–125.00%) is as BE as another with a PE of 85% (CI 80.00–90.31%). In the former case you were extremely lucky and in the second you have a statistically significant difference (100% not contained in the CI).
- Schuirmann DJ. A comparison of the Two One-Sided Tests Procedure and the Power Approach for Assessing the Equivalence of Average Bioavailability. J Pharmacokin Biopharm. 1987; 15(6): 657–80. doi:10.1007/BF01068419.
- Wasserstein RL, Schirm AL, Lazar NA. Moving to a World Beyond “p < 0.05”. Am Stat. 2019; 73(sup1): 1–19. doi:10.1080/00031305.2019.1583913. Open access.
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- Treatment effect justification Sivakrishna 2020-10-07 08:15 [General Statistics]
- Treatment effect justification ElMaestro 2020-10-07 10:19
- statistically significant ≠ clinically relevantHelmut 2020-10-07 11:07
- statistically significant ≠ clinically relevant Sivakrishna 2020-10-09 10:24
- Problems with low variability Helmut 2020-10-09 15:04
- statistically significant ≠ clinically relevant Sivakrishna 2020-10-09 10:24