Adjusting α [RSABE / ABEL]
Hi Pjs,
extending the end of my post. No R-code yet because it requires the development-version of
As you can see, power is compromised (
Molin’s approach is extremely conservative. Imagine an observed CVwR of 100%. According to ABEL we will employ the maximum expansion (69.84–143.19%) of the BE limits and the decision will practically lead by the GMR-restriction (80.00–125.00%). But how likely is a true CVwR of 30%? Less than 10–15! The adjusted α (4-period full replicate, n 68) will be 0.02748 and power 0.7161. Borderline.
Will it help to use a less strict CI of CVwR?
Only a little.
extending the end of my post. No R-code yet because it requires the development-version of
PowerTOST
. Essentially we have three options:- Assuming that the observed CVwR is the true one.
- Since the true CVwR is unknown, assume the worst, i.e., always adjust for a CVwR of 30%.
- Calculate a conservative CI of the observed CVwR. If the CI includes 30%, adjust for CVwR=30%. If not adjust for the CL which is closer to 30%.
NA
denotes cases where no adjustment is necessary (since the Type I Error with the nominal α is ≤0.05).Adjustment based on observed CVwR (Labes and Schütz 2016):
reg design n CVwR TIE.nom alpha.adj TIE.adj pwr.des pwr.act
EMA 2x2x4 14 0.175 0.04988 NA NA 0.8007 0.8007
EMA 2x2x4 18 0.200 0.04998 NA NA 0.8007 0.8007
EMA 2x2x4 24 0.225 0.04958 NA NA 0.8229 0.8229
EMA 2x2x4 28 0.250 0.05180 0.04825 0.05 0.8116 0.8069
EMA 2x2x4 32 0.275 0.05980 0.04148 0.05 0.8082 0.7824
EMA 2x2x4 34 0.300 0.08163 0.02857 0.05 0.8028 0.7251
EMA 2x2x4 34 0.325 0.06971 0.03418 0.05 0.8010 0.7492
EMA 2x2x4 34 0.350 0.06557 0.03630 0.05 0.8118 0.7728
EMA 2x2x4 32 0.375 0.06271 0.03782 0.05 0.8110 0.7760
EMA 2x2x4 30 0.400 0.05912 0.04024 0.05 0.8066 0.7800
EMA 2x2x4 30 0.425 0.05451 0.04454 0.05 0.8219 0.8094
EMA 2x2x4 28 0.450 0.04889 NA NA 0.8112 0.8112
EMA 2x2x4 28 0.475 0.04114 NA NA 0.8162 0.8162
EMA 2x2x4 28 0.500 0.03317 NA NA 0.8143 0.8143
EMA 2x2x4 28 0.525 0.03787 NA NA 0.8073 0.8073
EMA 2x2x4 30 0.550 0.04165 NA NA 0.8211 0.8211
EMA 2x2x4 30 0.575 0.04420 NA NA 0.8047 0.8047
EMA 2x2x4 32 0.600 0.04630 NA NA 0.8101 0.8101
EMA 2x2x4 34 0.625 0.04779 NA NA 0.8128 0.8128
EMA 2x2x4 36 0.650 0.04856 NA NA 0.8127 0.8127
'Worst case' adjustment based on CVwR=30% (Molins et al. 2017):
reg design n CVwR CV.adj TIE.nom alpha.adj TIE.adj pwr.des pwr.act
EMA 2x2x4 14 0.175 0.3 0.07904 0.03022 0.05 0.8007 0.4073
EMA 2x2x4 18 0.200 0.3 0.07928 0.02988 0.05 0.8007 0.4942
EMA 2x2x4 24 0.225 0.3 0.08064 0.02927 0.05 0.8229 0.5951
EMA 2x2x4 28 0.250 0.3 0.08057 0.02910 0.05 0.8116 0.6517
EMA 2x2x4 32 0.275 0.3 0.08075 0.02904 0.05 0.8082 0.7055
EMA 2x2x4 34 0.300 0.3 0.08160 0.02864 0.05 0.8028 0.7239
EMA 2x2x4 34 0.325 0.3 0.08160 0.02864 0.05 0.8010 0.7239
EMA 2x2x4 34 0.350 0.3 0.08160 0.02864 0.05 0.8118 0.7239
EMA 2x2x4 32 0.375 0.3 0.08075 0.02904 0.05 0.8110 0.7055
EMA 2x2x4 30 0.400 0.3 0.08123 0.02888 0.05 0.8066 0.6753
EMA 2x2x4 30 0.425 0.3 0.08123 0.02888 0.05 0.8219 0.6753
EMA 2x2x4 28 0.450 0.3 0.08057 0.02910 0.05 0.8112 0.6517
EMA 2x2x4 28 0.475 0.3 0.08057 0.02910 0.05 0.8162 0.6517
EMA 2x2x4 28 0.500 0.3 0.08057 0.02910 0.05 0.8143 0.6517
EMA 2x2x4 28 0.525 0.3 0.08057 0.02910 0.05 0.8073 0.6517
EMA 2x2x4 30 0.550 0.3 0.08123 0.02888 0.05 0.8211 0.6753
EMA 2x2x4 30 0.575 0.3 0.08123 0.02888 0.05 0.8047 0.6753
EMA 2x2x4 32 0.600 0.3 0.08075 0.02904 0.05 0.8101 0.7055
EMA 2x2x4 34 0.625 0.3 0.08160 0.02864 0.05 0.8128 0.7239
EMA 2x2x4 36 0.650 0.3 0.08173 0.02852 0.05 0.8127 0.7454
Conservative adjustment based on 99.9% CI of observed CVwR:
reg design n CVwR lower CL upper CL CV.adj TIE.nom alpha.adj TIE.adj pwr.des pwr.act
EMA 2x2x4 14 0.175 0.1022 0.4536 0.3000 0.07818 0.03072 0.05 0.8007 0.4096
EMA 2x2x4 18 0.200 0.1237 0.4407 0.3000 0.07944 0.03005 0.05 0.8007 0.4935
EMA 2x2x4 24 0.225 0.1475 0.4300 0.3000 0.08040 0.02933 0.05 0.8229 0.5952
EMA 2x2x4 28 0.250 0.1683 0.4503 0.3000 0.08122 0.02886 0.05 0.8116 0.6513
EMA 2x2x4 32 0.275 0.1892 0.4737 0.3000 0.08130 0.02860 0.05 0.8082 0.7018
EMA 2x2x4 34 0.300 0.2082 0.5088 0.3000 0.08163 0.02857 0.05 0.8028 0.7251
EMA 2x2x4 34 0.325 0.2251 0.5547 0.3000 0.08163 0.02857 0.05 0.8010 0.7251
EMA 2x2x4 34 0.350 0.2420 0.6014 0.3000 0.08163 0.02857 0.05 0.8118 0.7251
EMA 2x2x4 32 0.375 0.2560 0.6641 0.3000 0.08130 0.02860 0.05 0.8110 0.7018
EMA 2x2x4 30 0.400 0.2694 0.7334 0.3000 0.08102 0.02888 0.05 0.8066 0.6777
EMA 2x2x4 30 0.425 0.2856 0.7864 0.3000 0.08102 0.02888 0.05 0.8219 0.6777
EMA 2x2x4 28 0.450 0.2979 0.8680 0.3000 0.08122 0.02886 0.05 0.8112 0.6513
EMA 2x2x4 28 0.475 0.3136 0.9263 0.3136 0.07365 0.03238 0.05 0.8162 0.6668
EMA 2x2x4 28 0.500 0.3291 0.9864 0.3291 0.06893 0.03485 0.05 0.8143 0.6814
EMA 2x2x4 28 0.525 0.3446 1.0480 0.3446 0.06639 0.03625 0.05 0.8073 0.6965
EMA 2x2x4 30 0.550 0.3646 1.0730 0.3646 0.06362 0.03748 0.05 0.8211 0.7421
EMA 2x2x4 30 0.575 0.3800 1.1360 0.3800 0.06183 0.03838 0.05 0.8047 0.7580
EMA 2x2x4 32 0.600 0.4000 1.1610 0.4000 0.05905 0.04008 0.05 0.8101 0.8035
EMA 2x2x4 34 0.625 0.4198 1.1890 0.4198 0.05430 0.04416 0.05 0.8128 0.8464
EMA 2x2x4 36 0.650 0.4397 1.2170 0.4397 0.04819 NA NA 0.8127 0.8127
As you can see, power is compromised (
pwr.des
= achieved power in sample size estimation, pwr.act
= actual power if the study is evaluated with the adjusted α). IMHO, power <0.7 is not desirable.Molin’s approach is extremely conservative. Imagine an observed CVwR of 100%. According to ABEL we will employ the maximum expansion (69.84–143.19%) of the BE limits and the decision will practically lead by the GMR-restriction (80.00–125.00%). But how likely is a true CVwR of 30%? Less than 10–15! The adjusted α (4-period full replicate, n 68) will be 0.02748 and power 0.7161. Borderline.
Will it help to use a less strict CI of CVwR?
Conservative adjustment based on 99.0% CI of observed CVwR:
reg design n CVwR lower CL upper CL CV.adj TIE.nom alpha.adj TIE.adj pwr.des pwr.act
EMA 2x2x4 14 0.175 0.1135 0.3535 0.3000 0.07818 0.03072 0.05 0.8007 0.4096
EMA 2x2x4 18 0.200 0.1359 0.3603 0.3000 0.07944 0.03005 0.05 0.8007 0.4935
EMA 2x2x4 24 0.225 0.1604 0.3660 0.3000 0.08040 0.02933 0.05 0.8229 0.5952
EMA 2x2x4 28 0.250 0.1822 0.3895 0.3000 0.08122 0.02886 0.05 0.8116 0.6513
EMA 2x2x4 32 0.275 0.2039 0.4146 0.3000 0.08130 0.02860 0.05 0.8082 0.7018
EMA 2x2x4 34 0.300 0.2240 0.4471 0.3000 0.08163 0.02857 0.05 0.8028 0.7251
EMA 2x2x4 34 0.325 0.2423 0.4863 0.3000 0.08163 0.02857 0.05 0.8010 0.7251
EMA 2x2x4 34 0.350 0.2605 0.5261 0.3000 0.08163 0.02857 0.05 0.8118 0.7251
EMA 2x2x4 32 0.375 0.2763 0.5758 0.3000 0.08130 0.02860 0.05 0.8110 0.7018
EMA 2x2x4 30 0.400 0.2914 0.6292 0.3000 0.08102 0.02888 0.05 0.8066 0.6777
EMA 2x2x4 30 0.425 0.3090 0.6725 0.3090 0.07533 0.03144 0.05 0.8219 0.6875
EMA 2x2x4 28 0.450 0.3231 0.7326 0.3231 0.07043 0.03402 0.05 0.8112 0.6755
EMA 2x2x4 28 0.475 0.3402 0.7787 0.3402 0.06698 0.03590 0.05 0.8162 0.6924
EMA 2x2x4 28 0.500 0.3573 0.8257 0.3573 0.06493 0.03691 0.05 0.8143 0.7083
EMA 2x2x4 28 0.525 0.3742 0.8736 0.3742 0.06322 0.03784 0.05 0.8073 0.7255
EMA 2x2x4 30 0.550 0.3952 0.9005 0.3952 0.05981 0.03969 0.05 0.8211 0.7744
EMA 2x2x4 30 0.575 0.4121 0.9487 0.4121 0.05704 0.04200 0.05 0.8047 0.7946
EMA 2x2x4 32 0.600 0.4330 0.9759 0.4330 0.05217 0.04695 0.05 0.8101 0.8410
EMA 2x2x4 34 0.625 0.4539 1.0040 0.4539 0.04565 NA NA 0.8128 0.8128
EMA 2x2x4 36 0.650 0.4747 1.0330 0.4747 0.03821 NA NA 0.8127 0.8127
Only a little.
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png)
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png)
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- question of adjustment Yura 2017-04-25 15:14 [RSABE / ABEL]
- TIE depends on CVwR (and n) Helmut 2017-04-26 14:17
- TIE depends on CVwR (and n) Yura 2017-04-26 17:28
- TIE depends on CVwR (and n) Helmut 2017-04-26 18:00
- TIE depends on CVwR (and n) Yura 2017-04-26 18:55
- TIE depends on CVwR (and n) Yura 2017-04-28 11:13
- TIE = p(BE) at expanded limits Helmut 2017-04-28 19:16
- TIE = p(BE) at expanded limits Yura 2017-04-29 13:01
- TIE = p(BE) at expanded limits Helmut 2017-04-28 19:16
- TIE depends on CVwR (and n) Yura 2017-04-28 11:13
- TIE depends on CVwR (and n) Yura 2017-04-26 18:55
- TIE depends on CVwR (and n) Helmut 2017-04-26 18:00
- TIE depends on CVwR (and n) pjs 2018-02-28 14:33
- TIE depends on CVwR (and n) Helmut 2018-02-28 14:48
- TIE depends on CVwR (and n) pjs 2018-03-01 07:35
- Comparing methods for (S)ABE Helmut 2018-03-01 13:32
- Comparing methods for (S)ABE pjs 2018-03-05 14:50
- Simulating the Null Helmut 2018-03-05 17:40
- Comparing methods for (S)ABE pjs 2018-03-05 14:50
- Comparing methods for (S)ABE Helmut 2018-03-01 13:32
- TIE depends on CVwR (and n) pjs 2018-03-01 07:35
- Adjusting αHelmut 2018-03-07 16:21
- TIE depends on CVwR (and n) Helmut 2018-02-28 14:48
- TIE depends on CVwR (and n) Yura 2017-04-26 17:28
- TIE depends on CVwR (and n) Helmut 2017-04-26 14:17