Wonnemann’s Figure 4 on steroids [RSABE / ABEL]
To whom it may concern,
Figure 4 (both parts) is misleading. Let’s concentrate on the lower part. The sample sizes (× Ntotal) are for an expected GMR of 1 and 80% power (see Table II below). These sample sizes were taken from the paper of the two Lászlós. Their table is a little bit unfortunate because sometimes sample sizes are imbalanced (e.g., 25 mod 3 ≠ 0). The filled circles give the chance to pass BE with a GMR of 1.25, which – as Alfredo correctly noted – is irrelevant if scaling is applied. Therefore, the labeling of the y-axis isnot unfortunate. If the Null is modified, the relevant value for the GMR would be one of he scaled limits.
Example for the partial replicate design GMR 1, ≥80% power (105 simulations for the sample size and 106 sim’s for the type I error):
![[image]](img/uploaded/image287.png)
![[image]](img/uploaded/image288.png)
Maximum inflation of the TIE 0.0684 (at CV 30.1%). If one designs the study for higher power, the TIE would further increase. In studies with ≥90% power the maximum TIE would be 0.0697.
As Alfredo also noted (and mentioned by the Lászlós as well) we could expect higher inflation of the TIE if the GMR moves away from 1. This time 0.9 (recommended for HVDs):
![[image]](img/uploaded/image289.png)
![[image]](img/uploaded/image290.png)
Maximum inflation of the TIE 0.0716 (or 0.0727 if designed for ≥90% power).
Generally slightly higher inflation is seen in fully replicated designs. Overview:
In the last case we are already close to the TIE of 1–(1–0.05)²=0.0975 expected for two simultaneous tests at α 0.05.
@Alfredo: Since the inflation of the TIE not only depends on the observed CVwR but also – though to a minor degree – on the sample size (dropouts…) it is difficult to counteract the inflation by a pre-specified adjusted α. I’m not sure yet whether even Bonferroni’s 0.025 could maintain the TIE at 5% in all cases.* However, generally less adjustment will be required (especially for a higher CVwR). “Iteratively adjusted α” is a procedure to find a suitable value (given the study’s CVwR and n) which would keep the patient’s risk for the modified Null (i.e., at the scaled limits) at 5%. Only the procedure could be stated in the protocol. Hence, since the CI will be wider than the GL’s 90% CI I think that – as a conservative approach – it should be acceptable for regulators.
❝ Ehhm, do I get this completely wrong or does Fig.4, lower half in Wonnemann, 2014 tell a different story? Just asking...
Figure 4 (both parts) is misleading. Let’s concentrate on the lower part. The sample sizes (× Ntotal) are for an expected GMR of 1 and 80% power (see Table II below). These sample sizes were taken from the paper of the two Lászlós. Their table is a little bit unfortunate because sometimes sample sizes are imbalanced (e.g., 25 mod 3 ≠ 0). The filled circles give the chance to pass BE with a GMR of 1.25, which – as Alfredo correctly noted – is irrelevant if scaling is applied. Therefore, the labeling of the y-axis is
Example for the partial replicate design GMR 1, ≥80% power (105 simulations for the sample size and 106 sim’s for the type I error):
![[image]](img/uploaded/image287.png)
![[image]](img/uploaded/image288.png)
Maximum inflation of the TIE 0.0684 (at CV 30.1%). If one designs the study for higher power, the TIE would further increase. In studies with ≥90% power the maximum TIE would be 0.0697.
As Alfredo also noted (and mentioned by the Lászlós as well) we could expect higher inflation of the TIE if the GMR moves away from 1. This time 0.9 (recommended for HVDs):
![[image]](img/uploaded/image289.png)
![[image]](img/uploaded/image290.png)
Maximum inflation of the TIE 0.0716 (or 0.0727 if designed for ≥90% power).
Generally slightly higher inflation is seen in fully replicated designs. Overview:
design GMR % power TIEmax
2×3×3 1.0 80 0.0684
0.9 80 0.0716
1.0 90 0.0697
0.9 90 0.0727
2×2×4 1.0 80 0.0782
0.9 80 0.0817
1.0 90 0.0795
0.9 90 0.0824
2×2×3 1.0 80 0.0835
0.9 80 0.0878
1.0 90 0.0845
0.9 90 0.0891
2×3×3: TRR|RRT|RTR, 2×2×4: TRTR|RTRT, 2×2×3: TRT|RTR.
In the last case we are already close to the TIE of 1–(1–0.05)²=0.0975 expected for two simultaneous tests at α 0.05.
@Alfredo: Since the inflation of the TIE not only depends on the observed CVwR but also – though to a minor degree – on the sample size (dropouts…) it is difficult to counteract the inflation by a pre-specified adjusted α. I’m not sure yet whether even Bonferroni’s 0.025 could maintain the TIE at 5% in all cases.* However, generally less adjustment will be required (especially for a higher CVwR). “Iteratively adjusted α” is a procedure to find a suitable value (given the study’s CVwR and n) which would keep the patient’s risk for the modified Null (i.e., at the scaled limits) at 5%. Only the procedure could be stated in the protocol. Hence, since the CI will be wider than the GL’s 90% CI I think that – as a conservative approach – it should be acceptable for regulators.
- Extreme example: Design TRT|RTR, GMR 0.85, power ≥99%, n 506. At CV 0.301 TIE 0.0927. With Bonferroni’s αadj 0.025 the TIE is still 0.0532 (significantly >0.05; binomial test).
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png)
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png)
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- EMA oracle Helmut 2015-01-27 16:27 [RSABE / ABEL]
- EMA oracle nobody 2015-01-27 17:07
- Personal opinion ElMaestro 2015-01-27 17:56
- EMA oracle d_labes 2015-01-27 19:44
- EMA oracle nobody 2015-01-28 09:04
- EMA oracle ElMaestro 2015-01-28 13:11
- EMA oracle nobody 2015-01-28 09:04
- Response from Alfredo García-Arieta Helmut 2015-01-30 10:55
- Response from Alfredo García-Arieta nobody 2015-01-30 14:55
- Wonnemann’s Figure 4 on steroidsHelmut 2015-02-01 01:35
- Wonnemann’s Figure 4 on steroids nobody 2015-02-01 11:18
- Wonnemann’s Figure 4 on steroids Helmut 2015-02-01 14:49
- General remarks ElMaestro 2015-02-01 18:43
- General remarks nobody 2015-02-01 22:04
- General remarks Helmut 2015-02-02 00:48
- General remarks nobody 2015-02-02 08:09
- General remarks Helmut 2015-02-02 14:06
- TIE: questions mittyri 2015-02-03 12:44
- TIE: questions Helmut 2015-02-03 13:24
- TIE: questions mittyri 2015-02-03 12:44
- General remarks Helmut 2015-02-02 14:06
- FDA RSABE & alpha inflation d_labes 2015-02-03 12:02
- FDA RSABE & alpha inflation Helmut 2015-02-03 12:22
- FDA RSABE & alpha inflation d_labes 2015-02-03 13:07
- FDA RSABE & alpha inflation Helmut 2015-02-03 13:41
- FDA RSABE & alpha inflation d_labes 2015-02-03 13:07
- FDA RSABE & alpha inflation Helmut 2015-02-03 12:22
- General remarks nobody 2015-02-02 08:09
- Wonnemann’s Figure 4 on steroids nobody 2015-02-01 11:18
- Wonnemann’s Figure 4 on steroidsHelmut 2015-02-01 01:35
- Response from Alfredo García-Arieta nobody 2015-01-30 14:55
- New Zealand nobody 2015-01-31 11:25
- New Zealand Helmut 2015-02-01 16:08
- New Zealand nobody 2015-02-01 22:08
- New Zealand Helmut 2015-02-01 23:54
- New Zealand nobody 2015-02-02 08:07
- New Zealand Helmut 2015-02-01 23:54
- New Zealand nobody 2015-02-01 22:08
- New Zealand Helmut 2015-02-01 16:08