## Type I Error (TIE) [General Statistics]

Dear Helmut and others,

I have several theoretical thoughts about TIE inflation in BE studies under some circumstances.

First of all, I can't see the reason of TIE inflation for FDA 2x2 model with groups (as mentioned at the end of this presentation). In many of cases discussed here on BEBAC forum where TIE inflation presents, it is clear to me why. So it led me to various topics where the TIE inflation is more obvious (as two-stage design because of data evaluation twice or even TIE inflation for replicate designs with widening the acceptance limits - because of floating acceptance limits) - as described in different topics many times.

And this way of thinking bring me to the qustion "Can a pilot study be used as pivotal if 90% CI is in 80-125% (i.e. BE is concluded)?". The answer is regulatory dependend as I read here in several topics with many of reasons (ethical or maybe also obtained intra-subject CV can lead to the same number of subjects in pivotal study as was used in the pilot study, etc.). Though I thought that repetition of the study is not ethical or just increasing the sponsor's risk. I tend to the fact that a pilot study is a pilot study is a pilot study.

For simplicity consider following example: The plan would be to make a pilot study with 24 subjects.

If 90% CI will be in 80-125% - stop - BE concluded.

Else pivotal study will be performed to evaluate BE.

This scenario as the whole plan has an adventage of the increasing the power - as in pilot study we have chance to conclude BE (even for underpowered cases by chance). If BE will not be concluded in pilot study it is planned to have at least 80% or 90% power in the pivotal study.

I.e. in the whole plan we have more power than only in a pivotal study (some not known power for pilot study and desired power for pivotal study).

But TIE is a probability of BE conclusion in case of true GMR=0.8 (or 1.25) so we have 5% TIE for pilot (or maybe less as illustrated here but surely nonzero TIE) and 5% TIE for pivotal.

If the treatments are not bioequivalent (borderline case) and the pilot study is not underpowered (which can happen, nevertheless even for underpowered study the TIE would be still > 0%), we have 5% probability of demonstrating BE in pilot study. Then according to the whole plan stop - BE concluded.

Else pivotal study (e.g. with some more subjects added) will be performed with again 5% TIE. When we look at the plan as whole:

If I imagine simulations:

The pilot study is performed everytime (5% of all cases conclude BE after pilot study - STOP)

95% of cases continue to pivotal (5% from these 95% of cases conclude BE, i.e. 4.75% of all cases conclude BE after pivotal study, as 0.05*0.95=0.0475)

The aggregate TIE is 5+4.75 = 9.75%.

Note this is the maximum TIE, as it was shown in the mentioned topic, the real TIE can be between 5-9.75% (wenn ich mich nicht irre).

I don't know about any problems with regulatory acceptance of that and I found quite similar example with study repetition in this interesting PAR - 90% CIs from study 1 and study 2 do not even overlap but the second larger study concluded BE.

But I see nothing of that in the FDA model with groups which could result in TIE inflation. Maybe I simplified it in my imagination too much...

What I imagine is simulations of studies (similar as above, i.e. borderline case when the treatments are not bioequivalent) where for each simulated study we calculate ANOVA to know the significance of Group*Treatment interaction which splits the cases to 2 ways:

These my expectations are far away from the Helmut's simulations presented in the (mentioned presentation), only maybe the 4.5% for Model II when the groups can be pooled seems to be quite close to my thoughts.

Best regards,

zizou

I have several theoretical thoughts about TIE inflation in BE studies under some circumstances.

First of all, I can't see the reason of TIE inflation for FDA 2x2 model with groups (as mentioned at the end of this presentation). In many of cases discussed here on BEBAC forum where TIE inflation presents, it is clear to me why. So it led me to various topics where the TIE inflation is more obvious (as two-stage design because of data evaluation twice or even TIE inflation for replicate designs with widening the acceptance limits - because of floating acceptance limits) - as described in different topics many times.

And this way of thinking bring me to the qustion "Can a pilot study be used as pivotal if 90% CI is in 80-125% (i.e. BE is concluded)?". The answer is regulatory dependend as I read here in several topics with many of reasons (ethical or maybe also obtained intra-subject CV can lead to the same number of subjects in pivotal study as was used in the pilot study, etc.). Though I thought that repetition of the study is not ethical or just increasing the sponsor's risk. I tend to the fact that a pilot study is a pilot study is a pilot study.

For simplicity consider following example: The plan would be to make a pilot study with 24 subjects.

If 90% CI will be in 80-125% - stop - BE concluded.

Else pivotal study will be performed to evaluate BE.

This scenario as the whole plan has an adventage of the increasing the power - as in pilot study we have chance to conclude BE (even for underpowered cases by chance). If BE will not be concluded in pilot study it is planned to have at least 80% or 90% power in the pivotal study.

I.e. in the whole plan we have more power than only in a pivotal study (some not known power for pilot study and desired power for pivotal study).

But TIE is a probability of BE conclusion in case of true GMR=0.8 (or 1.25) so we have 5% TIE for pilot (or maybe less as illustrated here but surely nonzero TIE) and 5% TIE for pivotal.

If the treatments are not bioequivalent (borderline case) and the pilot study is not underpowered (which can happen, nevertheless even for underpowered study the TIE would be still > 0%), we have 5% probability of demonstrating BE in pilot study. Then according to the whole plan stop - BE concluded.

Else pivotal study (e.g. with some more subjects added) will be performed with again 5% TIE. When we look at the plan as whole:

If I imagine simulations:

The pilot study is performed everytime (5% of all cases conclude BE after pilot study - STOP)

95% of cases continue to pivotal (5% from these 95% of cases conclude BE, i.e. 4.75% of all cases conclude BE after pivotal study, as 0.05*0.95=0.0475)

The aggregate TIE is 5+4.75 = 9.75%.

Note this is the maximum TIE, as it was shown in the mentioned topic, the real TIE can be between 5-9.75% (wenn ich mich nicht irre).

I don't know about any problems with regulatory acceptance of that and I found quite similar example with study repetition in this interesting PAR - 90% CIs from study 1 and study 2 do not even overlap but the second larger study concluded BE.

But I see nothing of that in the FDA model with groups which could result in TIE inflation. Maybe I simplified it in my imagination too much...

What I imagine is simulations of studies (similar as above, i.e. borderline case when the treatments are not bioequivalent) where for each simulated study we calculate ANOVA to know the significance of Group*Treatment interaction which splits the cases to 2 ways:

- The interaction is not significant (approx 90% of all cases) - 5% from these cases will conclude BE, i.e. 4.5% (as 0.05*0.90=0.045)

- The interaction is significant (approx 10% of all cases) - we are not allowed to pool the data (per protocol) so we evaluate the data from groups separately by standard model. For simplicity the study was done in 2 groups, there are options:

- if per protocol BE is based on the data from one (larger) group only

- 5% from these cases will conclude BE, i.e. 0.5% (as 0.05*0.10=0.005) of all cases conclude BE - if underpowered, real result may be lower than 0.5%

I would expect the maximum aggregate TIE for case when larger group must meet the acceptance limits as 4.5+0.5 = 5%.

- if per protocol both groups must meet the acceptance limits for the conclusion of BE

- 0.25% (as 0.05*0.05=0.0025) from these cases will conclude BE in both groups 1 and 2 at the same time, i.e. 0.025% (as 0.0025*0.10=0.00025) of all cases conclude BE

I would expect the maximum aggregate TIE for case when both groups must meet the acceptance limits as 4.5+0.025 = 4.525%.

(But I wouldn't preffer this option at all because of decreasing power ... better would be to state in the protocol that BE will be assessed in the first group only)

- if per protocol BE is based on the data from one of the groups

- 5% from these cases will conclude BE in group 1, i.e. 0.5% (as 0.05*0.10=0.005) - this calculation will be probably underpowered, so real result will be probably lower than 0.5%

- 5% from these cases will conclude BE in group 2, i.e. 0.5% (as 0.05*0.10=0.005) - this calculation will be probably underpowered, so real result will be probably lower than 0.5%

- Moreover there will be cases in which both groups conclude BE, so simulations should provide the result smaller by 0.25% (as 0.05*0.05=0.0025), i.e. smaller by 0.025% (as 0.0025*0.10=0.00025).

I would expect the maximum aggregate TIE for case when one of 2 groups must meet the acceptance limits as 4.5+0.5+0.5-0.025 = 5.475%.

Oh in this case I surprisingly found me to be able to see quite possibly the TIE inflation. But I am really not sure about what I wrote so far.

- if per protocol BE is based on the data from one (larger) group only

These my expectations are far away from the Helmut's simulations presented in the (mentioned presentation), only maybe the 4.5% for Model II when the groups can be pooled seems to be quite close to my thoughts.

Best regards,

zizou

### Complete thread:

- Type I Error (TIE)zizou 2018-11-18 16:38
- Type I Error (TIE) of group models d_labes 2018-11-23 15:40