Bioequivalence and Bioavailability Forum

GM
★

India,
2017-03-27 21:40
(2953 d 08:30 ago)

Posting: # 17188
Views: 11,329

Relationship between calculated 90% CI and sign. treatment effect in BE [General Statistics]

Dear All,

I have a doubt that, is there any relationship between calculated 90% CI and significant treatment effect in BE studies.

If the 90%CI is one side of hundred, i.e. either (85.00-98.00) or (102.00-119.00) then there should be significant treatment effect. Is this statement correct...?

And why we are generally checking the treatment effect at 5% level of significance...?

Please clarify.

Many thanks in advance.
Regards,
GM.

—
Best Regards,
GM

Helmut
★★★

Vienna, Austria,
2017-03-28 01:47
(2953 d 04:23 ago)

@ GM
Posting: # 17189
Views: 10,313

1–2α CI and TOST at α 0.05

Post reply

Hi GM,

❝ If the 90%CI is one side of hundred, i.e. either (85.00-98.00) or (102.00-119.00) then there should be significant treatment effect. Is this statement correct...?

Absolutely.

❝ And why we are generally checking the treatment effect at 5% level of significance...?

The inclusion of a 100(1–2α) confidence interval within the common acceptance range [L, U] of 80–125% is operationally equivalent to Two One-Sided t-Tests (TOST). One t-test is for ≤80% and the other one for ≥125%, both at a level of α 0.05. Null and alternative hypotheses:

Inclusion of the 100(1–2α) CI:

[image]

TOST at α 0.05:

[image]

In BE the Null is inequivalence.

Extreme example:

library(PowerTOST) lower <- 0.80 upper <- 1.25 pe <- sqrt(lower*upper) n <- 24 alpha <- 0.05 CV <- CVfromCI(lower=lower, upper=upper, n=n, alpha=alpha, design="2x2") p <- pvalues.TOST(pe=pe, CV=CV, n=n, design="2x2") cat(sprintf("%g%% %s %.2f–%.2f%%", 100*(1-2*alpha), "CI:", 100*lower, 100*upper), "\n"); print(p) # 90% CI: 80.00–125.00% # p.left p.right # 0.05 0.05

Although the treatment-effect (and its p-value) is part of the standard output of most software packages, it is not relevant for the BE-decision. Statistical significant ≠ clinically relevant…
I don’t know why you are checking it – I don’t.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

GM
★

India,
2017-03-28 08:50
(2952 d 21:20 ago)

@ Helmut
Posting: # 17190
Views: 10,133

1–2α CI and TOST at α 0.05

Post reply

❝ ❝ If the 90%CI is one side of hundred, i.e. either (85.00-98.00) or (102.00-119.00) then there should be significant treatment effect. Is this statement correct...?

❝

❝ Absolutely.

Hi Helmut,

Thank you for the reply.

I was also found that most of the cases above statement is correct only. But I have a situation where treatment effect is not significant even the calculated 90%CI falls one side of hundred. Is it possible..?

Actually I was used Proc GLM in SAS for this 2×2 crossover study with the model statement mentioned below:

model logCmax = Sequence Period Form Cohort Form*Cohort Subject(Sequence*Cohort)/ SS3;

Cohort was mentioned as a fixed effect because the study conducts in two groups.

The fixed effects mentioned in above the model will effect the treatment effect...? Please explain.

Regards,
GM.

—
Best Regards,
GM

DavidManteigas ★ Portugal, 2017-03-28 13:48 (2952 d 16:22 ago) @ GM Posting: # 17191 Views: 10,179	1–2α CI and TOST at α 0.05 Post reply
	Hi GM, The problem might be on the estimate statement. If the 90% CI is significantly different from 1, then the p-value for "formulation" should also be significant at the 10% significance level. Maybe you're using a different denominator than expected... It would be easier if you post the full glm code.

ElMaestro
★★★

Denmark,
2017-03-28 15:12
(2952 d 14:58 ago)

@ DavidManteigas
Posting: # 17192
Views: 10,133

1–2α CI and TOST at α 0.05

Post reply

Hi DavidManteigas,

❝ If the 90% CI is significantly different from 1, then the p-value for "formulation" should also be significant at the 10% significance level. Maybe you're using a different denominator than expected...

(...) the p-value for "formulation" should also be significant at the 5% significance level assuming we work with alpha=5% and therefore 90% CI's.
Alpha is alpha, unless we talk two-stage approaches and the like.

—
Pass or fail!
ElMaestro

d_labes
★★★

Berlin, Germany,
2017-03-28 17:12
(2952 d 12:57 ago)

@ ElMaestro
Posting: # 17193
Views: 10,274

alpha TOST is not alpha 2-sided

Post reply

My Dear!

❝ (...) the p-value for "formulation" should also be significant at the 5% significance level assuming we work with alpha=5% and therefore 90% CI's.

❝ Alpha is alpha ...

I think here you err.
See this thread (evaluation without group effect):

p-value of treatment effect = 0.076998 (not significant at the 5% level)
90% CI = 90.76 ... 99.62% (doesn't contain 100%)

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2017-03-28 23:10
(2952 d 07:00 ago)

@ d_labes
Posting: # 17195
Views: 10,111

Interesting!

Post reply

Hi d_labes,

❝ p-value of treatment effect = 0.076998 (not significant at the 5% level)

❝ 90% CI = 90.76 ... 99.62% (doesn't contain 100%)

Thank you.
This somehow does rock my fundamental understanding.
I need to sit down with pen and paper and work it out.

If anova p<0.05 for treatment then the 90% CI does not include 1.00. But vice versa isn't the case?!??? This is not at all intuitive to me since the Null is equality at alpha 5%. I must look a bit into this, I believe.

—
Pass or fail!
ElMaestro

d_labes ★★★ Berlin, Germany, 2017-03-29 10:20 (2951 d 19:50 ago) @ ElMaestro Posting: # 17197 Views: 10,070	Google has the answer Post reply
	Dear Öberster Größter Meister, "duality confidence interval hypothesis testing" may help in understanding. — Regards, Detlew

DavidManteigas
★

Portugal,
2017-03-29 13:16
(2951 d 16:54 ago)

@ ElMaestro
Posting: # 17200
Views: 9,965

Interesting!

Post reply

Hi d_labes and ElMaestro,

I'm also struggling with the question now. A 90% CI compares with a hypothesis test at 10%. The 90% CI is equivalent to a statistical assessment of equivalente at the 5% level due the TOST approach, since you're not assessing significance for the null hypothesis of difference in means. Nevertheless, when you apply the model the 90% CI interval is an interval for difference in means regardless of the interpretation of the results in the bioequivalence context. As the statistical conclusion of "difference in means" is obtained at the 10% level and not 5% level, and the term for formulation is assessing whether there is a "difference in means" and not equivalence, the p value for formulation will be significant at the 10% significance level if the 90% confidence interval does not contains 1. So it is completly plausible for me to have a 90% CI without 1 and a non-significant p value for formulation at the 5% significance level.

Am I understanding the issue wrongly?

Regards,
David

ElMaestro
★★★

Denmark,
2017-03-29 13:20
(2951 d 16:49 ago)

@ DavidManteigas
Posting: # 17201
Views: 9,985

Interesting!

Post reply

Hi DavidManteigas and d_labes,

❝ I'm also struggling with the question now. A 90% CI compares with a hypothesis test at 10%. The 90% CI is equivalent to a statistical assessment of equivalente at the 5% level due the TOST approach, since you're not assessing significance for the null hypothesis of difference in means.

I beg to differ; the 90% CI approach applies a 5% alpha. A product which in not truly BE (GMR is 0.8 or below; or 1.25 or higher, can't be both), will have at most 5% chance of passing BE; te CI is made from 1.0-2alpha but that does not mean 10% chance of approving a non-BE product.

It is the same alpha 5% that is used in the ANOVA where the null hypo is sameness.

If I am wrong here then it is my very basic understanding of statistics that needs thorough remodeling.

—
Pass or fail!
ElMaestro

DavidManteigas
★

Portugal,
2017-03-29 14:28
(2951 d 15:42 ago)

@ ElMaestro
Posting: # 17203
Views: 9,956

Interesting!

Post reply

Hi ElMaestro,

What I've said is that the p-value for formulation as nothing to do with the statistical conclusion of bioequivalence. For the statistical evaluation of "difference in means", to be compared with the same statistical conclusion of "difference in means" with the 90% confidence interval, the p value must be assessed against the 10% significance level. So, if your 90% Confidence Interval does not contain 1, then the p-value for formulation is also significant at the 10% significance level and may or not be also significant at the 5% significance level.

The model is build the same way whether you are assessing difference in means or bioequivalence. The hypothesis are, however, different. For the hypothesis of "bioequivalence" the alpha level is 5%, and for the hypothesis of "difference in means" the alpha level is 10%. And the p-value only assess the hypothesis of difference in means. I'm sorry if I'm not explaining myself right, I'll let someone smarter try to do a better job :-D

d_labes
★★★

Berlin, Germany,
2017-03-29 16:24
(2951 d 13:45 ago)

@ DavidManteigas
Posting: # 17205
Views: 10,096

95% CI for a test on difference

Post reply

Dear David, Dear ElMaestro!

❝ The model is build the same way whether you are assessing difference in means or bioequivalence. The hypothesis are, however, different.

Totally correct.

The equivalence test is TOST (two-one-sided tests) each with alpha=0.05 which is operationally equivalent
to 1-2*alpha CI.
Difference test has a two-sided alternative H_A, and thus is dual to an 1-alpha CI, i.e. 95%CI if you set alpha to 0.05.
The 95% CI of the above cited example is

95% CI = 89.86 ... 100.61% (contains 100%)

so no significant treatment effect, same conclusion as via the ANOVA p-value.

—
Regards,

Detlew

GM
★

India,
2017-03-29 22:06
(2951 d 08:03 ago)

@ DavidManteigas
Posting: # 17209
Views: 9,915

Interesting!

Post reply

Dear David,

❝ For the statistical evaluation of "difference in means", to be compared with the same statistical conclusion of "difference in means" with the 90% confidence interval, the p value must be assessed against the 10% significance level. So, if your 90% Confidence Interval does not contain 1, then the p-value for formulation is also significant at the 10% significance level and may or not be also significant at the 5% significance level.

As Helmut also told in the above discussion, generally we are assessing the treatment effect @5% significance level only.

But based on your statement, it is 10% level of significance. Now I am in confusion :confused:

... whether 5% level or 10% level of significance we need to consider for BE...? Please clarify.

Thanks.
Regards,
GM

—
Best Regards,
GM

nobody
nothing

2017-03-30 10:24
(2950 d 19:46 ago)

@ GM
Posting: # 17210
Views: 9,952

Interesting!

Post reply

Helmut wrote:

"The inclusion of a 100(1–2α) confidence interval within the common acceptance range [L, U] of 80–125% is operationally equivalent to Two One-Sided t-Tests (TOST). One t-test is for ≤80% and the other one for ≥125%, both at a level of α 0.05."

Two tests (<80%, >125%), each with a 5% chance to fail, 5+5=10, gives an overall 10% chance to fail... Therefore 90% CI is equivalent to the TOST with 5% each.

—
Kindest regards, nobody

GM
★

India,
2017-03-29 14:11
(2951 d 15:58 ago)

@ DavidManteigas
Posting: # 17202
Views: 10,019

Relationship between calculated 90% CI and sign. treatment effect in BE

Post reply

Hi David,

❝ The problem might be on the estimate statement. If the 90% CI is significantly different from 1, then the p-value for "formulation" should also be significant at the 10% significance level. Maybe you're using a different denominator than expected...

❝

❝ It would be easier if you post the full glm code.

Here is the code used for my analysis.

Proc GLM data=logdata;

   class Sequence Subject Period Form Cohort;

   model Log(Param) = Sequence Period Form Cohort Form*Cohort Subject(Sequence*Cohort)/ SS3;

   output out=outlier rstudent=student;

   test h=Sequence e=Subject(Sequence*Cohort) / htype=3 etype=3;

   lsmeans Form / pdiff CL alpha=0.10;

   estimate 'A VS B' Form 1 -1;

run;

I got the values for CI is 77.27-98.78 (doesn't contain 100%) and p-value of treatment effect is 0.0709 (not significant at the 5% level).

My observation is that when removing the Form*Cohort term from the model, treatment effect is significant @5% level of significance.

My question is that the terms which are used in the model are sufficient or not...? and if it is correct, why p-value of treatment effect is not significant...? :confused:

Thanks,
GM.

—
Best Regards,
GM

DavidManteigas ★ Portugal, 2017-03-29 14:30 (2951 d 15:40 ago) @ GM Posting: # 17204 Views: 10,089	Relationship between calculated 90% CI and sign. treatment effect in BE Post reply
	Hi Gm, Your code seems ok to me. In my opinion, your result is completly normal since the p-value is significant at the 10% significance level, which compares with the 90% CI with obtained. ElMaestro has a different opinion though, so you should follow our discussion Regards, David

Relationship between calculated 90% CI and sign. treatment effect in BE [General Sta­tis­tics]

1–2α CI and TOST at α 0.05

1–2α CI and TOST at α 0.05

1–2α CI and TOST at α 0.05

1–2α CI and TOST at α 0.05

alpha TOST is not alpha 2-sided

Interesting!

Google has the answer

Interesting!

Interesting!

Interesting!

95% CI for a test on difference

Interesting!

Interesting!

Relationship between calculated 90% CI and sign. treatment effect in BE

Relationship between calculated 90% CI and sign. treatment effect in BE

Relationship between calculated 90% CI and sign. treatment effect in BE [General Statistics]