GM
★    

India,
2017-03-27 21:40
(2557 d 16:40 ago)

Posting: # 17188
Views: 9,615
 

 Relationship between calculated 90% CI and sign. treatment effect in BE [General Sta­tis­tics]

Dear All,

I have a doubt that, is there any relationship between calculated 90% CI and significant treatment effect in BE studies.

If the 90%CI is one side of hundred, i.e. either (85.00-98.00) or (102.00-119.00) then there should be significant treatment effect. Is this statement correct...?

And why we are generally checking the treatment effect at 5% level of significance...?

Please clarify.

Many thanks in advance.
Regards,
GM.

Best Regards,
GM
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-03-28 01:47
(2557 d 12:33 ago)

@ GM
Posting: # 17189
Views: 8,862
 

 1–2α CI and TOST at α 0.05

Hi GM,

❝ If the 90%CI is one side of hundred, i.e. either (85.00-98.00) or (102.00-119.00) then there should be significant treatment effect. Is this statement correct...?


Absolutely.

❝ And why we are generally checking the treatment effect at 5% level of significance...?


The inclusion of a 100(1–2α) confidence interval within the common acceptance range [L, U] of 80–125% is operationally equivalent to Two One-Sided t-Tests (TOST). One t-test is for ≤80% and the other one for ≥125%, both at a level of α 0.05. Null and alternative hypotheses:

Inclusion of the 100(1–2α) CI:

[image]

TOST at α 0.05:

[image]


In BE the Null is inequivalence.

Extreme example:

library(PowerTOST)
lower <- 0.80
upper <- 1.25
pe    <- sqrt(lower*upper)
n     <- 24
alpha <- 0.05
CV    <- CVfromCI(lower=lower, upper=upper, n=n, alpha=alpha, design="2x2")
p     <- pvalues.TOST(pe=pe, CV=CV, n=n, design="2x2")
cat(sprintf("%g%% %s %.2f–%.2f%%", 100*(1-2*alpha),
    "CI:", 100*lower, 100*upper), "\n"); print(p)

# 90% CI: 80.00–125.00%
#  p.left p.right
#    0.05    0.05


Although the treatment-effect (and its p-value) is part of the standard output of most software packages, it is not relevant for the BE-decision. Statistical significant  clinically relevant…
I don’t know why you are checking it – I don’t.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
GM
★    

India,
2017-03-28 08:50
(2557 d 05:30 ago)

@ Helmut
Posting: # 17190
Views: 8,693
 

 1–2α CI and TOST at α 0.05

❝ ❝ If the 90%CI is one side of hundred, i.e. either (85.00-98.00) or (102.00-119.00) then there should be significant treatment effect. Is this statement correct...?


❝ Absolutely.


Hi Helmut,

Thank you for the reply.

I was also found that most of the cases above statement is correct only. But I have a situation where treatment effect is not significant even the calculated 90%CI falls one side of hundred. Is it possible..?

Actually I was used Proc GLM in SAS for this 2×2 crossover study with the model statement mentioned below:

model logCmax = Sequence Period Form Cohort Form*Cohort Subject(Sequence*Cohort)/ SS3;

Cohort was mentioned as a fixed effect because the study conducts in two groups.

The fixed effects mentioned in above the model will effect the treatment effect...? Please explain.

Regards,
GM.

Best Regards,
GM
DavidManteigas
★    

Portugal,
2017-03-28 13:48
(2557 d 00:33 ago)

@ GM
Posting: # 17191
Views: 8,734
 

 1–2α CI and TOST at α 0.05

Hi GM,

The problem might be on the estimate statement. If the 90% CI is significantly different from 1, then the p-value for "formulation" should also be significant at the 10% significance level. Maybe you're using a different denominator than expected...

It would be easier if you post the full glm code.
ElMaestro
★★★

Denmark,
2017-03-28 15:12
(2556 d 23:08 ago)

@ DavidManteigas
Posting: # 17192
Views: 8,689
 

 1–2α CI and TOST at α 0.05

Hi DavidManteigas,

❝ If the 90% CI is significantly different from 1, then the p-value for "formulation" should also be significant at the 10% significance level. Maybe you're using a different denominator than expected...


(...) the p-value for "formulation" should also be significant at the 5% significance level assuming we work with alpha=5% and therefore 90% CI's.
Alpha is alpha, unless we talk two-stage approaches and the like.

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2017-03-28 17:12
(2556 d 21:08 ago)

@ ElMaestro
Posting: # 17193
Views: 8,789
 

 alpha TOST is not alpha 2-sided

My Dear!

❝ (...) the p-value for "formulation" should also be significant at the 5% significance level assuming we work with alpha=5% and therefore 90% CI's.

❝ Alpha is alpha ...


I think here you err.
See this thread (evaluation without group effect):

p-value of treatment effect = 0.076998 (not significant at the 5% level)
90% CI = 90.76 ... 99.62% (doesn't contain 100%)

Regards,

Detlew
ElMaestro
★★★

Denmark,
2017-03-28 23:10
(2556 d 15:10 ago)

@ d_labes
Posting: # 17195
Views: 8,682
 

 Interesting!

Hi d_labes,

❝ p-value of treatment effect = 0.076998 (not significant at the 5% level)

❝ 90% CI = 90.76 ... 99.62% (doesn't contain 100%)


Thank you.
This somehow does rock my fundamental understanding.
I need to sit down with pen and paper and work it out.

If anova p<0.05 for treatment then the 90% CI does not include 1.00. But vice versa isn't the case?!??? This is not at all intuitive to me since the Null is equality at alpha 5%. I must look a bit into this, I believe.

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2017-03-29 10:20
(2556 d 04:00 ago)

@ ElMaestro
Posting: # 17197
Views: 8,613
 

 Google has the answer

Dear Öberster Größter Meister,

:google: "duality confidence interval hypothesis testing" may help in understanding.

Regards,

Detlew
DavidManteigas
★    

Portugal,
2017-03-29 13:16
(2556 d 01:04 ago)

@ ElMaestro
Posting: # 17200
Views: 8,521
 

 Interesting!

Hi d_labes and ElMaestro,

I'm also struggling with the question now. A 90% CI compares with a hypothesis test at 10%. The 90% CI is equivalent to a statistical assessment of equivalente at the 5% level due the TOST approach, since you're not assessing significance for the null hypothesis of difference in means. Nevertheless, when you apply the model the 90% CI interval is an interval for difference in means regardless of the interpretation of the results in the bioequivalence context. As the statistical conclusion of "difference in means" is obtained at the 10% level and not 5% level, and the term for formulation is assessing whether there is a "difference in means" and not equivalence, the p value for formulation will be significant at the 10% significance level if the 90% confidence interval does not contains 1. So it is completly plausible for me to have a 90% CI without 1 and a non-significant p value for formulation at the 5% significance level.

Am I understanding the issue wrongly?

Regards,
David
ElMaestro
★★★

Denmark,
2017-03-29 13:20
(2556 d 01:00 ago)

@ DavidManteigas
Posting: # 17201
Views: 8,558
 

 Interesting!

Hi DavidManteigas and d_labes,

❝ I'm also struggling with the question now. A 90% CI compares with a hypothesis test at 10%. The 90% CI is equivalent to a statistical assessment of equivalente at the 5% level due the TOST approach, since you're not assessing significance for the null hypothesis of difference in means.


I beg to differ; the 90% CI approach applies a 5% alpha. A product which in not truly BE (GMR is 0.8 or below; or 1.25 or higher, can't be both), will have at most 5% chance of passing BE; te CI is made from 1.0-2alpha but that does not mean 10% chance of approving a non-BE product.

It is the same alpha 5% that is used in the ANOVA where the null hypo is sameness.

If I am wrong here then it is my very basic understanding of statistics that needs thorough remodeling.

Pass or fail!
ElMaestro
DavidManteigas
★    

Portugal,
2017-03-29 14:28
(2555 d 23:52 ago)

@ ElMaestro
Posting: # 17203
Views: 8,537
 

 Interesting!

Hi ElMaestro,

What I've said is that the p-value for formulation as nothing to do with the statistical conclusion of bioequivalence. For the statistical evaluation of "difference in means", to be compared with the same statistical conclusion of "difference in means" with the 90% confidence interval, the p value must be assessed against the 10% significance level. So, if your 90% Confidence Interval does not contain 1, then the p-value for formulation is also significant at the 10% significance level and may or not be also significant at the 5% significance level.

The model is build the same way whether you are assessing difference in means or bioequivalence. The hypothesis are, however, different. For the hypothesis of "bioequivalence" the alpha level is 5%, and for the hypothesis of "difference in means" the alpha level is 10%. And the p-value only assess the hypothesis of difference in means. I'm sorry if I'm not explaining myself right, I'll let someone smarter try to do a better job :-D
d_labes
★★★

Berlin, Germany,
2017-03-29 16:24
(2555 d 21:56 ago)

@ DavidManteigas
Posting: # 17205
Views: 8,637
 

 95% CI for a test on difference

Dear David, Dear ElMaestro!

❝ The model is build the same way whether you are assessing difference in means or bioequivalence. The hypothesis are, however, different.


Totally correct.

The equivalence test is TOST (two-one-sided tests) each with alpha=0.05 which is operationally equivalent
to 1-2*alpha CI.
Difference test has a two-sided alternative HA, and thus is dual to an 1-alpha CI, i.e. 95%CI if you set alpha to 0.05.
The 95% CI of the above cited example is

95% CI = 89.86 ... 100.61% (contains 100%)

so no significant treatment effect, same conclusion as via the ANOVA p-value.

Regards,

Detlew
GM
★    

India,
2017-03-29 22:06
(2555 d 16:14 ago)

@ DavidManteigas
Posting: # 17209
Views: 8,483
 

 Interesting!

Dear David,

❝ For the statistical evaluation of "difference in means", to be compared with the same statistical conclusion of "difference in means" with the 90% confidence interval, the p value must be assessed against the 10% significance level. So, if your 90% Confidence Interval does not contain 1, then the p-value for formulation is also significant at the 10% significance level and may or not be also significant at the 5% significance level.


As Helmut also told in the above discussion, generally we are assessing the treatment effect @5% significance level only.

But based on your statement, it is 10% level of significance. Now I am in confusion :confused:... whether 5% level or 10% level of significance we need to consider for BE...? Please clarify.

Thanks.
Regards,
GM

Best Regards,
GM
nobody
nothing

2017-03-30 10:24
(2555 d 03:56 ago)

@ GM
Posting: # 17210
Views: 8,524
 

 Interesting!

Helmut wrote:

"The inclusion of a 100(1–2α) confidence interval within the common acceptance range [L, U] of 80–125% is operationally equivalent to Two One-Sided t-Tests (TOST). One t-test is for ≤80% and the other one for ≥125%, both at a level of α 0.05."

Two tests (<80%, >125%), each with a 5% chance to fail, 5+5=10, gives an overall 10% chance to fail... Therefore 90% CI is equivalent to the TOST with 5% each.

Kindest regards, nobody
GM
★    

India,
2017-03-29 14:11
(2556 d 00:09 ago)

@ DavidManteigas
Posting: # 17202
Views: 8,574
 

 Relationship between calculated 90% CI and sign. treatment effect in BE

Hi David,

❝ The problem might be on the estimate statement. If the 90% CI is significantly different from 1, then the p-value for "formulation" should also be significant at the 10% significance level. Maybe you're using a different denominator than expected...


❝ It would be easier if you post the full glm code.


Here is the code used for my analysis.

Proc GLM data=logdata;
   class Sequence Subject Period Form Cohort;
   model Log(Param) = Sequence Period Form Cohort Form*Cohort Subject(Sequence*Cohort)/ SS3;
   output out=outlier rstudent=student;
   test h=Sequence e=Subject(Sequence*Cohort) / htype=3 etype=3;
   lsmeans Form / pdiff CL alpha=0.10;
   estimate 'A VS B' Form 1 -1;
run;


I got the values for CI is 77.27-98.78 (doesn't contain 100%) and p-value of treatment effect is 0.0709 (not significant at the 5% level).

My observation is that when removing the Form*Cohort term from the model, treatment effect is significant @5% level of significance.

My question is that the terms which are used in the model are sufficient or not...? and if it is correct, why p-value of treatment effect is not significant...?:confused:

Thanks,
GM.

Best Regards,
GM
DavidManteigas
★    

Portugal,
2017-03-29 14:30
(2555 d 23:50 ago)

@ GM
Posting: # 17204
Views: 8,663
 

 Relationship between calculated 90% CI and sign. treatment effect in BE

Hi Gm,

Your code seems ok to me.

In my opinion, your result is completly normal since the p-value is significant at the 10% significance level, which compares with the 90% CI with obtained. ElMaestro has a different opinion though, so you should follow our discussion :-D

Regards,
David
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,639 registered users;
74 visitors (0 registered, 74 guests [including 5 identified bots]).
Forum time: 13:21 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5