Bioequivalence and Bioavailability Forum

d_labes
★★★

Berlin, Germany,
2012-12-05 13:13
(4939 d 14:10 ago)

Posting: # 9672
Views: 11,644

CV point estimate outside CI [General Statistics]

Dear All!

Sometimes it may be desirable to calculate the confidence interval for a coefficient of variation. See this thread.

PowerTOST therefore contains a function CVCL() to do this. Helmut, in a personal communication, pointed out a peculiarity of the confidence intervals if one uses some rather unusual values for alpha, the probability of error:

R-code (new defined CVCL() in an upcoming version of PowerTOST):

CVCL <- function(CV, df, side=c("upper", "lower", "2-sided"), alpha=0.05)

{

  ssintra <- log(1.0 + CV^2)*df

  side    <- match.arg(side)

  

  limits <- switch(EXPR=side,

      upper= c(0, ssintra/qchisq(alpha,df)),

      lower= c(ssintra/qchisq(1-alpha,df), Inf),

      # 2-sided

      c(ssintra/qchisq(1-alpha/2,df), ssintra/qchisq(alpha/2,df)))

  limits <-(sqrt(exp(limits)-1))

  names(limits) <- c("lower CL", "upper CL")

  return(limits)

}



alphas <- c(0,0.001,0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.5545, 0.6, 0.7,

            0.8, 0.9, 0.95, 0.99, 1)

data.frame(alpha=alphas, t(sapply(FUN=CVCL, X=alphas, CV=0.3, df=12, side="upper")))

Have a look at the output
(lower limit of course always zero for a upper one-sided CI):

    alpha lower.CL   upper.CL

   0.0000        0       Inf

   0.0010        0   0.7715381

   0.0100        0   0.5795936

   0.0500        0   0.4677793

   0.1000        0   0.4222230

   0.2000        0   0.3763370

   0.3000        0   0.3482476

   0.4000        0   0.3269586

   0.5000        0   0.3089950

   0.5545        0   0.2999708

   0.6000        0   0.2926612

   0.7000        0   0.2767673

   0.8000        0   0.2599767

   0.9000        0   0.2394445

   0.9500        0   0.2245282

   0.9900        0   0.2005828

   1.0000        0   0.0000000

Above alpha ~0.5545 the confidence interval does not cover the point estimate. This is astonishing to me and fascinating, because in my belief the point estimate was always bracketed by a confidence interval.

Is this correct? Or is there a hidden bug? The latter I can't imagine because the calculation is simple enough.

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2012-12-05 19:41
(4939 d 07:41 ago)

@ d_labes
Posting: # 9676
Views: 10,223

What the heck?

Post reply

Dear Detlew!

❝ Above alpha ~0.5545 the confidence interval does not cover the point estimate. This is astonishing to me and fascinating, because in my belief the point estimate was always bracketed by a confidence interval.

❝ Is this correct? Or is there a hidden bug? The latter I can't imagine because the calculation is simple enough.

What’s going on here? I always followed blindly Chow & Liu (Chapter 7); calculated a two-sided interval and used only the upper limit. I tried not to fall in the trap of using your code (which is fine, IMHO), but recalculated an example (the famous Clayton & Leslie data set from Table 6.9.1). Straightforward analysis in log-scale, SS_intra 4.94398, MS_intra 0.308999, 16 df). Now the confusion starts already. For CV_intra I get 0.601715. They write

“The estimated CV is about 36.55%…”

I beg your pardon? Do they mean CV_inter? No, that would be 0.334785. Anyhow, maybe just another typo.*
I’m using their symbols.

alpha <- 0.05 n1 <- 9; n2 <- 9 df <- n1+n2-2 MSintra <- 0.308999 CVintra <- sqrt(exp(MSintra)-1) SSintra <- MSintra*df Chi2L <- qchisq(alpha/2, df) Chi2U <- qchisq(1-alpha/2, df) Le <- SSintra/Chi2U Ue <- SSintra/Chi2L cat("(",round(Le,2),",",round(Ue,2),")\n")

0.17 – 0.72 agrees with their Table 7.3.1. But this is the 95% CI of σ²_e and we want the CI of CV_intra. Therefore

CVL <- sqrt(exp(Le)-1) CVU <- sqrt(exp(Ue)-1) cat("alpha",alpha,":",round(CVL,4),"]", round(CVintra,4),"[",round(CVU,4),"\n") alpha 0.05 : 0.4324 ] 0.6017 [ 1.0226

Fine.
Give it a try with alpha <- 0.9065 :confused:

I guess (!) they calculated the CV from log-transformed data like in the untransformed (!) case with $CV=\sqrt{MSE}/\bar{x}_R=\sqrt{0.308999}/1.520898=0.36549235.$ Crazy.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2012-12-06 10:21
(4938 d 17:01 ago)

@ Helmut
Posting: # 9680
Views: 10,191

Code validated

Post reply

Dear Helmut!

❝ What’s going on here? I always followed blindly Chow & Liu (Chapter 7); calculated a two-sided interval and used only the upper limit.

Blindly

following somebody is not always the best strategy to not falling into a trap.
Two-sided is identical with one-sided calculated with alpha/2, I think.
Further I think that you don't remember your own habit correctly. See your post dating back to the year 2006:
"Since we are interested only in the upper limit of CV – if CV in the main study is lower than predicted, we are only gaining power – it may be better to apply a one-side approach instead." :-D

❝ I tried not to fall in the trap of using your code (which is fine, IMHO), but recalculated an example (the famous Clayton & Leslie data set from Table 6.9.1). Straightforward analysis in log-scale, SSintra 4.94398, MSintra 0.308999, 16 df). Now the confusion starts already. For CVintra I get 0.601715. They write

“The estimated CV is about 36.55%…”

I beg your pardon? Do they mean CVinter? No, that would be 0.334785. Anyhow, maybe just a typo.*

Chow & Liu's book (throughout all editions) is known to contain a massive number of typos. Thus be careful!

❝ I’m using their symbols.

❝ alpha <- 0.05

❝ n1 <- 9; n2 <- 9

❝ df <- n1+n2-2

❝ ...

❝ alpha 0.05 : 0.4324 ] 0.6017 [ 1.0226

❝ Fine.

❝ Give it a try with alpha <- 0.9065 :confused:

Your code gives the same answer as mine:

CVCL(alpha=0.05, CV=0.6017, df=16, side="2-sided")

 lower CL  upper CL 

0.4323797 1.0225483 

CVCL(alpha=0.9065, CV=0.6017, df=16, side="2-sided")

 lower CL  upper CL 

0.6017906 0.6322904

Point estimate not contained in CI!

Thus I think the code is validated and the effect we see is true.
May be that my belief that a CI is bracketing the point estimate is relying on CIs for means.

Of course no one rational thinking human being, except nitpickers like you and me :-D

, will set the error probability to such values >0.5.

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2012-12-06 12:14
(4938 d 15:09 ago)

@ d_labes
Posting: # 9681
Views: 10,139

PE outside its CI?!

Post reply

Dear Detlew!

❝ Blindly :cool: following somebody is not always the best strategy to not falling into a trap.

Agree.

❝ Two-sided is identical with one-sided calculated with alpha/2, I think.

This is what I thought (and still think). These formulas are so simple that I cannot imagine what’s going on here.

❝ Further I think that you don't remember your own habit correctly. See your post dating back to the year 2006:

❝ "Since we are interested only in the upper limit of CV – if CV in the main study is lower than predicted, we are only gaining power – it may be better to apply a one-side approach instead." :-D

OK, OK.

❝ Thus I think the code is validated and the effect we see is true.

Ha! In my presentations on bioanalytics I use to make a distinction between “validated” and “valid”. Whilst the former means following a set of instructions (the validation plan) the latter means getting the “correct” results. That’s why I’m also skeptical about software vendor’s “validation packages”. If both the software and the validation package contains the same bug (2×2=5) we are proud about getting the expected results (validated) – which is still wrong… Following our private conversation your source was SA Julious. Now we followed different guide-dogs and got the same results. The code is validated. But is it valid? Would need a professional statistician to sort that out. Martin? Jack?

❝ May be that my belief that a CI is bracketing the point estimate is relying on CIs for means.

Duno. IMHO valid for any kind of point estimator (the mean being just one of them). Now I’m lost. Like you I always thought the two-sided CI gives the range containing the expected value at 1–α/2. In the extreme cases of α=0 the CI should be -∞,+∞^* and for α=1 CL_lo=PE=CL_hi. As our Indian friends use to say: Correct me if I’m wrong.

❝ Of course no one rational thinking human being, except nitpickers like you and me :-D , will set the error probability to such values >0.5.

Right. Sorry opening another can of worms.

For the normal distribution. Here $]0,+\infty]\ldots$

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Jack
☆

Lancaster, United Kingdom,
2012-12-06 14:20
(4938 d 13:03 ago)

@ Helmut
Posting: # 9682
Views: 10,113

PE outside its CI?!

Post reply

Since called upon, I will comment although it may not be overly useful in practice.

In principle it is possible to have a confidence interval that does not include the point estimate that is still valid (i.e. has the correct coverage). It is, however, not something that would be considered a "good" confidence interval because the result of a hypothesis test would not necessarily match the conclusion based on a CI which obviously has all sort of flaws.

The main reasons (but not the only ones) why that is happening is because the point estimate is biased or some non-linear transformation is applied.

Having said all that, I dont really know anything about how these intervals are derived to say something more specific.

Helmut
★★★

Vienna, Austria,
2012-12-07 16:51
(4937 d 10:32 ago)

@ Jack
Posting: # 9691
Views: 10,120

PE outside its CI?!

Post reply

Hi Jack!

❝ In principle it is possible to have a confidence interval that does not include the point estimate that is still valid (i.e. has the correct coverage). It is, however, not something that would be considered a "good" confidence interval because the result of a hypothesis test would not necessarily match the conclusion based on a CI which obviously has all sort of flaws.

THX for the clarification from a Pro. Shows once more that common sense is not always applicable in science. ;-)

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2012-12-06 16:04
(4938 d 11:19 ago)

@ Helmut
Posting: # 9683
Views: 10,115

PE outside its CI!

Post reply

Dear Helmut!

❝ ... Now I’m lost. Like you I always thought the two-sided CI gives the range containing the expected value at 1–α/2. In the extreme cases of α=0 the CI should be -∞,+∞^*

❝ * For the normal distribution. Here $]0,+\infty]\ldots$

Agree. (Although I think this is no longer statistics)
This is what the formulas give.

CVCL(CV=0.3, df=10, side='upper',alpha=0)

lower CL upper CL 

       0      Inf 

CVCL(CV=0.3, df=10, side='2-sided',alpha=0)

lower CL upper CL 

       0      Inf

❝ and for α=1 CL_lo=PE=CL_hi. As our Indian friends use to say: Correct me if I’m wrong.

Do not agree. α=1 means that we err in 100% of cases. Means that in no case the confidence interval includes the population parameter of interest, mean, variance or whatever we are interested in.
For a variance >0 this can only achieved for the interval (0,0) if we talk one-sided upper.

Dunno a same argument for two-sided. Here we have to err in 50% of the cases where population value is > (≥?) PE and another 50% where population value is < (≤?) PE.

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2012-12-07 16:40
(4937 d 10:43 ago)

@ d_labes
Posting: # 9690
Views: 10,116

PE outside its CI!

Post reply

Dear Detlew!

❝ ❝ and for α=1 CL_lo=PE=CL_hi. As our Indian friends use to say: Correct me if I’m wrong.

❝ Do not agree. α=1 means that we err in 100% of cases. Means that in no case the confidence interval includes the population parameter of interest, mean, variance or whatever we are interested in.

Exactly. Maybe this is not statistics any more. Would you say there is a cut-off (e.g., 0.5) rendering a CI meaningless?

❝ For a variance >0 this can only achieved for the interval (0,0) if we talk one-sided upper.

To be honest my brain works two-sided only. I can work with one-sided CIs but never mastered a full understanding of them.
∞ somehow scares me (a case of horror vacui?)…

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro
★★★

Denmark,
2012-12-07 23:40
(4937 d 03:42 ago)

@ Helmut
Posting: # 9693
Views: 10,066

PE outside its CI!

Post reply

Meine Herren,

if I understand what a PE is and if I understand what a CI is then this thread is rather meaningless. I will be happy to conclude that then I probably don't understand these matters. I will even be happy to admit that I see no point in CIs that do not span over the PEs.
An alternative: It should be possible to use a Monte Carlo approach here. Simulate a gazillion trials, get the CVs and see within which limits they are distributed?
While we are at it: The formulae above involve chi-squares, but I do not see what the relationship is to the sample distribution of CVs, cf. possibly Hendricks and Robey 1936. Is it the same?
We are beyond my personal understanding of maths but shouldn't that paper be where this discussion should start? :-D

—
Pass or fail!
ElMaestro

d_labes
★★★

Berlin, Germany,
2012-12-09 17:20
(4935 d 10:03 ago)

@ ElMaestro
Posting: # 9698
Views: 10,208

CV of log-normal data

Post reply

Dear Ol'Pirate,

❝ if I understand what a PE is and if I understand what a CI is then this thread is rather meaningless. I will be happy to conclude that then I probably don't understand these matters. I will even be happy to admit that I see no point in CIs that do not span over the PEs.

And if you don't understand?

❝ While we are at it: The formulae above involve chi-squares, but I do not see what the relationship is to the sample distribution of CVs, cf. possibly Hendricks and Robey 1936. Is it the same?

❝ We are beyond my personal understanding of maths but shouldn't that paper be where this discussion should start? :-D

No definitely not. IMHO your reference deals with the conventional CV i.e sd/µ but we here are dealing with the CV of log-normal data. That means we are dealing basically with the distribution of the sampling variance. And this distribution and CIs based on it are already described in basic statistic textbooks.

Astonishing enough these long and well-known CIs have the above discussed feature. But notice: only if you set the error probability to values >0.5. No one will do that in practice of course. Thus you are totally right: This thread is rather meaningless practically. In the same sense as this thread which was kicked off by an well known nitpicker :cool:

.

But beside the original simple aim to validate some code it may be one can learn something about such basic statistics if one pushes them at their extreme.

—
Regards,

Detlew

d_labes ★★★ Berlin, Germany, 2012-12-10 09:25 (4934 d 17:58 ago) @ Helmut Posting: # 9700 Views: 10,036	Two-sided brain Post reply
	Dear Helmut! ❝ To be honest my brain works two-sided only. I can work with one-sided CIs but never mastered a full understanding of them. One of best justification of a two-sided test/CI ... I have ever seen . And this from a guy which works nearly all the days with TOST. — Regards, Detlew

Helmut
★★★

Vienna, Austria,
2012-12-11 15:54
(4933 d 11:29 ago)

@ d_labes
Posting: # 9711
Views: 10,299

Operationally equivalent…

Post reply

Dear Detlew!

❝ ❝ To be honest my brain works two-sided only. […]

❝

❝ One of best justification of a two-sided test/CI ... I have ever seen :-D .

❝ And this from a guy which works nearly all the days with TOST.

No, I don’t. Only in less than a handful of studies 25 years ago – upon sponsor’s wish (“Hey, that’s new = fashionable and suggested by an FDA guy!”) – I applied TOST, i.e., reported a result like:

p(<0.8) 0.0014 ʌ p(>1.25) 0.0034 [image]

When I started in the early 1980s (without knowing any literature at all) I naïvely calculated a two-sided 95% CI. Later I adapted for the 90% CI, though the term “operationally equivalent” to TOST gave me headaches. I wouldn’t go that far calling it just “an algebraic coincidence”¹ but I don’t get the point why the straightforward confidence interval inclusion (one H₀ and H₁)
$$H_0:\frac{\mu_T}{\mu_R}\notin [\theta_1,\theta_2]\;\mathit{vs.}\;H_1:\theta_1<\frac{\mu_T}{\mu_R}<\theta_2$$
should be replaced by a {set} of two Nulls and Alternatives,
$$H_{0L}:\frac{\mu_T}{\mu_R}\leq \theta_1\;{vs.}\;H_{1L}:\frac{\mu_T}{\mu_R}>\theta_1$$
$$H_{0U}:\frac{\mu_T}{\mu_R}\geq \theta_1\;{vs.}\;H_{1U}:\frac{\mu_T}{\mu_R}<\theta_1$$
tested on the same body of data.

For an overview see Berger & Hsu (including comments by Hauck, Anderson, Meredith, Heise, Liu, Chow, Schuirmann, and Hwang).² ;-)

Brown LD, Casella G, Hwang JTG. Optimal Confidence Sets, Bioequivalence, and the Limaçon of Pascal. J Amer Statist Assoc. 1995;90(431):880–9. doi:10.1080/01621459.1995.10476587. free resource.
Berger RL, Hsu JC. Bioequivalence Trials, Intersection–Union Tests and Equivalence Confidence Sets. Stat Sci. 1996;11(4):283–319. doi:10.1214/ss/1032280304. free resource.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro
★★★

Denmark,
2012-12-11 16:41
(4933 d 10:42 ago)

@ Helmut
Posting: # 9712
Views: 9,968

Austrian logic?

Post reply

Hi Helmut, you illogical caveman,

❝ To be honest my brain works two-sided only. I can work with one-sided CIs but never mastered a full understanding of them.

❝ ∞ somehow scares me (a case of horror vacui?)…

If you think one one-sided test is difficult to work with, then how come you find two one-sided tests easier?? Two plus two equals four, at least that's what I got last time I used Excel.

—
Pass or fail!
ElMaestro

Helmut
★★★

Vienna, Austria,
2012-12-11 17:12
(4933 d 10:10 ago)

@ ElMaestro
Posting: # 9713
Views: 10,054

Fuzzy logic

Post reply

Hi ElMaestro!

❝ […] you illogical caveman,

How did you know?

[image]

❝ If you think one one-sided test is difficult to work with, then how come you find two one-sided tests easier??

Didn’t say that. Outing myself a statistical ignorant: I find one two-sided test easier.

❝ Two plus two equals four, at least that's what I got last time I used Excel.

That’s a professional deformation I owe to my engineering background. I’m fine with statements like “about two plus about two gives roughly four”. If we know how large the two “abouts” are we can even state “four ± somefink”.

[image]

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

CV point estimate outside CI [General Sta­tis­tics]

What the heck?

Code validated

PE outside its CI?!

PE outside its CI?!

PE outside its CI?!

PE outside its CI!

PE outside its CI!

PE outside its CI!

CV of log-normal data

Two-sided brain

Operationally equivalent…

Austrian logic?

Fuzzy logic

CV point estimate outside CI [General Statistics]