ElMaestro
★★★

Belgium?,
2019-01-07 14:52
(578 d 01:59 ago)

Posting: # 19744
Views: 2,889

confint and adjust in R [General Sta­tis­tics]

Hi all,

happy new year.

I am exploring the confint function in R and the oppotunities for multiplicity adjustment.
In fact, the confint function allows an adjust argument :

In my current models I can apply adjust= e.g. "bonferroni", "none", "sidak", "mvt" and "tukey" (which is the default).

I cannot, however, find any documentation which tells me what the options really are; the ones above are only the options I found by googling, and finding pages like this one.
??adjust gets me nowhere in practice.

I emphasize that adjust  under confint  is not the same as p.adjust which is documented here, but naïvely I thought the arguments would be the same.
If I specify adjust="holm" or adjust="hochberg" it defaults to Bonferroni in models I am looking at, so either these methods are not implemented for confint or they achieve the same in my models, not sure. If I do adjust="eat.my.shorts" etc then I don't get an error but just bonferroni

Note that the adjust argument is one of the "..."'s so isn't part of the function documentation. It must be mapped out somewhere???

So... Can anyone point me to some online documentation which tells which options the adjust argument allows? Or can any tell me how how I could systematically look up this type of question generally?

I could be wrong, but...

Best regards,
ElMaestro

"Pass or fail" (D. Potvin et al., 2008)
d_labes
★★★

Berlin, Germany,
2019-01-07 17:02
(577 d 23:49 ago)

@ ElMaestro
Posting: # 19749
Views: 2,570

Dear ElMaestro,

» I am exploring the confint function in R and the oppotunities for multiplicity adjustment.
» In fact, the confint function allows an adjust argument :
»
» In my current models I can apply adjust= e.g. "bonferroni", "none", "sidak", "mvt" and "tukey" (which is the default).

Sorry, but AFAIK the default confint method or that for lm() objects has no adjust argument, or more precisely spoken accepts any string for adjust= but without any effect. Example:
> fit <- lm(100/mpg ~ disp + hp + wt + am, data = mtcars) > confint(fit)                    2.5 %      97.5 % (Intercept) -0.774822875 2.256118188 disp        -0.002867999 0.008273849 hp          -0.001400580 0.011949674 wt           0.380088737 1.622517536 am          -0.614677730 0.926307310 > confint(fit, adjust="eat.my.shorts")                    2.5 %      97.5 % (Intercept) -0.774822875 2.256118188 disp        -0.002867999 0.008273849 hp          -0.001400580 0.011949674 wt           0.380088737 1.622517536 am          -0.614677730 0.926307310
There is no error thrown because confint() has an ... argument, for which there is no error check available.

What you describe must be an effect of using an add-on package with comes with the possibility to consider multiplicity in calculation of confidence intervals.
If you are lucky, the documentation of that package will contain details of what the adjust= argument will do. Look into the ./doc sub-directory of the package and look for vignettes or other documentation material.
But here I must say that being lucky is not so often the case in R. Many authors of packages assume that you have sufficient knowledge of the statistical methods underlying a package they create. Eventually you may figure out whats going on if you look at the code of the confint.class() function, where class is the model class, defined in the package you uses.

Regards,

Detlew
ElMaestro
★★★

Belgium?,
2019-01-07 18:12
(577 d 22:38 ago)

@ d_labes
Posting: # 19754
Views: 2,561

Dear d_labes,

» What you describe must be an effect of using an add-on package with comes with the possibility to consider multiplicity in calculation of confidence intervals.

Thanks a lot for commenting

Can you try this:

 library("lsmeans") set.seed(12341) v1=c(rep("A",10),rep( "B",10), rep( "C", 10)) v2=c(rep("Q",10),rep( "W",10), rep( "E",10)) Foo=sample(v1) Bar=sample(v2) y=runif(30) M=lm(y~factor(Foo)+factor(Bar)) confint(pairs(lsmeans(M, "Foo")), adjust="none") confint(pairs(lsmeans(M, "Foo")), adjust="bonferroni") confint(pairs(lsmeans(M, "Foo")), adjust="eat.my.shorts") confint(pairs(lsmeans(M, "Foo"))) # HAPPY NEW YEAR 

It gives four different results here. You can see adjust is passed as an argument to the confint function so can't see how it would be part of lsmeans or pairs args.

I could be wrong, but...

Best regards,
ElMaestro

"Pass or fail" (D. Potvin et al., 2008)
d_labes
★★★

Berlin, Germany,
2019-01-07 19:19
(577 d 21:31 ago)

@ ElMaestro
Posting: # 19756
Views: 2,559

Dear Anders!

» » library("lsmeans")
I see that my suspicion was true. adjust= is an argument only available with package lsmeans (or another package overriding the behavior of confint() for other classes then "lm".
» ...
» confint(pairs(lsmeans(M, "Foo")))
» # HAPPY NEW YEAR [/code]
»
» It gives four different results here. You can see adjust is passed as an argument to the confint function so can't see how it would be part of lsmeans or pairs args.

Correct concerning four different results.
adjust="eat.my.shorts" is outstanding because this "method" isn't implemented but transferred via "..." and thus doesn't underlay any error checks. To react on this undesirable feature the programmer decided to set adjust= in such cases to adjust="bonferroni". Reasonable. Or?
This behavior is described in the man page of lsmeans/pairs or summary.emmGrid.
But adjust= is part of package lsmeans. Reason: class(pairs(lsmeans(M, "Foo"))).
This results in calling confint.emmGrid() with the possibility of reacting on multiplicity adjusted CI's instead of confint.lm() without the possibilty of multiplicity adjustments if you call confint().

Regards,

Detlew
ElMaestro
★★★

Belgium?,
2019-01-07 22:22
(577 d 18:28 ago)

@ d_labes
Posting: # 19757
Views: 2,585

Dear d_labes,

» adjust="eat.my.shorts" is outstanding because this "method" isn't implemented but transferred via "..." and thus doesn't underlay any error checks. To react on this undesirable feature the programmer decided to set adjust= in such cases to adjust="bonferroni". Reasonable. Or?
» This behavior is described in the man page of lsmeans/pairs or summary.emmGrid.

Thanks for the explanation; I had no idea R worked this way.
And you are exactly right the options are listed in the emmeans documentation (added value for me: it seems I must kiss the lsmeans package goodbye and install emmeans in stead).

Whether or not "eat.my.shorts" should result in "bonferroni" or the default "tukey" or something else is probably a matter of personal taste. I don't think I have a qualified opinion on that aspect and it is not a crucial matter at all for me and probably for anyone on the planet.

I am just very delighted I got an answer to exactly the questions that baffled me. Thanks for the help. Muchas gracias.

I could be wrong, but...

Best regards,
ElMaestro

"Pass or fail" (D. Potvin et al., 2008)