simulants of the world unite [Nonparametrics]
Dear Martin!
Wow! Really interesting. And unbelievable
.
I was told that the Wilcoxon rank sum test is due to the discrete nature of the permutation distribution function always to some degree conservative (achievable confidence with n1, n2=10 is 0.956743).
And here it comes out that the implementation in the package coin is distinct anti-conservative! And the implementation in wilcox.exact does not show signs of conservativeness.
I have one small doubt. I'm not quite sure but should we not include the CI borders in counting the coverage due to the definition of the CI?
If we do this one gets (only 5000 sims, see BTW below):
Moreover there seems to be some interaction between both packages. If I simulate without coin I get coverage of wilcox.exact = 0.9688 (including the CI borders in counting coverage).
BTW: Warning to all! The code is really a stress-test for the CPU of your computer (at least for mine). Submit the code and go for one and another beers. Next morning the result is astonishing enough
.
Therefore I could answer not until today.
❝ you may find the following simulation results interesting
❝ ...
❝ print(cover1/nsim)
❝ [1] 0.937 # coverage of wilcox_test (coin)
❝ print(cover2/nsim)
❝ [1] 0.9505 # coverage of wilcox.exact (exactRankTests)
Wow! Really interesting. And unbelievable

I was told that the Wilcoxon rank sum test is due to the discrete nature of the permutation distribution function always to some degree conservative (achievable confidence with n1, n2=10 is 0.956743).
And here it comes out that the implementation in the package coin is distinct anti-conservative! And the implementation in wilcox.exact does not show signs of conservativeness.
I have one small doubt. I'm not quite sure but should we not include the CI borders in counting the coverage due to the definition of the CI?
If we do this one gets (only 5000 sims, see BTW below):
[1] 0.9608 # coverage of wilcox_test (coin)
[1] 0.9608 # coverage of wilcox.exact (exactRankTests)
Moreover there seems to be some interaction between both packages. If I simulate without coin I get coverage of wilcox.exact = 0.9688 (including the CI borders in counting coverage).
BTW: Warning to all! The code is really a stress-test for the CPU of your computer (at least for mine). Submit the code and go for one and another beers. Next morning the result is astonishing enough

Therefore I could answer not until today.
—
Regards,
Detlew
Regards,
Detlew
Complete thread:
- tmax in case of ties: R vs. R vs. SAS d_labes 2010-09-10 12:05 [Nonparametrics]
- tmax in case of ties: StatXact, Phoenix, and... Helmut 2010-09-10 18:28
- Ties, no ties, ties, no ties ... d_labes 2010-09-13 13:21
- simulation martin 2010-09-14 08:33
- simulants of the world united_labes 2010-09-15 10:14
- simulants of the world unite martin 2010-09-15 14:53
- simulants of the world unite Helmut 2010-09-15 16:17
- simul ants with no ties d_labes 2010-09-16 11:19
- simul ants with no ties Helmut 2010-09-16 14:01
- two different approaches martin 2010-09-16 15:35
- simul ants with no ties Helmut 2010-09-16 14:01
- simul ants with no ties d_labes 2010-09-16 11:19
- simulants of the world unite Helmut 2010-09-15 16:17
- simulants of the world unite martin 2010-09-15 14:53
- simulants of the world united_labes 2010-09-15 10:14
- simulation martin 2010-09-14 08:33
- Ties, no ties, ties, no ties ... d_labes 2010-09-13 13:21
- evaluation of tmax: use of relative effects? martin 2010-09-16 18:04
- Not positive about that Helmut 2010-09-16 19:52
- Not positive about that martin 2010-09-16 21:00
- Stupidity Helmut 2010-09-17 13:10
- Not positive about that martin 2010-09-16 21:00
- Not positive about that Helmut 2010-09-16 19:52
- tmax in case of ties: R vs. R vs. SAS Jack 2010-09-20 14:03
- R packages Helmut 2010-09-20 14:20
- R packages code d_labes 2010-09-27 11:27
- R packages Jack 2010-09-27 16:14
- Is Exact exact? d_labes 2010-09-27 09:53
- R packages Helmut 2010-09-20 14:20
- tmax in case of ties: StatXact, Phoenix, and... Helmut 2010-09-10 18:28