two different approaches [Nonparametrics]
dear d_labes and HS !
I included two additional bootstrap based approaches implemented in R package
the empirical coverage of
best regards
martin
PS.: you can interpret the empirical coverage as a proportion based on a sample size of
I included two additional bootstrap based approaches implemented in R package
pairwiseCI
where investigation of empirical coverage of bootstrap based CIs is a real stress test (7 hours)set.seed(150910)
n <- 10
nsim <- 1E4
conf.level <- 0.9
cover <- rep(0,4)
grp <- as.factor(c(rep('A', n), rep('B', n)))
for(i in 1:nsim){
timeForecast(i, nsim)
x <- round(rnorm(2*n, mean=0, sd=1),1)
res1 <- confint(wilcox_test(x~grp, distribution='exact',
conf.int=TRUE, conf.level=conf.level))$conf.int
res2 <- wilcox.exact(x~grp, conf.int=TRUE,
conf.level=conf.level)$conf.int
res3 <- Median.diff(x=x[1:n], y=x[(n+1):(2*n)],
conf.level=conf.level)$conf.int
res4 <- HD.diff(x=x[1:n], y=x[(n+1):(2*n)],
conf.level=conf.level)$conf.int
if(res1[1] <= 0 & res1[2] >= 0){cover[1] <- cover[1] + 1}
if(res2[1] <= 0 & res2[2] >= 0){cover[2] <- cover[2] + 1}
if(res3[1] <= 0 & res3[2] >= 0){cover[3] <- cover[3] + 1}
if(res4[1] <= 0 & res4[2] >= 0){cover[4] <- cover[4] + 1}
}
-> estimated simulation time <h:mm:ss> (done / all): 0:00:00 / -:--:--
-:--:-- / 7:02:49
print(cover/nsim)
[1] 0.9283 0.9372 0.9319 0.9053
the empirical coverage of
0.9053
for the difference of Harrell-Davis estimates for location looks nice. best regards
martin
PS.: you can interpret the empirical coverage as a proportion based on a sample size of
nsim
enabling to calculate a CI enabling to assess simulation accuracybinom.test(x=0.9053*nsim, n=nsim, p=0.9)
Exact binomial test
data: 0.9053 * nsim and nsim
number of successes = 9053, number of trials = 10000, p-value = 0.07729
alternative hypothesis: true probability of success is not equal to 0.9
95 percent confidence interval:
0.8993926 0.9109701
sample estimates:
probability of success
0.9053
Complete thread:
- tmax in case of ties: R vs. R vs. SAS d_labes 2010-09-10 12:05 [Nonparametrics]
- tmax in case of ties: StatXact, Phoenix, and... Helmut 2010-09-10 18:28
- Ties, no ties, ties, no ties ... d_labes 2010-09-13 13:21
- simulation martin 2010-09-14 08:33
- simulants of the world unite d_labes 2010-09-15 10:14
- simulants of the world unite martin 2010-09-15 14:53
- simulants of the world unite Helmut 2010-09-15 16:17
- simul ants with no ties d_labes 2010-09-16 11:19
- simul ants with no ties Helmut 2010-09-16 14:01
- two different approachesmartin 2010-09-16 15:35
- simul ants with no ties Helmut 2010-09-16 14:01
- simul ants with no ties d_labes 2010-09-16 11:19
- simulants of the world unite Helmut 2010-09-15 16:17
- simulants of the world unite martin 2010-09-15 14:53
- simulants of the world unite d_labes 2010-09-15 10:14
- simulation martin 2010-09-14 08:33
- Ties, no ties, ties, no ties ... d_labes 2010-09-13 13:21
- evaluation of tmax: use of relative effects? martin 2010-09-16 18:04
- Not positive about that Helmut 2010-09-16 19:52
- Not positive about that martin 2010-09-16 21:00
- Stupidity Helmut 2010-09-17 13:10
- Not positive about that martin 2010-09-16 21:00
- Not positive about that Helmut 2010-09-16 19:52
- tmax in case of ties: R vs. R vs. SAS Jack 2010-09-20 14:03
- R packages Helmut 2010-09-20 14:20
- R packages code d_labes 2010-09-27 11:27
- R packages Jack 2010-09-27 16:14
- Is Exact exact? d_labes 2010-09-27 09:53
- R packages Helmut 2010-09-20 14:20
- tmax in case of ties: StatXact, Phoenix, and... Helmut 2010-09-10 18:28