Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-30 17:36

Posting: # 7559
Views: 6,466
 

 Two-stage (any method) and multiplicity [General Sta­tis­tics]

Dear all!

Here comes the bonus-question for hard-core-simulants: How to deal with α in a dose-proportionality study when a two-stage design is to be planned? In the past I have used a Bonferroni-correction (see this post), which is
  1. conservative and
  2. does not require independent tests (like e.g. the Šidák-correction).
Since in dose-proportionality (aside from setting up a power model) we often compare dose-adjusted responses to one dose-level (and do not perform all pairwise tests) a variant of Dunnett’s test might be suitable. Unfortunately Dunnett is only applicable for nominal scales, not for continuous ones (doses).

Opinions, ideas?

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Belgium?,
2011-10-30 18:26

@ Helmut
Posting: # 7560
Views: 5,727
 

 Two-stage (any method) and multiplicity

Dear HS,

» Opinions, ideas?

There are quite a few SPC's out there which state dose proportionality or dose linearity and where the originator as far as I know hasn't really done that fancy stuff but just administered a series of doses and checked AUC or Cmax or Cl. That's good enough isn't it.
At least for EU, I guess part of the story is to live with the BE-guideline's requirement "Assessment of linearity will consider whether differences in dose-adjusted AUC meet a criterion of ± 25%." which can be dealt with very simply.
Perhaps your angle is different and that where the complexity comes in? I am eager to hear.

Btw, in spite of the links to previous definitions and discussions I still think we need some useful definitions, preferably in guidelines. Cmax etc vs dose being a straight line, I am perfectly willing to call that linearity; Wikipedia does an excellent job at confusing readers.

I could be wrong, but...

Best regards,
ElMaestro

"Pass or fail" (D. Potvin et al., 2008)
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-11-02 22:30

@ ElMaestro
Posting: # 7590
Views: 5,677
 

 Dose-dependent PK

Dear ElMaestro!

» At least for EU, I guess part of the story is to live with the BE-guideline's requirement "Assessment of linearity will consider whether differences in dose-adjusted AUC meet a criterion of ± 25%." which can be dealt with very simply.
» Perhaps your angle is different and that where the complexity comes in? I am eager to hear.

Yes. I’m aiming at a confirmatory study – not the stuff mentioned in the GL (“If you come up with some serious writing about the drug’s PK from the Library of Alexandria – whether it might be on parchment or papyrus – we accept that and you have to perform the study on the highest dose only.”)

» […] Cmax etc vs dose being a straight line, I am perfectly willing to call that linearity;

I’m not happy with assessing dose-proportionality based on Cmax. It’s a f**g composite metric.
I prefer AUC by far.

» Wikipedia does an excellent job at confusing readers.

Yeah, that’s funny reading matter. Should read the talk-page as well.

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-11-01 11:11

@ Helmut
Posting: # 7574
Views: 5,747
 

 Dunnett not for continuous scales? Really?

Dear Helmut!

» ... Since in dose-proportionality (aside from setting up a power model) we often compare dose-adjusted responses to one dose-level (and do not perform all pairwise tests) a variant of Dunnett’s test might be suitable. Unfortunately Dunnett is only applicable for nominal scales, not for continuous ones (doses).
(emphasis by me)

Where does this opinion come from :confused:. AFAIK is Dunnett's test a post-hoc test within the ANOVA framework to compare many means to one control. ANOVA always deals with measurements on continuous (metric) scales. Or do I miss somefink here?

Moreover Hauschke, Steinijans and Pigeot [1] explicitly recommend Dunnett's test for evaluation of studies with more than one Test formulations versus one reference. See Chapter 7.

For dose linearity studies (comparing more then 2 dose adjusted PK characteristics) they derive from the intersection-union principle "... Hence for a joint decision rule where all requirements must be fulfilled, no adjustment of the comparison wise type I error is needed ...". See page 170 of the reference. The argumentation given is plausible for me also as an amateur in statistics I'm not able to prove it.

BTW: Where does the 2-stage design come into play for dose-proportionality studies?


[1] Hauschke, Steinijans and Pigeot
Bioequivalence Studies in Drug Development
Wiley, Chichester 2007

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-11-02 22:18

@ d_labes
Posting: # 7589
Views: 5,759
 

 Yes, yes – but another construction site

Dear Detlew!

» » […] Unfortunately Dunnett is only applicable for nominal scales, not for continuous ones (doses).
» (emphasis by me)
» Where does this opinion come from :confused:. AFAIK is Dunnett's test a post-hoc test within the ANOVA framework to compare many means to one control. ANOVA always deals with measurements on continuous (metric) scales. Or do I miss somefink here?

No, no – sorry. Should have done my homework before. I had in mind that Dunnett is applicable if a continuous variable if tested based on an ordered (nominal) covariate. You are right.

» Moreover Hauschke, Steinijans and Pigeot […]. See page 170 of the reference. The argumentation given is plausible for me also as an amateur in statistics I'm not able to prove it.

Yes, sounds good for another amateur. Now for the big but (p170–171):

[…] no adjustment of the comparisonwise type I error is needed to keep the familywise type I error under control. However, this intersection-union testing procedure inflates the type II error, that is the probability of erroneously failing to reject at least one of the null-hypotheses. This inflation has to be taken into account by an adequate sample size determination.
(my emphasis)

Your reference lead my again to CPMP’s ’Points to Consider on Multiplicity Issues in Clinical Trials’ (2002). My gut-feeling seems to be right at the end: more comparisons = less power. ‘Adequate sample size determination’ smells like the respective section in the BE GL. Let’s look at the CPMP document:

This procedure inflates the relevant type II error […], which in the worst case scenario is the sum of the type II error errors connected with the individual hypotheses.

Oh wow!

» BTW: Where does the 2-stage design come into play for dose-proportionality studies?

For some background see the end of this post. I have the frivolous idea to plan for a similar study. If I want to go with the worst-case scenario (3 simultaneous comparisons, each at β 0.20/3) in my specific case (T/R 0.95, CV 20%) I would end up with a sample size of 28 instead of 20 in a single comparison with 80% power. +40% penalty. ;-)

BTW: Can you please check the new requirements for R-packages? Since the update to R 2.14.0 the help-files of PowerTOST are not accessible any more. Has something to do with a missing vignette / base URL?

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-11-03 13:18

@ Helmut
Posting: # 7593
Views: 5,638
 

 PowerTOST help

Dear Helmut!

» BTW: Can you please check the new requirements for R-packages? Since the update to R 2.14.0 the help-files of PowerTOST are not accessible any more. Has something to do with a missing vignette / base URL?

Sorry for the inconvenience. It had to do with a violation of name conventions which was not fully documented. After a hint from Prof. Ripleys about that I had created a new version 0.8-7 last week which should meanwhile be accessible via CRAN.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-11-03 14:28

@ d_labes
Posting: # 7594
Views: 5,622
 

 PowerTOST help (solved)

Dear Detlew!

» Sorry for the inconvenience. It had to do with a violation of name conventions which was not fully documented. After a hint from Prof. Ripleys about that I had created a new version 0.8-7 last week which should meanwhile be accessible via CRAN.

Sorry for the confusion - forget it. Seems that after installing 2.14.0 and update.packages(checkBuilt=TRUE, ask=FALSE) the local HTTPD-server had hiccups. After a reboot the help system of 0.8-7 is working like a charm. Same on my notebook, where I have 0.8-7 on 2.13.2 installed.

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
martin
★★  

Austria,
2011-11-03 22:28
(edited by martin on 2011-11-04 14:16)

@ Helmut
Posting: # 7600
Views: 5,759
 

 intersection-union test

Dear HS!

A good „introduction“ to the intersection-union test (IUT) for hypothesis of equivalence is given in chapter 7.1 in Wellek (2010). I set up a simulation study to evaluate the power and the type I error when testing 3 hypothesis of equivalence simultaneously based on the IUT using a common CV and log-normal distributed random variables. I used identical magical margins (i.e. 0.8 to 1.25) for all three comparisons. I evaluated the power (2 scenarios) and the type I error (2 scenarios) for a parallel group design. You may find the code useful.

library(gmodels)
set.seed(645326)

simfun <- function(mue, n, cv, nsim){

   grp <- factor(c(rep('A', n[1]), rep('B', n[2]), rep('C', n[3])))
   count <- 0

   for(i in 1:nsim){
      x <- c(rlnorm(n=n[1], meanlog=log(mue[1])-0.5*log(cv^2+1), sdlog=sqrt(log(cv^2+1))),
             rlnorm(n=n[2], meanlog=log(mue[2])-0.5*log(cv^2+1), sdlog=sqrt(log(cv^2+1))),
             rlnorm(n=n[3], meanlog=log(mue[3])-0.5*log(cv^2+1), sdlog=sqrt(log(cv^2+1))))

      # contrasts of interest obtained by an ANOVA model using the pooled SE
      mod <- lm(log(x) ~ 0 + grp)
      res1 <- exp(estimable(mod, c(1,-1,0), conf.int=0.9)[c(6,7)])
      res2 <- exp(estimable(mod, c(1,0,-1), conf.int=0.9)[c(6,7)])
      res3 <- exp(estimable(mod, c(0,1,-1), conf.int=0.9)[c(6,7)])
 
      if(res1[1] >= 0.8 & res1[2] <= 1.25 &
         res2[1] >= 0.8 & res2[2] <= 1.25 &
         res3[1] >= 0.8 & res3[2] <= 1.25){count <- count + 1}

    }
    return(count/nsim)
}

# set parameters for simulation
n <- c(20,20,20)
cv <- 0.2
nsim <- 1E3

# equivalent scenarios: probability for showing overall equivalence
simfun(mue=c(1, 1, 1), n=n, cv=cv, nsim=nsim)
simfun(mue=c(1.05, 1.05, 1), n=n, cv=cv, nsim=nsim)

# inequivalent scenarios: probability for erroneously showing overall equivalence
simfun(mue=c(1, 1, 0.8-1E-16), n=n, cv=cv, nsim=nsim)
simfun(mue=c(1, 1.1251, 0.9), n=n, cv=cv, nsim=nsim)


best regards

Martin

PS.: There are different definitions of power available when comparing more than 2 groups. I simulated the power for claiming equivalence for all three comparisons (i.e. overall equivalence) suitable for the IUT.

PPS.: I implemented the confidence interval inclusion approach but IMHO a SOST (six one-sided tests ;-)) approach should also be applicable.
martin
★★  

Austria,
2011-11-05 23:43
(edited by martin on 2011-11-06 11:04)

@ martin
Posting: # 7615
Views: 5,634
 

 IUT and Dunnett: code for comparison of power

Dear HS !

Please find enclosed a modified (rather slow) code for evaluation of power and type I error for Dunnett and IUT for 2-to-1 comparisons (i.e. B vs. A and C vs. A) providing:
  • Probability for showing equivalence in both comparisons with IUT
  • Probability for showing equivalence in both comparisons with Dunnett
  • Probability for showing equivalence in at least one comparison with Dunnett
My summary:
  • When you are interested in showing equivalence in both comparisons use IUT
  • When you are interested in showing equivalence in at least on comparison use Dunnett with the drawback of a loss in power regarding showing equivalence in both comparisons.
best regards

Martin

library(multcomp)
library(gmodels)
set.seed(24354)

simfun <- function(mue, n, cv, nsim){

    # 2-to-1 comparisons: B vs. A and C vs. A
    grp <- factor(c(rep('A', n[1]), rep('B', n[2]), rep('C', n[3])))
    count1 <- 0; count2 <- 0; count3 <- 0

    # define logical variables necessary for evaluating the probability
    # for showing equivalence in at least one comparison
    comp1 <- TRUE; comp2 <- TRUE; scenario <- 'NA'
    if(mue[2]/mue[1] < 0.8 | mue[2]/mue[1] > 1.25){comp1 <- FALSE} # comparison 1 is true inequivalent
    if(mue[3]/mue[1] < 0.8 | mue[3]/mue[1] > 1.25){comp2 <- FALSE} # comparison 2 is true inequivalent
    if((comp1==TRUE & comp2==TRUE) | (comp1==FALSE & comp2==FALSE)){scenario <- '0'}
    if(comp1==FALSE & comp2==TRUE){scenario <- '1'}
    if(comp1==TRUE & comp2==FALSE){scenario <- '2'}
    print(scenario)
 
    for(i in 1:nsim){
       x <- c(rlnorm(n=n[1], meanlog=log(mue[1])-0.5*log(cv^2+1), sdlog=sqrt(log(cv^2+1))),
              rlnorm(n=n[2], meanlog=log(mue[2])-0.5*log(cv^2+1), sdlog=sqrt(log(cv^2+1))),
              rlnorm(n=n[3], meanlog=log(mue[3])-0.5*log(cv^2+1), sdlog=sqrt(log(cv^2+1))))

        # contrasts of interest: ANOVA model to get unadjusted many-to-one comparisons
        mod.raw <- lm(log(x) ~ 0 + grp)
        res1.raw <- exp(estimable(mod.raw, c(-1,1,0), conf.int=0.9)[c(6,7)])
        res2.raw <- exp(estimable(mod.raw, c(-1,0,1), conf.int=0.9)[c(6,7)])

        # contrasts of interest: ANOVA model to get adjusted many-to-one comparisons using Dunnett
        mod.dun <- summary(glht(aov(log(x) ~ grp), linfct=mcp(grp='Dunnett')))
        res1.dun <- exp(confint(mod.dun, level=0.9)$confint[1,c(2,3)])
        res2.dun <- exp(confint(mod.dun, level=0.9)$confint[2,c(2,3)])

        # IUT: probability for equivalence in both comparisons
        if(res1.raw[1] >= 0.8 & res1.raw[2] <= 1.25 &
           res2.raw[1] >= 0.8 & res2.raw[2] <= 1.25){count1 <- count1 + 1}

        # Dunnett: probability for equivalence in both comparisons
        if(res1.dun[1] >= 0.8 & res1.dun[2] <= 1.25 &
           res2.dun[1] >= 0.8 & res2.dun[2] <= 1.25){count2 <- count2 + 1}
       
        # Dunnett: probability for equivalence in at least one comparison
        switch(scenario,
           '0' = {if((res1.dun[1] >= 0.8 & res1.dun[2] <= 1.25) |
                     (res2.dun[1] >= 0.8 & res2.dun[2] <= 1.25)){count3 <- count3 + 1}
        }, '1' = {if(res1.dun[1] >= 0.8 & res1.dun[2] <= 1.25){count3 <- count3 + 1}
        }, '2' = {if(res2.dun[1] >= 0.8 & res2.dun[2] <= 1.25){count3 <- count3 + 1}
        })
     }
     res <- c(count1/nsim, count2/nsim, count3/nsim)
     names(res) <- c('IUT: overall', 'Dunnett: overall', 'Dunnett: at least one')
     return(res)
 }

 # define simulation parameters
 n <- c(20,20,20)
 cv <- 0.2
 nsim <- 1E3

 ## 2 equivalent scenarios: power
 simfun(mue=c(1, 1, 1), n=n, cv=cv, nsim=nsim)
 simfun(mue=c(1, 1.05, 1.05), n=n, cv=cv, nsim=nsim)

 ## 3 inequivalent scenarios: type I error
 # A and B & A and C are inequivalent
 simfun(mue=c(1, 0.8-1E-16, 0.8-1E-16), n=n, cv=cv, nsim=nsim)

 # A and B are equivalent & A and C are inequivalent
 simfun(mue=c(1, 0.8, 0.8-1E-16), n=n, cv=cv, nsim=nsim)

 # A and B are inequivalent & A and C are equivalent
 simfun(mue=c(1, 0.8-1E-16, 0.8), n=n, cv=cv, nsim=nsim)
Activity
 Admin contact
20,809 posts in 4,355 threads, 1,447 registered users;
online 26 (0 registered, 26 guests [including 12 identified bots]).
Forum time: 17:24 CEST (Europe/Vienna)

Medical statistician: One who will not accept that Columbus discovered America…
because he said he was looking for India in the trial plan.    Stephen Senn

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5