d_labes
★★★

Berlin, Germany,
2010-04-07 12:04
(5551 d 16:29 ago)

Posting: # 5029
Views: 7,841
 

 Power and sample size with 'uncertain' CV [Power / Sample Size]

Stephen Senn in Statistical issues in drug development: "Power: That which statisticians are always calculating but never have."
Dear All!

As discussed frequently here in the forum the CV used in the formulas / software for sample size estimation in BE studies are usually not "carved in stone" like Moses ten commandments (to cite the famous admin of this forum ;-)) but known to us only with less or more imprecision as estimated CV with some degrees of freedom from one or more (pilot) studies done in the past or taken from the literature.

Helmut has described in his famous lectures how to combine such CV's and how to get an one-sided upper limit of the variability. With that upper limit one can then do a sensitivity analysis of the power / sample size.

Just discovered a paper and a book written by the well-known S.A. Julious

S. A. Julious, R. J. Owen
"Sample size calculations for clinical studies allowing for uncertainty about the variance"
Pharmaceutical Statistics, Vol 5(1), 29 - 37 (2006)

Steven A. Julious
"Sample Sizes for Clinical Trials"
CRC press, Chapman/Hall, Boca Raton - London - New York
© 2010 by Taylor and Francis Group, LLC
ISBN 978-1-58488-739-3

The latter had in chapter 7 "Sample Size Calculations for Bioequivalence Trials" - 7.2.6 or 7.3.2 "Calculations Taking Account of the Imprecision of the Variance Used in the Sample Size Calculations" some handy formulas how to calculate (expected) power taking into account the uncertainty of the variance estimate. Covered are cross-over studies and parallel-group design.

These formulas, approximate of course although not so named in the book, rely on the non-central t-distribution and are not so hard to code in any statistical software which has that distribution at hand, f.i. in R.

Using the design helper functions of my super-duper code here now the additions for the Julious formulas:

# approximate "expected" power according to Joulious book
# taking into account the uncertainty of a se estimated with dfse degrees of freedom
# raw function: see the call in the next function

.exp.power.TOST <- function(alpha=0.05, ltheta1, ltheta2, diffm,
                            se, dfse, n, df, bk=2)
{
  tval   <- qt(1 - alpha, df, lower.tail = TRUE)
  d1 <- sqrt(n*(diffm-ltheta1)^2/(bk*se^2))
  d2 <- sqrt(n*(diffm-ltheta2)^2/(bk*se^2))
  # Julious formula(s), the rest of code is decoration
  pow <- pt(d1,dfse,tval) + pt(d2,dfse,tval) - 1

  return(pow)
 
}
# --------------------------------------------------------------------
# "Expected" power according to Julious
# See known.designs() for implemented study designs
# Only for log-transformed data
# uses helper functions for the various BE designs

exp.power.TOST <- function(alpha=0.05, theta1=0.8, theta2, diff=0.95,
                           CV, dfCV, n, design="2x2")
{
  # check if design is implemented
  d.no <- .design.no(design)
  if (is.na(d.no)) stop("Err: unknown design!")
 
  # design characteristics
  ades <- .design.props(d.no)
  dfe  <- parse(text=ades$df[1],srcfile=NULL) #degrees of freedom as expression
  bk   <- ades$bk                             #design const.
 
  if (missing(CV)) stop("Err: CV must be given!")
  if (missing(n))  stop("Err: number of subjects must be given!")

  if (missing(theta2)) theta2 <- 1/theta1
  ltheta1 <- log(theta1)
  ltheta2 <- log(theta2)
  ldiff   <- log(diff)
  se      <- sqrt(log(1.+CV^2))
  df <- eval(dfe)
  pow <- .exp.power.TOST(alpha, ltheta1, ltheta2, ldiff, se, dfCV, n, df, bk)

  return( pow )
}


Homework: Calculate the "expected" power with diff=0.95, CV=0.3, dfCV=10 (12 subjects pilot) and n=40 and compare it to the 'classical' solution using D. Labes Eierlegende Wollmilchsau or ElMaestro's Apfelstrudel :-D.

The story will continue within a short time. (Oh no, that guy. There we go again!)

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2010-04-07 12:41
(5551 d 15:51 ago)

@ d_labes
Posting: # 5030
Views: 6,489
 

 Power and sample size with 'uncertain' CV - part II

Stephen Senn in Statistical issues in drug development: "The sample size calculation is an excuse for a sample size and not a reason"
Dear All!

Here part II of the story: Sample size

#------------------------------------------------------------------------------
# helper function to calculate std err from CV of lognormal data

.CV2se <- function(CV) return(sqrt(log(1.+CV^2)))
# reverse helper function
.se2CV <- function(se) return(sqrt(exp(se*se)-1))
#------------------------------------------------------------------------------
# sample size start for Joulious "expected" power

.exp.sampleN0 <- function(alpha=0.05, targetpower=0.8, ltheta1, ltheta2, diffm,
                          se, dfse, steps=2, bk=2)
{
  Z1 <- qnorm(1-alpha)
  if (abs(diffm)>0.0001) tinv <- qt(targetpower, dfse, Z1)  else
    tinv <- qt(1-(1-targetpower)/2, dfse, Z1)
 
  # is factor 2 in julious = bk?
  n01  <- bk*(se*tinv/(ltheta1-diffm))^2
  n02  <- bk*(se*tinv/(ltheta2-diffm))^2
  n0 <- ceiling(max(n01,n02))
  #make an even multiple of step (=2 in case of 2x2 cross-over)
  n0 <- steps*trunc(n0/steps)
  if (n0<4) n0 <- 6   # minimum sample size
 
  return(n0)
}
#-------------------------------------------------------------------------
# Sample size for a desired "expected" power according to Julious:
# see known.designs() for covered experimental designs
# Only for log-transformed data
# leave upper BE margin (ltheta2) empty and the function will use 1/lower
# CV and dfCV can be vectors, if then a pooled CV, df will be calculated

exp.sampleN.TOST <- function(alpha=0.05, targetpower=0.8,
                             theta1=0.8, theta2, diff=0.95, CV, dfCV,
                             design="2x2", print=TRUE, details=FALSE)
{
  #number of the design and check if design is implemented
  d.no <- .design.no(design)
  if (is.na(d.no)) stop("Err: design not known", call.=FALSE)
 
  # design charcteristics
  ades   <-.design.props(d.no)
  d.name <- ades$name  # nice name of design
  # get the df for the design as an unevaluated expression
  dfe    <- parse(text=ades$df,srcfile=NULL)
  steps  <- ades$steps   # stepsize for sample size search
  bk     <- ades$bk # get design constant
 
  if (missing(CV) | missing(dfCV)) {
    stop("Err: CV  and df must be given!", call.=FALSE)
  }
 
  # calculate pooled data if CV and dfCV are vectors
  if (length(CV)>1){
    if (length(dfCV)!=length(CV)) {
      stop("Err: CV and df must have equal number of entries!", call.=FALSE)
    }
    dfse <- sum(dfCV)
      CVp  <- .CV2se(CV)^2
      CVp  <- CVp * dfCV
      CVp  <- sum(CVp)/dfse
      CVp  <- .se2CV(CVp)
  } else {
    dfse <- dfCV
    CVp  <- CV
  }
  # print the configuration:
  if (print) {
    cat("\n++++++++ Equivalence test - TOST ++++++++\n")
    cat("   Sample size est. with uncertain CV\n")
    cat("-----------------------------------------\n")
    cat("Study design: ",d.name,"\n")
    if (details) {
      cat("Design characteristics:\n")
      cat("df = ",ades$df,", design const. = ",bk,
          ", step = ",steps,"\n\n",sep="")
    }     
  }
  if (missing(theta2)) theta2=1/theta1
  if ( (diff<=theta1) | (diff>=theta2) ) {
      stop("Err: ratio not between margins!", call.=FALSE)
    }
  ltheta1 <- log(theta1)
  ltheta2 <- log(theta2)
  diffm   <- log(diff)
  se      <- sqrt(log(1.+CVp^2))
  if (print) cat("log-transformed data (multiplicative model)\n\n")
 
  if (print) {
    cat("alpha = ",alpha,", target power = ", targetpower,"\n", sep="")
    cat("BE margins        =",theta1,"...", theta2,"\n")
    cat("Null (true) ratio = ",diff,"\n", sep="")
    if (length(CV)>1){
      cat("Variability data\\n")
      print(data.frame(CV=CV,df=dfCV), row.names = FALSE)
      cat("CV(pooled) = ", CVp, " with ", dfse," df\n", sep="")
   } else    cat("CV = ", CVp, " with ", dfse," df\n", sep="")
  }
 
  #start value from large sample approx.
  n  <- .exp.sampleN0(alpha, targetpower, ltheta1, ltheta2, diffm,
                      se, dfse, steps, bk)
  df <- eval(dfe)
 
  pow <- .exp.power.TOST(alpha, ltheta1, ltheta2, diffm, se, dfse, n=n, df, bk)
  if (details) {
    cat("\\nSample size search\n")
    if (d.no == 0) cat("(n is sample size per group)\n") #parallel group design
    cat(" n    exp. power\n")
    cat( n," ", formatC(pow, digits=6, format="f"),"\n")
  }
  # --- loop until power >= target power
  iter <- 0
  # iter>50 is emergency brake
  # this is eventually not necessary, depends on quality of sampleN0
  # in experimentation I have seen max of six steps

  if (pow<targetpower) {
    while (pow<targetpower) {
      n    <- n+steps
      iter <- iter+1
      df   <- eval(dfe)
      pow <- .exp.power.TOST(alpha, ltheta1, ltheta2, diffm, se, dfse,
                             n=n, df, bk)
      if (details) cat( n," ", formatC(pow, digits=6, format="f"),"\n")
      if (iter>50) break
    }
  } else {
    while (pow>targetpower) {
      if (n<=6) break     # min. sample size
      n    <- n-steps     # step down if start power is to high
      iter <- iter+1
      df   <- eval(dfe)
      pow  <- .exp.power.TOST(alpha, ltheta1, ltheta2, diffm, se, dfse,
                              n=n, df, bk)
      if (details) cat( n," ", formatC(pow, digits=6),"\n")
      if (iter>50) break 
    }
    if ( pow<targetpower ) {
      n    <- n+steps     #step up once to have n with pow>=target
      df   <- eval(dfe)
      pow  <- .exp.power.TOST(alpha, ltheta1, ltheta2, diffm, se, dfse,
                              n=n, df, bk)
    }
  }
  if (print && !details) {
    cat("\nSample size\n")
    if (d.no == 0) cat("(n is sample size per group)\n") #parallel group
    cat(" n    exp. power\\n")
    cat( n," ", formatC(pow, digits=6, format="f"),"\n")
  }
 
  if (print) cat("\n")
 
  #return results as data.frame
  res <- data.frame(design=design, alpha=alpha, CV=CV, theta0=diff,
                    theta1=theta1, theta2=theta2, n=n, power=pow,
                    targetpower=targetpower)
  names(res) <-c("Design","alpha","CV","theta0","theta1","theta2",
                 "Sample size", "Achieved power", "Target power")
 
  if (print) return(invisible(res))
  else return(res)
 
}


BTW: If you copy this code take extremely care of the backslash-eating devil lurking round the corner :-P.


Edit: Code corrected according to an Erratum of D Labes in a later post to allow easy copying. [Helmut]
d_labes
★★★

Berlin, Germany,
2010-04-07 16:43
(5551 d 11:49 ago)

@ d_labes
Posting: # 5032
Views: 6,372
 

 Helper for rescue

Dear All!

Just seen: I had not delivered a helper function used here in my original posts.

Sorry if anybody had inconvenience with that.

Here the helper:
#--- return all properties of the design as dataframe ---
.design.props <- function(design.no)
{
   des <- known.designs()
   return (des[des$no==design.no,])
}   

Regards,

Detlew
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2010-04-08 11:10
(5550 d 17:22 ago)

(edited on 2010-04-08 13:43)
@ d_labes
Posting: # 5035
Views: 6,406
 

 Power and sample size with 'uncertain' CV

Dear d_labes,

Thanks for your great efforts. I just wonder if there is any way to VALIDATE (or how to validate) this method, supposed we have imprecise/uncertain CV (how imprecise can it be allowed?) obtained from a pilot trial (not from the sequential two-staged design). There was no reference or example in Julious SA's text. I have not got the other reference yet that you've pointed here. But I'll get it lately. I was thinking the possibility to solve this using Bayesian inference approach recently.

All the best,
-- Yung-jin Lee
bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
d_labes
★★★

Berlin, Germany,
2010-04-08 12:04
(5550 d 16:29 ago)

@ yjlee168
Posting: # 5036
Views: 6,388
 

 Validation necessary

Dear Yung-jin,

❝ ... I just wonder if there is any way to

❝ VALIDATE (or how to validate) this method


Unfortunately Julious has not given worked examples in his book considering the case of uncertain variance.
But he has given Inflation factors (factors to multiply the classical "carved in stone" sample sizes) on page 115.
Here an excerpt with the usual powers:

         ------- alpha ---------
  m beta 0.010 0.025 0.050 0.100
--------------------------------
  5 0.10 2.167 2.068 1.980 1.875
    0.20 1.776 1.711 1.652 1.581
 10 0.10 1.463 1.425 1.392 1.353
    0.20 1.328 1.301 1.276 1.248
 25 0.10 1.163 1.150 1.139 1.125
    0.20 1.119 1.109 1.101 1.091
 50 0.10 1.078 1.072 1.067 1.060
    0.20 1.058 1.053 1.049 1.044
 75 0.10 1.052 1.047 1.044 1.040
    0.20 1.038 1.035 1.032 1.029
100 0.10 1.038 1.035 1.033 1.030
    0.20 1.029 1.026 1.024 1.022


But they are only for the case that the assumed true ratio is 1.

❝ ... imprecise/uncertain CV (how imprecise can it be allowed?) ...


As you can see up to df(=m) around 75 there is still approximately a 5% higher sample size compared to the classical calculation depending on alpha, beta. How big this excess is for the true ratio assumed !=1 can be answered using the code supplied above.
Note also the nearly doubling of the sample size for df=5 corresponding roughly to a CV from a pilot with 6 subjects! :yes:

❝ ... But I'll late. I was thinking the possibility to solve

❝ this using Bayesian inference approach recently.


"He who comes too late will be punished by life." :-D
(Michail Gorbatschow in 1989 to Erich Honnecker shortly before the opening of the Berlin Wall)

Julious has not given much details, not to say nearly nothing, about the theory behind his formulas. But I think its sort of Bayesian reasoning. Expected power (aka some sort of average) seems the power averaged over the distribution of the variability namely a chi-squared distribution we (Helmut) up to now used in sensitivity analysis aka upper confidence limit.

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2010-04-08 15:36
(5550 d 12:57 ago)

@ yjlee168
Posting: # 5038
Views: 6,402
 

 Scarcely validation

Dear bears, dear all,

here some brief calculation results with 'true' ratio =1 and a classical 2x2 cross-over using the common acceptance limits 0.8 - 1.25:

                  sample size (alpha=0.05)
                 carved   uncertain 
      power CV   in stone   CV       ratio
--------------------------------------------
m=5   0.8   0.2     16      24        1.5
            0.25    24      36        1.5
            0.3     32      52        1.625
            0.35    42      68        1.619
            0.4     54      86        1.593
      0.9   0.2     20      36        1.8
            0.25    28      54        1.929
            0.3     40      76        1.9
            0.35    52     102        1.962
            0.4     66     130        1.967


Note that we cannot meet Joulious factors exactly because of the discrete nature of the sample size estimation (steps of 2, meeting target power with different excess). But we are very near of his theoretical derived factors (which do not depend on CV) of 1.652 for power=0.8 and 1.980 for power=0.9 :ok:.
Moreover it must be noted that Julious factors are derived with Z-quantiles instead of t-quantiles in the sample size formula i.e. are large sample results.

Same calculations with ratio=0.95 but only for power=80%:

                  sample size (alpha=0.05)
                 carved   uncertain 
      power CV   in stone   CV       ratio
--------------------------------------------
m=5   0.8   0.2     20      28        1.4
            0.25    28      44        1.571
            0.3     40      60        1.5
            0.35    52      80        1.538
            0.4     66     104        1.576

Stephen Senn in "Statistical issues in drug development": "Clinically relevant difference: That which is used to justify the sample size but will be claimed to have been used to find it."

Regards,

Detlew
UA Flag
Activity
 Admin contact
23,424 posts in 4,927 threads, 1,674 registered users;
33 visitors (0 registered, 33 guests [including 6 identified bots]).
Forum time: 04:33 CEST (Europe/Vienna)

Philosophy, like medicine, has plenty of drugs, few good remedies,
and hardly any specific cures.    Sebastien-Roch Nicolas de Chamfort

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5