Bioequivalence and Bioavailability Forum

Main page Policy/Terms of Use Abbreviations Latest Posts

 Log in |  Register |  Search

Back to the forum  2018-07-17 05:54 UTC (UTC+2h)
Yura
Regular

Belarus,
2018-01-23 14:02

Posting: # 18269
Views: 3,042
 

 Power R WinNonLin [Power / Sample Size]

Hi everybody

From the research 2x2x2 (34 - balanced) for power estimation the following results are taken: β=0.20, α=0.05, CV = 0.32, PE = 0.92

FARTSSIE17: 55%
R: 55%
WinNonLin: Power_80_20: 89%
Why is this happening?
Regards
d_labes
Hero

Berlin, Germany,
2018-01-23 14:25

@ Yura
Posting: # 18271
Views: 2,562
 

 Power R WinNonLin

Dear Yura

have a look at this thread from the year 2011.

And tell us which version of WinNonlin you used for your calculations.
AFAIK Phoenix had corrected this nonsense power calculations later on.

Regards,

Detlew
Helmut
Hero
avatar
Homepage
Vienna, Austria,
2018-01-23 15:05

@ d_labes
Posting: # 18274
Views: 2,582
 

 Power_TOST in PHX/WNL 6.4+

Dear Detlew & Yura

» And tell us which version of WinNonlin you used for your calculations.
» AFAIK Phoenix had corrected this nonsense power calculations later on.

The additional result Power_TOST was introduced in Phoenix WinNonlin v6.4 (2014). During development it was cross-validated against power.TOST(..., method="nct"). ;-)
Power_80_20 (for the FDA’s 80/20-rule) is still there for backwards compatibility. Don’t use it.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
Hero

Berlin, Germany,
2018-01-23 15:54

@ Helmut
Posting: # 18275
Views: 2,527
 

 Power_TOST in PHX/WNL 6.4+

Dear Helmut!

» The additional result Power_TOST was introduced in Phoenix WinNonlin v6.4 (2014). During development it was cross-validated against power.TOST(..., method="nct"). ;-)

Funny :cool:.

Regards,

Detlew
Helmut
Hero
avatar
Homepage
Vienna, Austria,
2018-01-23 16:25

@ d_labes
Posting: # 18276
Views: 2,577
 

 Power_TOST in PHX/WNL 6.4+

Dear Detlew!

» » […] Power_TOST was introduced in Phoenix WinNonlin v6.4 (2014). […] it was cross-validated against power.TOST(..., method="nct"). ;-)
»
» Funny :cool:.

I bombarded Pharsight for years asking for an update. I was a beta-tester of PHX v6.0–6.4. Guess who suggested which software to compare to. :-D

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Yura
Regular

Belarus,
2018-01-23 20:55

@ Helmut
Posting: # 18284
Views: 2,498
 

 Power_TOST in PHX/WNL 6.4+

Hi Detlew & Helmut
» Power_80_20 (for the FDA’s 80/20-rule) is still there for backwards compatibility. Don’t use it.

Thanks for the advice
WNL's Power do not use

Regards
pjs
Regular

India,
2018-03-23 09:41

@ Yura
Posting: # 18589
Views: 1,647
 

 Power R WinNonLin

Hi All,

Below is the power reported for the assumed parameters with different software.

» From the research 2x2x2 (34 - balanced) for power estimation the following results are taken: β=0.20, α=0.05, CV = 0.32, PE = 0.92
»
» FARTSSIE17: 55%
» R: 55%
» WinNonLin: Power_80_20: 89%

Can you pls guide for FARTSSIE and R which method for power estimation is being used.

Currently i am cross verifying power value with the published paper of the potvin for two stage. They had calculated power with modification of Hauschke et al. For example if i take N=12, CV=40% and ratio 95% power by method C is reported to be 0.7505. For the same parameters power by FARTSSIE is -24.19%.

Regards
Pjs
d_labes
Hero

Berlin, Germany,
2018-03-23 16:02

@ pjs
Posting: # 18592
Views: 1,596
 

 Power of two stage design

Dear Pjs,

» Currently i am cross verifying power value with the published paper of the potvin for two stage. They had calculated power with modification of Hauschke et al. For example if i take N=12, CV=40% and ratio 95% power by method C is reported to be 0.7505. For the same parameters power by FARTSSIE is -24.19%.

This is a fundamental misunderstanding.
What you try to do is, if I understand you correctly, to obtain the power of a two stage design.
This is not possible in FARTSSIE and also not in the R package PowerTOST.

The framework of TSDs is so complicated that no algebraic solution for obtaining power is available. What you have to do is to use simulations, like the ones described in the Potvin paper.

Have a look into the R add-on package Power2Stage. With the function power.2stage() you are able to verify the type 1 error and the power reported in the Potvin paper.

Example:
library(Power2Stage)
# power at true ratio=0.95
power.2stage(alpha=c(0.0294, 0.0294), method="C", CV=0.4, n1=12, GMR=0.95, theta0=0.95)

gives
TSD with 2x2 crossover
Method C: alpha0 = 0.05, alpha (s1/s2) = 0.0294 0.0294
Target power in power monitoring and sample size est. = 0.8
Power calculation via non-central t approx.
CV1 and GMR = 0.95 in sample size est. used
No futility criterion
BE acceptance range = 0.8 ... 1.25

CV = 0.4; n(stage 1) = 12; GMR= 0.95

1e+05 sims at theta0 = 0.95 (p(BE) = 'power').
p(BE)    = 0.75058
p(BE) s1 = 0.00986
Studies in stage 2 = 98.95%

Distribution of n(total)
- mean (range) = 78.9 (12 ... 302)
- percentiles
 5% 50% 95%
 32  74 142


# empirical type 1 error
power.2stage(alpha=c(0.0294, 0.0294), method="C", CV=0.4, n1=12, GMR=0.95, theta0=1.25)

gives
TSD with 2x2 crossover
Method C: alpha0 = 0.05, alpha (s1/s2) = 0.0294 0.0294
Target power in power monitoring and sample size est. = 0.8
Power calculation via non-central t approx.
CV1 and GMR = 0.95 in sample size est. used
No futility criterion
BE acceptance range = 0.8 ... 1.25

CV = 0.4; n(stage 1) = 12; GMR= 0.95

1e+06 sims at theta0 = 1.25 (p(BE) = TIE 'alpha').
p(BE)    = 0.034468
p(BE) s1 = 0.003881
Studies in stage 2 = 99.5%

Distribution of n(total)
- mean (range) = 79.1 (12 ... 360)
- percentiles
 5% 50% 95%
 32  74 142


BTW: If you aim to work exactly like Potvin et al. you have to set the argument pmethod="shifted" in the function calls. This calculates power in the power monitoring step of the two stage schemes via a somewhat crude approximation and was used in the Potvin paper for speed reasons only.

Nevertheless you will not obtain exacly the same numbers due to the unknown auxillary conditions for the simulations like e.g. the seed of the random number genarator or its type.

BTW2: The negative power from FARTSSIE is of course nonsense and results from an approximation for the power calculation. The exact power for a fixed (1-stage) design with your settings is
library(PowerTOST)
power.TOST(CV=0.4, n=12)
[1] 0.02843316

Regards,

Detlew
Helmut
Hero
avatar
Homepage
Vienna, Austria,
2018-03-24 14:49

@ pjs
Posting: # 18593
Views: 1,426
 

 Power2Stage / power.2stage()

Hi Pjs,

extending Detlew’s post. Try this code:

library(Power2Stage)
CV      <- 0.4
n1      <- 12
method  <- "C"
pmethod <- c("shifted", "nct", "exact")
st      <- proc.time()[[3]]
res     <- data.frame(method=method, CV=CV, n1=n1, pmethod=pmethod,
                      power=NA, TIE=NA, n.tot=NA, pct05=NA, pct50=NA,
                      pct95=NA, pct.stg2=NA)
for (j in seq_along(pmethod)) {
  x <- power.2stage(alpha=rep(0.0294, 2), method=method, CV=CV, n1=n1,
                    GMR=0.95, theta0=0.95, pmethod=pmethod[j],
                    npct=c(0.05, 0.5, 0.95))
  y <- power.2stage(alpha=rep(0.0294, 2), method=method, CV=CV, n1=n1,
                    GMR=0.95, theta0=1.25, pmethod=pmethod[j])
  res[j, "power"]    <- round(x$pBE, 4)
  res[j, "TIE"]      <- round(y$pBE, 4)
  res[j, "n.tot"]    <- round(x$nmean, 1)
  res[j, 8:10]       <- x$nperc
  res[j, "pct.stg2"] <- round(x$pct_s2, 1)
}
et      <- proc.time()[[3]] - st
cat("run time:", signif(et/60, 1), "minutes\n"); print(res, row.names=FALSE)

Gives:

run time: 8.9 minutes
 method  CV n1 pmethod  power    TIE n.tot pct05 pct50 pct95 pct.stg2
      C 0.4 12 shifted 0.7509 0.0342  79.0    34    74   142     99.0
      C 0.4 12     nct 0.7506 0.0345  78.9    32    74   142     98.9
      C 0.4 12   exact 0.7506 0.0345  78.9    32    74   142     98.9

Columns: power overall power, TIE empiric Type I Error (for true ratio 1.25), n.tot expected average total sample size; pct05, pct50, pct95 5%, 50%, 95% percentiles of expected total sample size, pct.stg2 percent of studies expected to proceed to the 2nd stage.
Reported by Potvin et al. in Tables I & II:

method  CV n1 pmethod  power    TIE n.tot pct05 pct50 pct95 pct.stg2
      C 0.4 12 shifted
0.7505 0.0346  79.0    32    74   142     99.0

Potvin et al. employed FORTRAN-code (hence, we don’t know which pseudorandom number generator was used). In later papers (Montague et al., Xu et al.) R was used as well where the Mersenne-Twister is the default PRNG. However, we don’t know which seed was used. Therefore, slight differences in the simulations’ results are normal. BTW, Fuglsang employed also the Mersenne-Twister (C-code) in his papers.
Try this code with a different seed in each run:

runs <- 25
res  <- data.frame(power=rep(NA, runs), TIE=rep(NA, runs))
st   <- proc.time()[[3]]
for (j in 1:runs) {
  res[j, "power"]  <- power.2stage(alpha=rep(0.0294, 2), method=method,
                                   CV=CV, n1=n1, GMR=0.95, theta0=0.95,
                                   pmethod="shifted", setseed=FALSE)$pBE
  res[j, "TIE"]    <- power.2stage(alpha=rep(0.0294, 2), method=method,
                                   CV=CV, n1=n1, GMR=0.95, theta0=1.25,
                                   pmethod="shifted", setseed=FALSE)$pBE
}
et   <- proc.time()[[3]] - st
cat("run time:", signif(et/60, 2), "minutes\n"); summary(res)

Gives:

run time: 6.4 minutes
     power             TIE         
 Min.   :0.7482   Min.   :0.03403 
 1st Qu.:0.7503   1st Qu.:0.03443 
 Median :0.7511   Median :0.03455 
 Mean   :0.7511   Mean   :0.03451 
 3rd Qu.:0.7522   3rd Qu.:0.03460 
 Max.   :0.7537   Max.   :0.03485


If you want to estimate the sample size of the 2nd stage I recommend the function sampleN2.TOST() of Power2Stage instead of sampleN.TOST() of PowerTOST and subtract n1. In the final analysis we have one degree of freedom less than in the conventional model (due to the additional stage-term). Comparison:

x <- sampleN2.TOST(alpha=0.0294, CV=CV, n1=n1, theta0=0.95)
y <- sampleN.TOST(alpha=0.0294, CV=CV, theta0=0.95, details=FALSE, print=FALSE)
print(x, row.names=FALSE) # n2 directly with correct df
print(y, row.names=FALSE) # total n with incorrect df

Gives:

 Design  alpha  CV theta0 theta1 theta2 n1 Sample size Achieved power Target power
    2x2 0.0294 0.4   0.95    0.8   1.25 12          68      0.8107469          0.8
 Design  alpha  CV theta0 theta1 theta2 Sample size Achieved power Target power
    2x2 0.0294 0.4   0.95    0.8   1.25          80      0.8105103          0.8


Let’s see whether we can reproduce Potvin’s examples for ‘Method C’:

potvin.examples <- function(alpha0=0.05, alpha=0.0294, n1,
                            log.GMR1, S.sq1, log.GMR2, S.sq2) {
  require(PowerTOST)
  require(Power2Stage)
  method   <- "C"
  ltheta1  <- log(0.80)
  ltheta2  <- log(1.25)
  ltheta0  <- log(0.95)
  pass.1   <- pass.2 <- FALSE
  pwr.1    <- power.TOST(alpha=alpha0, CV=mse2CV(S.sq1), n=n1,
                         method="shifted")
  if (pwr.1 >= 0.80) {
    CI.stg1 <- round(100*CI.BE(alpha=alpha0, pe=exp(log.GMR1),
                               CV=mse2CV(S.sq1), n=n1), 2)
  } else {
    CI.stg1 <- round(100*CI.BE(alpha=alpha, pe=exp(log.GMR1),
                               CV=mse2CV(S.sq1), n=n1), 2)
  }
  if (CI.stg1[["lower"]] < 0.80 | CI.stg1[["upper"]] > 1.25 & pwr.1 < 0.80) {
    n2 <- sampleN2.TOST(alpha=alpha, CV=mse2CV(S.sq1), n1=n1,
                        theta0=0.95, method="shifted")[["Sample size"]]
  } else {
    pass.1 <- TRUE
  }
  if (!pass.1) {
    df2     <- n1 + n2 - 3
    sem     <- sqrt(2*S.sq2/(n1+n2))
    CI.stg2 <- round(100*exp(log.GMR2 + c(-1, +1) *
                             qt(1-alpha, df2)*sem), 2) # balanced!
    names(CI.stg2) <- c("lower", "upper")
    if (CI.stg2[["lower"]] >= 0.80 | CI.stg2[["upper"]] <= 1.25) pass.2 <- TRUE
  }
  txt <- paste0("Power in the interim = ", round(100*pwr.1, 1), "%.",
                "\nSince power is ")
  if (pwr.1 >= 0.80) {
    txt <- paste0(txt, "\u226580%, BE assessed with alpha ", alpha0, ".")
  } else {
    txt <- paste0(txt, "<80%, BE assessed with alpha ", alpha, ".")
  }
  if (pass.1) {
    txt <- paste0(txt, "\nStudy passed in the 1st stage: ")
    if (pwr.1 >= 0.80) {
      txt <- paste0(txt, "90% CI: ")
    } else {
      txt <- paste0(txt, 100*(1-2*alpha), "% CI = ")
    }
    txt <- paste0(txt, CI.stg1[["lower"]], "% \u2013 ", CI.stg1[["upper"]], "%.")
  } else {
    txt <- paste0(txt, "\nStudy failed in the 1st stage: ")
    if (pwr.1 >= 0.80) {
      txt <- paste0(txt, "90% CI = ")
    } else {
      txt <- paste0(txt, 100*(1-2*alpha), "% CI = ")
    }
    txt <- paste0(txt, CI.stg1[["lower"]], "% \u2013 ", CI.stg1[["upper"]], "%.")
    if (pwr.1 < 0.80)
      txt <- paste0(txt, "\n2nd stage with ", n2, " subjects initiated.")
  }
  if (!pass.1 & pwr.1 < 0.80) {
    if (pass.2) {
      txt <- paste0(txt, "\nStudy passed in the 2nd stage: ")
    } else {
      txt <- paste0(txt, "\nStudy failed in the 2nd stage: ")
    }
  }
  if (!pass.1 & pwr.1 < 0.80) {
    pwr.21  <- Power2Stage:::.calc.power(alpha=alpha, ltheta1=ltheta1,
                                         ltheta2=ltheta2, diffm=ltheta0,
                                         sem=sem, df=df2, method="shifted")
    pwr.22  <- Power2Stage:::.calc.power(alpha=alpha, ltheta1=ltheta1,
                                         ltheta2=ltheta2, diffm=log.GMR2,
                                         sem=sem, df=df2, method="shifted")
    txt <- paste0(txt, 100*(1-2*alpha), "% CI = ",
                  CI.stg2[["lower"]], "% \u2013 ", CI.stg2[["upper"]], "%.")
    txt <- paste0(txt, "\n(Irrelevant) post hoc powers:",
                  "\nFor GMR 0.95 = ", round(100*pwr.21, 1), "%, ",
                  "for GMR ", round(exp(log.GMR2), 4), " = ",
                  round(100*pwr.22, 1), "%.")
  }
  cat(txt, "\n")
}

Example 1:

potvin.examples(n1=12, log.GMR1=0.16785, S.sq1=0.020977,
                log.GMR2=0.14401, S.sq2=0.0212240)
Power in the interim = 84.1%.
Since power is ≥80%, BE assessed with alpha 0.05.
Study failed in the 1st stage:
90% CI = 106.26% – 131.66%.

Example 2:

potvin.examples(n1=12, log.GMR1=0.08396, S.sq1=0.032634,
                log.GMR2=0.014439, S.sq2=0.045896)
Power in the interim = 64.9%.
Since power is <80%, BE assessed with alpha 0.0294.
Study failed in the 1st stage:
94.12% CI = 92.93% – 127.28%.
2nd stage with
8 subjects initiated.
Study passed in the 2nd stage:
94.12% CI = 88.45% – 116.38%.
(Irrelevant) post hoc powers:
For GMR 0.95 =
66.3%, for GMR 1.0145 = 76.9%.


Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Back to the forum Activity
 Thread view
Bioequivalence and Bioavailability Forum |  Admin contact
18,540 posts in 3,939 threads, 1,189 registered users;
online 37 (0 registered, 37 guests [including 26 identified bots]).

If you cannot measure, then your knowledge
is meagre and unsatisfactory.    Lord Kelvin

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
HTML5 RSS Feed