## The secret of WinNonlin's BE's post-hoc Power analysis [Software]

As asked by HS and provoked by this, it's time to talk about WNL's BE's post-hoc Power analysis.

For a long time, WNL's Power is considered as statistical NONSENSE, see:

HS's presentations such as

this page 26,

this page 12, and

this page 22.

Of course, I agree with our HS's opinion on this issue largely because of the strange method used by WNL for Power calculation, and the strange results as well.

Actually, I have consulted a medical statistician who is professional at design and analysis of clinical trials. He told me he usually do power analysis to determine sample size before starting a trial, but seldom calc post-hoc power, he also told me that even if the post-hoc power is a little lower than designed or expected power, it is no necessary to do additional trial to increase the sample size and expect to increase the power.

Consistent with HS’s and Potvin's opinions.

Regardless of the necessity, let's first look inside into WNL to see how it calc power.

----------From PNX WNL User Guide P341-342----------------------------------

**Power**

Power is the post-hoc probability of detecting a difference greater than or equal to a specified percentage of the reference least squares mean. In general,

Power = 1 – (probability of a type II error)

= probability of rejecting H_{0} when H_{1} is true.

In the bioequivalence calculations, the hypotheses being tested are:

H_{0}: RefLSM = TestLSM

H_{1}: RefLSM ≠ TestLSM

For the no-transform case, the power is the probability of rejecting H

_{0}given:

|TestLSM – RefLSM | ≥ fractionToDetect × RefLSM

For ln-transform, and data already ln-transformed, this changes to:

|TestLSM – RefLSM | ≥ – ln(1 – fractionToDetect),

and similarly for log10-transform and data already log10-transformed.

For the sake of illustration, assume fractionToDetect = 0.2, RefLSM>0, and no transform was done on the data. Also the maximum probability of rejecting H

_{0}occurs in the case of equality,

|TestLSM – RefLSM| = 0.2×RefLSM. So:

Power = Pr (rejecting H_{0} at the alpha level given |Difference| = 0.2×RefLSM)

= Pr (|Difference/DiffSE| > t_{(1-α/2),df} given |Difference| = 0.2×RefLSM)

= Pr (Difference/DiffSE > t_{(1-α/2),df} given |Difference| = 0.2×RefLSM)

+ Pr (Difference/DiffSE < –t_{(1-α/2),df} given |Difference| = 0.2×RefLSM)

Let:

t_{1} = t_{(1-α/2),df} – 0.2×RefLSM / DiffSE

t_{2} = – t_{(1-α/2),df} – 0.2×RefLSM / DiffSE

t_{stat} = (Difference – 0.2×RefLSM) / DiffSE

Then:

Power = Pr(t_{stat} > t_{1} given |Difference| = 0.2×RefLSM)

+ Pr(t_{stat} < t_{2} given |Difference| = 0.2×RefLSM)

Note that t

_{stat}is T-distributed. Let p

_{1}be the p-value associated with t

_{1}(the area in the tail), and let p

_{2}be the p-value associated with t

_{2}(the area in the tail; note that p

_{2}may be negligible). Then:

Power = 1 – (p_{1} – p_{2})

----------From PNX WNL User Guide P341-342----------------------------------

regretfully, WNL doesn't give us the formula when data is ln=transformed before BE analysis.

Before we start to validate WNL's formula, let me show my first puzzle:

As pointed by above WNL's user guide, it is obviouse that WNL is calculating the Power for TOST, but in WNL's ASCII output WNL gives us

` Power of ANOVA for Confidence Level 90.00`

Power at 20% = 0.993541

for ANOVA or for TOST? It is a question.

Now, let me validate WNL's formula using Chow and Liu's dataset.

-----For original data without transformation--------

WNL's result is

` Power of ANOVA for Confidence Level 90.00`

Power at 20% = 0.9935408

My formula and result is:

`t1 = t (1-α/2),df – (1 - 0.8)×RefLSM / DiffSE`

t2 = – t (1-α/2),df – (1.2 -1)×RefLSM / DiffSE

p1 = TDist(Abs(t1), n1 + n2 - 2, 1)

p2 = TDist(Abs(t2), n1 + n2 - 2, 1)

Power = 1 - (p1 - p2)

`Power Analysis`

--------------------------------------------------------------------------

Parameter T stat 1 T stat 2 P value 1 P value 2 Power

--------------------------------------------------------------------------

AUC -2.705765713 -6.140054462 0.006455207 0.000001760 0.993546553

--------------------------------------------------------------------------

------------For ln-transformed data---------------

WNL's result is

` Power of ANOVA for Confidence Level 90.00`

Power at 20% = 0.9839865

My formula and result is:

`t1 = t (1-α/2),df + Ln(0.8) / DiffSE`

t2 = – t (1-α/2),df - Ln(1.25) / DiffSE

p1 = TDist(Abs(t1), n1 + n2 - 2, 1)

p2 = TDist(Abs(t2), n1 + n2 - 2, 1)

Power = 1 - (p1 - p2)

`Power Analysis`

--------------------------------------------------------------------------

Parameter T stat 1 T stat 2 P value 1 P value 2 Power

--------------------------------------------------------------------------

ln(AUC) -2.289520539 -5.723809288 0.016004731 4.65843E-06 0.983999927

--------------------------------------------------------------------------

To ensure the sufficient precision of my calc, all the above calculations were done using MS Excel 2010 which is of much more higher precision than MS Excel 2000, 2003 and 2007 when Tinv() or Tdist() is used.

For comparison:

` Software Tinv(0.1,22)`

R 2.10.1 1.717144374380243

Excel 2010 1.71714437438024

Open Office 3.3.0 1.71714437438025

Gnumeric 1.10.16 1.71714437438148

Excel 2003 and 2007 1.71714433543983

WNL 5.1.1 1.71714434835526

It should be noted that the formula I used for Ln-transformed data is of no reference, only obtained by my guess, but it works well specially to obtain WNL's results. I don't know whether it is correct or not and don't know WNL is correct or not.

If we using PowerTOST as a gold standard for power calc of TOST, all WNL's results are and totally wrong.

It seems that WNL's post-hoc power calc doesn't directly use Observed Diff or Observed Ratio or intra-CV, only indirectly uses DiffSE and Expected Diff or Ratio such as 0.8, 1,2, 0.8, 1.25. Dear all, True or False of WNL's method?

Dear HS, have you reached Berlin and meet our D. Labes? I need your insight and comments on this issue.

### Complete thread:

- The secret of WinNonlin's BE's post-hoc Power analysisyicaoting 2011-11-08 14:40 [Software]
- The secret of WinNonlin's BE's post-hoc Power analysis yicaoting 2011-11-08 14:55
- The secret of WinNonlin's BE's post-hoc Power analysis Helmut 2011-11-08 15:06
- Power of superiority d_labes 2011-11-10 09:49
- Power of superiority Helmut 2011-11-10 14:39
- Power of equality d_labes 2011-11-10 16:42
- Power of equality Helmut 2011-11-10 17:00
- Power of equality test - 2nd try for the logs d_labes 2011-11-11 09:55
- Crtl-C/Ctrl-V Helmut 2011-11-11 14:39

- Power of equality test - 2nd try for the logs d_labes 2011-11-11 09:55
- Power of equality yicaoting 2011-11-12 07:05

- Power of equality Helmut 2011-11-10 17:00

- Power of equality d_labes 2011-11-10 16:42

- Power of superiority Helmut 2011-11-10 14:39