Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-02-07 20:05
(4820 d 21:03 ago)

Posting: # 6574
Views: 5,962
 

 Post hoc power [Power / Sample Size]

Dear all,

at the recent workshop in Mumbai I struggled (again) to deliver the message that a posteriori power does not make sense.

In the aftermath of this post on Potvin et al.'s paper I found a sentence soothing my soul:

We stop here and conclude BE, irrespective of the fact that we have not yet achieved the desired power of 80% (power = 66.3%).


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-02-08 10:18
(4820 d 06:49 ago)

@ Helmut
Posting: # 6580
Views: 5,035
 

 The battle is not over

Dear Helmut,

I recently had a question from an author of a paper in a medical journal. He had stated "A Student's t-test of ... :blahblah: give a non-significant result."

Statement of a reviewer to that sentence: "A power analysis is expected." [sic !]

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-02-08 12:27
(4820 d 04:41 ago)

@ d_labes
Posting: # 6588
Views: 5,109
 

 Fight the good fight

Dear D. Labes!

❝ [...] "A power analysis is expected." [sic !]



Hurrah! Maybe the reviewer should have been guided to CONSORT, especially the download section. The 2010 guideline was published in leading journals - maybe he/she doesn't read? Quote from Item 7 (Sample size):

There is little merit in a post hoc calculation of statistical power using the results of a trial; the power is then appropriately indicated by confidence intervals.


See also this nice discussion in the British Medical Journal after CONSORT 2001 was published.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
earlybird
☆    

2011-02-08 12:20
(4820 d 04:47 ago)

@ Helmut
Posting: # 6586
Views: 5,123
 

 Post hoc power

Dear HS,

you should take into considerations a statistic course for regulators ;-). As long as the regulators ask for the a-posteriori-power, you should be able to calculate it :confused:.

Kind reagards,
earlybird
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-02-08 13:22
(4820 d 03:46 ago)

@ earlybird
Posting: # 6589
Views: 5,160
 

 Post hoc power

Dear earlybird!

❝ you should take into considerations a statistic course for regulators ;-).


Well, I was asked by regulators from Austria and Saudi Arabia a good while ago. They were shocked by the fee. :-D
I performed a workshop in Turkey; was anybody of you asked for power (in other words, did I succeed)?

❝ As long as the regulators ask for the a-posteriori-power, …


Do they still? Which ones? I know that this was a hobby of regulators from Greece more than a decade ago. The last request I have seen came form Malaysia. They were asking for ‘Two One Sided t-Tests Total Probability’, ‘Anderson-Hauck Procedure p-value’, ‘Power of ANOVA at 20%’. By coincidence (?) these are the last lines of WinNonlin’s standard output. ;-)

❝ … you should be able to calculate it :confused:.


PowerTOST!

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dr_Dan
★★  

Germany,
2011-02-08 15:31
(4820 d 01:37 ago)

@ Helmut
Posting: # 6591
Views: 4,989
 

 Post hoc power

Dear Helmut
dear all
some time ago I had to answer the same question but since I am not a statistician I pointed out the following which is in my opinion the assessor's concern:
The risk for the patient that the study might show a false positive result is always 5%. The power only describes the risk for the sponsor to fail. Right?
LG
Dan

Kind regards and have a nice day
Dr_Dan
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-02-08 15:34
(4820 d 01:34 ago)

@ Dr_Dan
Posting: # 6592
Views: 5,039
 

 Post hoc power

Dear Dan!

❝ The risk for the patient that the study might show a false positive result is always 5%. The power only describes the risk for the sponsor to fail. Right?


Right.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-02-08 16:17
(4820 d 00:51 ago)

@ Helmut
Posting: # 6595
Views: 5,024
 

 Risk nitpicking

Dear Dan, dear Helmut!

❝ ❝ The risk for the patient that the study might show a false positive result is always 5%. Right?


❝ Right.


For answering regulatory concerns it is the best answer I know :cool:.

But let us keep in mind the so called "test size" of the statistical test method used.

In many cases the test method assures an size <= 0.05, thus the test may become conservative under circumstances. Then the patient's risk is at maximum 5% and otherwise <5%.

In case of "liberal" test methods the patients risk may become >5%. Such test methods should of course not used. But sometimes you have no choice ...

Remember f.i. Potvin et.al. methods for evaluating 2-stage designs which have been chosen under the assumption that 5.2% alpha inflation are allowed, considered as negligible.
Or remember the FDA scaled ABE approach according to the progesterone guidance which leads to an alpha inflation up to 6% - 7%. See here.

Regards,

Detlew
UA Flag
Activity
 Admin contact
22,988 posts in 4,825 threads, 1,655 registered users;
67 visitors (0 registered, 67 guests [including 4 identified bots]).
Forum time: 18:08 CEST (Europe/Vienna)

The whole purpose of education is
to turn mirrors into windows.    Sydney J. Harris

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5