α ≠ β [Study Assessment]
I agree with ElMaestro.
❝ We were told […]
❝ We were asked […]
By whom? By the sponsor? By a regulator? I give some points below, but if I would know who asked these questions I could give you a more detailed answer.
❝ […] revalidate the study since the power of study should be 24 subjects.
Was this term used in English or is it a translation? Sounds like plain nonsense to me. You planned the study according to information / assumptions for 24 subjects. If all (‼) assumptions turn out to be exactly (‼) correct and you don’t have a single (‼) drop-out then you have a chance of passing BE, which is exactly the power you aimed at. Below possible other combinations:
assumed in observed in producer’s power
study design actual study risk (β) (1–β)
────────────────────────────────────────────────
T/R ratio closer to 1 ⇓ ⇑
away from 1 ⇑ ⇓
CV lower ⇓ ⇑
higher ⇑ ⇓
# drop-outs lower ⇓ ⇑
higher ⇑ ⇓
The patient’s risk (α = probability of the type I error) – this is the only one regulators should care about! – is ≤5% in any case – independent from power (1–β, where β = probability of the type II error, aka producer’s risk).
❝ How do we revalidate the study ?
Forget it. The study passed BE. Go and have a party!
In planning the next study consider using more conservative assumptions (especially about the drop-out rate).
❝ CVintrasubject of AUC and Cmax were below 10%. […] Can we use the posthoc CV to show that the power is more than 80% even with 15 subjects ?
If you look at power-curves you will see that the CV is more important than the sample size. It might well be that the CV (lower than the one assumed in study planning) increased power more than what you “lost” due to the high drop-out rate. But:
Since <80% “post hoc power” does not invalidate a study which demonstrated BE – the opposite is true as well: ≥80% “post hoc power” does not further support a study which already showed BE.
BE is BE in any case. Full stop.*
If the questions (like many similar ones related to power in the past) came from a Malaysian regulator, tell the group that I’ll be happy to perform some basic statistical training for them. No offense, but these same old stories suck.
- A reminder for whoever asked these questions: BE is defined as the inclusion of a confidence interval within an acceptance range. Power does not show up in this definition and is not mentioned in any guideline worldwide.
Homework: 5,000–10,000 BE-studies are performed worldwide each year. How many of them pass BE with power ~60% and are still accepted by regulators?
Edit: See also this post.
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- Revalidate the study Ken Peh 2014-07-20 19:01 [Study Assessment]
- Revalidate the study ElMaestro 2014-07-20 20:04
- α ≠ βHelmut 2014-07-20 22:33