Log in
Register|
Marcel ★ 2011-03-08 11:56 (5166 d 10:49 ago) Posting: # 6713 Views: 7,231 |
|
|
Good morning, What is the consumer risk in having conducted a study with too few subjects (a posteriori judgement)? Despite this apparent lack of power, the confidence intervals were within the accepted range (80-125). Do you need more information than this to formulate a response? Thank you for any feedback on this. Edit: Category changed. [Helmut] |
|
ElMaestro ★★★ Denmark, 2011-03-08 12:30 (5166 d 10:16 ago) @ Marcel Posting: # 6714 Views: 6,342 |
|
|
Hi Marcel, ❝ What is the consumer risk in having conducted a study with too few subjects (a posteriori judgement)? Despite this apparent lack of power, the confidence intervals were within the accepted range (80-125). Do you need more information than this to formulate a response? The consumer risk is irrelevant at this stage. If you have proven BE with the standard 5% overall alpha then the product is BE. The fact that you may have initiated the trial with a low number of subjects is of potential ethical concern, but that does not affect the product. If at all interested, they should pick up on this at the trial application stage. It isn't against the law to have a bit of luck. — Pass or fail! ElMaestro |
|
Marcel ★ 2011-03-08 13:01 (5166 d 09:44 ago) @ ElMaestro Posting: # 6715 Views: 6,384 |
|
|
Thanks, ElMaestro, for the prompt response. They have picked up on it and are calling us on it (one CMS only). We have used the standard 5% overall alpha. If I understand correctly, this is the consumer risk and it has remained the same, correct? The CMS is taking the position that aposteriori power is not sufficient. I'm trying to get my head around that. Seems to me they are confusing consumer risk with producer risk. Anything else you can add? Thank you. Edit: Full quote removed. Please delete anything from the text of the original poster which is not necessary in understanding your answer; see also this post! [Helmut] |
|
ElMaestro ★★★ Denmark, 2011-03-08 13:13 (5166 d 09:33 ago) @ Marcel Posting: # 6716 Views: 6,475 |
|
|
Hi Marcel, ❝ They have picked up on it and are calling us on it (one CMS only). We have used the standard 5% overall alpha. If I understand correctly, this is the consumer risk and it has remained the same, correct? The CMS is taking the position that aposteriori power is not sufficient. I'm trying to get my head around that. Seems to me they are confusing consumer risk with producer risk. Anything else you can add? I can add that in such a situation I would keep cool. If the CMS triggers a referral the CMS will probably be butchered by the other member states (assuming of course that no other issues exist and everything else is perfect). There is no other parameter than α which controls the consumer risk for a given formulation and comparison because the acceptance criterion is a 1 – 2×α CI for T/R. You should of course point this out in a succinct manner in your response to the authorities. — Pass or fail! ElMaestro |
|
Helmut ★★★   Vienna, Austria, 2011-03-08 13:27 (5166 d 09:18 ago) @ Marcel Posting: # 6717 Views: 6,444 |
|
|
Dear Marcel! ❝ They have picked up on it and are calling us on it (one CMS only). The CMS was not by any chance Greece? They were notoriously asking for a posteriori power in the past.  ❝ We have used the standard 5% overall alpha. [...] this is the consumer risk and it has remained the same, correct? Exactly (see ElMaestro’s posts). ❝ The CMS is taking the position that aposteriori power is not sufficient. On the contrary – a posteriori power is meaningless. Can you give us the exact wording they have used? ❝ I'm trying to get my head around that. Maybe one of my presentations helps (slides 26, 40, 47 and the references given). ❝ Seems to me they are confusing consumer risk with producer risk. Probably – likely. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png) Helmut Schütz ![[image]](https://static.bebac.at/img/CC by.png) The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Marcel ★ 2011-03-08 13:50 (5166 d 08:55 ago) @ Helmut Posting: # 6718 Views: 6,332 |
|
|
Thanks to both of you. ❝ The CMS was not by any chance Greece? ❝ They were notoriously asking for a posteriori power in the past. They still do, but they withdrew their objection (it was in a different DCP). This is another Mediterranean country that just got a rolling over of assessors in this particular department (this country approved the product in a previous DCP!!!). ❝ On the contrary – a posteriori power is meaningless. Can you give us the exact wording they have used? I pointed to a paper by Hoenig and Hensey you also quote at the end of your presentation (which I'll give a more meticulous look at shortly). But they just tried to use it against us, but I couldn't follow their logic, since it basically contradicted their stance that post hoc power has meaning. This is the wording they are using: The reduced number of the enrolled subject confers to the study a weak perspective power (just above 0.5). Therefore the BE evaluation could not be reliable. I'm trying to figure out if they are talking about producer or consumer risk here. Thanks to the already very useful information provided. |
|
ElMaestro ★★★ Denmark, 2011-03-08 14:06 (5166 d 08:39 ago) @ Marcel Posting: # 6720 Views: 6,399 |
|
|
Hehe, ❝ This is the wording they are using: The reduced number of the enrolled subject confers to the study a weak perspective power (just above 0.5). Therefore the BE evaluation could not be reliable. "If the Agency believes that the application alpha of 5% and a 1-2*alpha confidence interval in some cases conveys an unacceptably high risk to consumers, then we suggest that the Agency introduces this concern to the Pharmacokinetics Working Party and the Statistics Working Party under the CHMP for a possible change of the current guidance. Until such a change has been justified and implemented we strongly believe that the risk is sufficiently and appropriately controlled using the current criteria that are based on alpha=5% and which other EU and EES countries routinely rely on." — Pass or fail! ElMaestro |
|
Helmut ★★★ Vienna, Austria, 2011-03-08 14:41 (5166 d 08:04 ago) @ Marcel Posting: # 6722 Views: 6,366 |
|
|
Dear Marcel! ❝ ❝ The CMS was not by any chance Greece? ❝ ❝ They still do, but they withdrew their objection (it was in a different DCP). Unbelievable!  ❝ I pointed to a paper by Hoenig and Hensey you also quote at the end of your presentation [...]. But they just tried to use it against us, but I couldn't follow their logic, since it basically contradicted their stance that post hoc power has meaning. ❝ ❝ This is the wording they are using: The reduced number of the enrolled subject confers to the study a weak perspective power (just above 0.5). Therefore the BE evaluation could not be reliable. Crazy. I guess you planned (!) for power of ≥80% and suffered from one of the followings (or any combination thereof)?
I’m not sure what they mean by: ❝ The reduced number of the enrolled subject... I hope you suffered from #2 above. If you really enrolled less subjects than planned, this might be of ethical concerns - but still does not invalidate the study. ❝ I'm trying to figure out if they are talking about producer or consumer risk here. Likely about the former. The latter is fixed at 5% (see ElMaestro’s nice answer). ❝ Therefore the BE evaluation could not be reliable. Nuts! If you initiated the study with power “just above 0.5” that’s bad practice. You took a high risk of failure – and succeeded. If you were a victim of #2 nothing to worry about. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Marcel ★ 2011-03-08 15:13 (5166 d 07:32 ago) @ Helmut Posting: # 6723 Views: 6,409 |
|
|
❝ Crazy. I guess you planned (!) for power of ≥80% and suffered from one of the followings (or any combination thereof)? ❝ ❝ ❝ During a pilot study, we obtained intra-subject CV about half of what is now published, so this authority basically is saying that we underpowered the study because the CV we obtained is not possible. Having read your presentation, I can see why pilot info isn't all that great for sample size estimation. Day 210 is tomorrow, so we'll likely get referred to CMD. Will keep you posted. Thank you both. |
|
Helmut ★★★ Vienna, Austria, 2011-03-08 15:27 (5166 d 07:19 ago) @ Marcel Posting: # 6724 Views: 6,363 |
|
|
Dear Marcel! ❝ During a pilot study, we obtained intra-subject CV about half of what is now published, so this authority basically is saying that we underpowered the study because the CV we obtained is not possible. Not possible‽ What were the CVs actually?
❝ Having read your presentation, I can see why pilot info isn’t all that great for sample size estimation. I wouldn’t say that. Only in a pilot study you can control everything (clinical setting, blood sampling time points, bioanalytical method,…). If I have the chance to choose I would always opt for a pilot study. But the size shouldn’t be too small and the CVintra not taken literally (allowing for a safety margin). In the future you might consider a Two-Stage design... ❝ Day 210 is tomorrow, so we'll likely get referred to CMD. Good luck! — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Marcel ★ 2011-03-08 15:49 (5166 d 06:56 ago) @ Helmut Posting: # 6725 Views: 6,367 |
|
|
Dear Helmut, ❝ What were the CVs actually? ❝ Typically in mid-40 range (ibandronate) ❝ Internal data showed mid-twenties ❝ Our study showed 26.1 ❝ ❝ Having read your presentation, I can see why pilot info isn't all that ❝ ❝ great for sample size estimation. We've attempted sequential design for other products. This option was not really available 4-5 years ago. Thanks for all your feedback. |
|
Helmut ★★★ Vienna, Austria, 2011-03-08 16:03 (5166 d 06:42 ago) @ Marcel Posting: # 6726 Views: 6,381 |
|
|
Dear Marcel, actually you have proven the CV from the pilot in your pivotal study. This renders the argument of the agency as straight  !It’s common for bisphonates that published (=old) studies show higher variability than more recent ones due to improvements in bioanalytics. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Marcel ★ 2011-03-08 16:28 (5166 d 06:17 ago) @ Helmut Posting: # 6727 Views: 6,370 |
|
|
Dear Helmut, Yes, I thought so. Thanks for the feedback (ElMaestro too) |
|
ElMaestro ★★★ Denmark, 2011-03-10 00:16 (5164 d 22:29 ago) @ Marcel Posting: # 6734 Views: 6,082 |
|
|
Hi Marcel, ❝ Day 210 is tomorrow, so we'll likely get referred to CMD. Let's hear the good news. — Pass or fail! ElMaestro |
Ing. Helmut Schütz