Log in
Register|
ElMaestro ★★★ Denmark, 2011-10-31 10:17 (5344 d 23:53 ago) Posting: # 7563 Views: 10,588 |
|
|
Hi all, what is your perceived absolute minimum limit for prospective power in order for a trial to be ethical and non-futile? My own opinion would be 80%, but I do acknowledge that (fortunately!) the rest of the world rarely agrees with me and the limit in practice differs from IEC/IRB to IEC/IRB etc. Best regards, EM |
|
Dr_Dan ★★ Germany, 2011-10-31 10:46 (5344 d 23:24 ago) @ ElMaestro Posting: # 7564 Views: 9,518 |
|
|
Dear ElMaestro The absolute minimum limit for prospective power in order for a trial to be ethical and non-futile should be defenitely >50%.  In the majority of the studies I conducted the prospective power was about 90% and IMHO you should always plan with a prospective power of more than 80% as long as you have a drug with an acceptable variability and no further information (for example data from a pilot study). Power is always connected with sample size. If you need 24 subjects to demonstrate bioequivalence with a power of 80% the difference in price of the study conducted with 22 or 26 subjects does not really matter. But if you need 200 subjects then the risk/cost balance becomes important. However, IMHO if you plan a study with less than 70% prospective power the study becomes un-ethical and non-futile. There are other possibilities to influence the risk/cost balance than just adjust the sample size to maximum price you are willing to spend. What about an add-on design? Kind regards Dan — Kind regards and have a nice day Dr_Dan |
|
ElMaestro ★★★ Denmark, 2011-11-01 10:59 (5343 d 23:10 ago) @ Dr_Dan Posting: # 7570 Views: 9,457 |
|
|
Hi Dr_Dan, ❝ What about an add-on design?  You are reading my thoughts, I think.A background for my question is that I am fiddling a teenytiny bit with two-stage designs like those of Potvin et al. In the second step the target nominal power is 80% but due to the nature of the approach this figure is in some cases not achieved. Thus if we can define a realistic bsolute lower limit for power then we can tweak the method parameters so as to keep power above the magical or arbitrary threshold. More info to follow, I hope. Best regards, EM. |
|
d_labes ★★★ Berlin, Germany, 2011-11-01 11:28 (5343 d 22:41 ago) @ ElMaestro Posting: # 7572 Views: 9,502 |
|
|
Dear EM, ❝ A background for my question is that I am fiddling a teenytiny bit with two-stage designs like those of Potvin et al. In the second step the target nominal power is 80% but due to the nature of the approach this figure is in some cases not achieved. Thus if we can define a realistic bsolute lower limit for power then we can tweak the method parameters so as to keep power above the magical or arbitrary threshold. IMHO you are totally right in seeing the 80% power as an empirical parameter of the 2-stage evaluation which has no direct connection to the real power attained. The Table I in Potvin et al. consistently show power empirical estimates (emphasis is for members of regulatory bodies  ) between 70 to 80% if the CV is above 30% and sample size in stage 1 is low. Since the authors don't discuss this attained power <80% they seem to see this not as an issue and seem to accept power >70% as sufficient.If you don't share this opinion you could of course adapt the 'power' in the sample size adaptation step to fit your needs. Another possibility is the use of sample sizes based on 'expected' power which are usually higher than the ordinary ones. This should let to a power increase of the 2-stage evaluation. — Regards, Detlew |
|
Dr_Dan ★★ Germany, 2011-10-31 13:55 (5344 d 20:15 ago) @ ElMaestro Posting: # 7565 Views: 9,531 |
|
|
Hi again For studies detecting clinical efficacy Cohen (1969) suggests to use a beta 4 times the value of the significance niveau. If alpha = 5% then the beta mistake niveau should be < 20% => Power = 1-beta = >80%. Cohen, J. (1969). Statistical power analysis for the behavioral sciences. San Diego, CA: Academic Press. Hope this helps Kind regards Dan — Kind regards and have a nice day Dr_Dan |
|
Helmut ★★★   Vienna, Austria, 2011-10-31 15:41 (5344 d 18:28 ago) @ ElMaestro Posting: # 7566 Views: 9,559 |
|
|
Dear ElMaestro! I would also go with 80%. If you want to get some hints from various re(li)gions:
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png) Helmut Schütz ![[image]](https://static.bebac.at/img/CC by.png) The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
vinayshed ☆ 2011-11-01 06:59 (5344 d 03:11 ago) @ Helmut Posting: # 7569 Views: 9,459 |
|
|
Dear HS, Whenever there is discussion on "Power", I remember your presentation this Jan on "Myth of Power" and the question asked by one participant at the end of the workshop. I am sure you remember it. Vinay |
|
d_labes ★★★ Berlin, Germany, 2011-11-01 11:04 (5343 d 23:06 ago) @ ElMaestro Posting: # 7571 Views: 9,418 |
|
|
Dear ElMaestro! To add my two cents: The original sample size tables of Diletti et al. [1] contain the target powers 70, 80 and 90%. Thus these Authors must had the feeling that 70% is a reasonable lower bound. That's also my feeling. But at the end it's a question what risk of erroneous not concluding BE the sponsor is willing to accept. The usual 80% power to attain is only a convention like 5% alpha. But had made it into regulations. The source of this number was already stated by Dan. But I would not give a cut-off value as futile because I think that a target power of f.i. 0.778 is equal sufficient like 0.8. As we all know the power is highly dependent on our presumptions concerning variability and true GMR. If we miss them to some extent in the real outcome of our study we are always in danger of having not correctly powered the study. This is the case for the sensitivity analysis of sample size planning Helmut is not tired to preach every time. [1] Diletti E, Hauschke D, Steinijans VW Sample Size Determination for Bioequivalence Assessment by Means of Confidence Intervals Int. J. Clin. Pharm. Ther. Tox., Vol. 30 Supl. 1, S51-58, 1992 — Regards, Detlew |
|
ElMaestro ★★★ Denmark, 2011-11-01 11:46 (5343 d 22:23 ago) @ d_labes Posting: # 7573 Views: 9,467 |
|
|
Good morning d_labes, very interesting, I think two of our perspectives here differ. ❝ But at the end it's a question what risk of erroneous not concluding BE the sponsor is willing to accept. ❝ But I would not give a cut-off value as futile because I think that a target power of f.i. 0.778 is equal sufficient like 0.8. ❝ (...) the sensitivity analysis of sample size planning Helmut is not tired to preach every time. EM. |
|
d_labes ★★★ Berlin, Germany, 2011-11-01 12:28 (5343 d 21:42 ago) @ ElMaestro Posting: # 7575 Views: 9,461 |
|
|
Good morning Mon Capitaine, ❝ For me at the end it is more about how low the power can be before ethics committees etc will flag trials as futile. Since I have always done sample size planning with a target power >80% I haven't seen any issue regarding this raised by an IEC/IRB in my career. Did you encounter something here? ❝ ❝ But I would not give a cut-off value as futile because I think that a target power of f.i. 0.778 is equal sufficient like 0.8. ❝ Personally, I would be inclined to think that if regulators want a 90% CI with two decimals then we should consider power in a similar strict fashion since there's an ethics perspective involved.  Didn't got your point.❝ ❝ (...) the sensitivity analysis of sample size planning Helmut is not tired to preach every time. ❝ Yeah, that guy talks a bloody lot doesn't he? Really. But he should change his target audience to regulatory bodies .— Regards, Detlew |
|
Ohlbe ★★★ France, 2011-11-01 23:57 (5343 d 10:12 ago) @ d_labes Posting: # 7578 Views: 9,395 |
|
|
Dear Maestro and D. Labes, ❝ ❝ ❝ But I would not give a cut-off value as futile because I think that a target power of f.i. 0.778 is equal sufficient like 0.8. ❝ ❝ Personally, I would be inclined to think that if regulators want a 90% CI with two decimals then we should consider power in a similar strict fashion since there's an ethics perspective involved. ❝ ❝ ❝ (...) the sensitivity analysis of sample size planning Helmut is not tired to preach every time. ❝ ❝ Yeah, that guy talks a bloody lot doesn't he? ❝ ❝ Really. But he should change his target audience to regulatory bodies He will have to change his target to Ethics Committees if they start asking for power to be higher than 80.00 %... I have always been told that you can't give a result with more digits than whatever you have used to calculate it. To calculate a power of 80.00 % you would need to assume T/R = 0.9500 and an intra CV of, say, 23.45 % from you pilot study... Ridiculous. El Maestro, I would consider differently the number of decimals expected for 90 % CI (where you have acceptance limits, which we know regulators now consider more and more as "death or life" limits for the product), and for power, where this thread shows that there is no universal acceptance threshold. Regards Ohlbe — Regards Ohlbe |
|
ElMaestro ★★★ Denmark, 2011-11-02 10:27 (5342 d 23:42 ago) @ Ohlbe Posting: # 7579 Views: 9,412 |
|
|
Dear Ohlbe, ❝ I have always been told that you can't give a result with more digits than whatever you have used to calculate it. To calculate a power of 80.00 % you would need to assume T/R = 0.9500 and an intra CV of, say, 23.45 % from you pilot study... Ridiculous. Diplomatic as always ❝ El Maestro, I would consider differently the number of decimals expected for 90 % CI (where you have acceptance limits, which we know regulators now consider more and more as "death or life" limits for the product), and for power, where this thread shows that there is no universal acceptance threshold. Please don't get me wrong. I am discussing a lower limit for power. We can found it in sponsor's willingness to accept risk or we can discuss it on basis of ethics or something else. A power of 7% would be ridiculous regardless of the number of decimals, and 100% is never achieved, so the limit must be somewhere in between. In practice the figure 80% or 90% is commonly used, and in practice many sources of info where sponsors get their CV's are not given with any decimals. That's not the issue. However, let's say purely hypothetically that somewhere a simulant is fiddling with his compiler and checking how second stage minimum power requirements and caps on subjects numbers affect type I errors and overall power. For a compiler 80% means the same as 80.00000000%. It's not because 17 decimals are relevant but that's just the way a double-precision floats works in practice. Lastly, if regulators find it relevant to specify acceptance ranges with two decimals (125.00% instead of just 125%) then I would be able to understand why a minimum power requirement on basics of ethics could be specified similarly. This does not mean I find power with two decimals relevant or that I would be able to understand why it is necessary to specify an acceptance range with two decimals. Rather it just means I find power with to decimals just about as relevant as acceptance ranges with two decimals. You can probably read a little between the lines here, too. And in practice of course a study is never powered to 80.00%, because noone has yet come up with a way to include 34.173 patients (or whatever) in a study. Have a great day. EM. |
|
Ohlbe ★★★ France, 2011-11-02 12:00 (5342 d 22:09 ago) @ ElMaestro Posting: # 7582 Views: 9,376 |
|
|
Dear El Maestro, ❝ Lastly, if regulators find it relevant to specify acceptance ranges with two decimals (125.00% instead of just 125%) then I would be able to understand why a minimum power requirement on basics of ethics could be specified similarly. This does not mean I find power with two decimals relevant or that I would be able to understand why it is necessary to specify an acceptance range with two decimals. Rather it just means I find power with to decimals just about as relevant as acceptance ranges with two decimals. You can probably read a little between the lines here, too. Yes, sure On the other hand, there are several factors that may have influenced regulators' decision to ask for these decimals. They are clearly not purely scientific, but I can understand that regulators may need to take them into account too: - one is the lack of transparency of some BE reports... If you have an upper limit of 125.49, and round it to 125 in the report, that's not totally fair, is it ? Maybe it is not relevant for that specific drug, but maybe the regulators may think otherwise. If they don't have that information, they can't make a clearly informed decision. As often, excess in one direction can result in excess in another direction. - one is increased transparency of regulators decisions and the need for regulators to justify what they decide. More and more data get publicly available, in public assessment reports or through freedom of information provisions. If it becomes known publicly that product A was approved with an upper limit of 125.49 %, the manufacturer of the originator will make a hell of a noise about it, go to court, politicians will become involved, propose measures to restore public confidence in generics, and make everybody's life hell. OK, I'm getting more and more off-topic here ! ❝ And in practice of course a study is never powered to 80.00%, because no one has yet come up with a way to include 34.173 patients (or whatever) in a study. That would be clearly unethical Best regards Ohlbe — Regards Ohlbe |
|
Dr_Dan ★★ Germany, 2011-11-01 14:10 (5343 d 20:00 ago) @ ElMaestro Posting: # 7576 Views: 9,401 |
|
|
Hi ElMaestro Do you really think that an ethics committee will focus on the power of the study? As already said power is a question what risk of erroneous not concluding BE the sponsor is willing to accept. In other words the power of a study is the sponsorś problem. Each study protocol includes a paragraph giving a justification for the sample size. Based on this sample size estimation the ethics committee will decide if the study would be futile or not. So if the sponsor provides a reasonable explanation why to use a power less than 80% than it is up to the ethics committee to accept this explanation or not. IMHO this is the right procedure and I am strictly against a cut-off value for the power. Kind regards Dan — Kind regards and have a nice day Dr_Dan |
|
ElMaestro ★★★ Denmark, 2011-11-01 14:53 (5343 d 19:17 ago) @ Dr_Dan Posting: # 7577 Views: 9,457 |
|
|
Hi Dr_Dan, ❝ Do you really think that an ethics committee will focus on the power of the study? As already said power is a question what risk of erroneous not concluding BE the sponsor is willing to accept. In other words the power of a study is the sponsorś problem. Yes, I firmly believe ethics committees must take power into consideration and give a red card if necessary. There is no reason to initiate a trial if it does not have a reasonable chance of leading to a conclusion. Note that inspection of the ethics committees, their work, their approvals, is part of FDA's inspection programme. In the EU it works a little differently but GCP (and other) principles, however vaguely they are formulated, must be observed. Best regards, EM. |
|
Dr_Dan ★★ Germany, 2011-11-02 11:55 (5342 d 22:15 ago) @ ElMaestro Posting: # 7581 Views: 9,432 |
|
|
Dear all I do not really understand this discussion. Is there really a need for setting a limit for the prospective power of a study? A) The prospective power is based on an assumption of an intra-subject variability. As you know the CV% varies from study to study and you will never find the one and only information that for elmaestromycin the true CV is 23.765%. B) From the sponsorś perspective why should I ask for further restrictions? C) Where is the scientific basis telling you that a prospective power of 69% is futile and a power of 71% is reasonable? When does power become unethical? I agree that we should avoid unnecessary burden for study participants but this should be judged using common sense. Kind regards Dan — Kind regards and have a nice day Dr_Dan |
|
ElMaestro ★★★ Denmark, 2011-11-02 13:01 (5342 d 21:08 ago) @ Dr_Dan Posting: # 7583 Views: 9,423 |
|
|
Dear Dr_Dan, ❝ I do not really understand this discussion. Is there really a need for setting a limit for the prospective power of a study? ❝ A) The prospective power is based on an assumption of an intra-subject variability. As you know the CV% varies from study to study and you will never find the one and only information that for elmaestromycin the true CV is 23.765%. Yes. And you'll be completely hallucinated after 5 mg. ❝ B) From the sponsorś perspective why should I ask for further restrictions? I agree that it is often a good idea to avoid complications. ❝ C) Where is the scientific basis telling you that a prospective power of 69% is futile and a power of 71% is reasonable? When does power become unethical? There isn't any scientific justification for a limit as far as I know. This is like the 5% significance limit commonly accepted in statistics, or the 100 km/hr speed limit on Danish high ways, or the 80%-125% BE acceptance range, or the limits for cucumber curvature. They are arbitrary. Best regards, EM. |
Ing. Helmut Schütz