Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-08 17:44 (3916 d 08:58 ago) Posting: # 13353 Views: 25,535 |
|
Dear all, in this recent post I claimed that drop-outs have a lower impact on power of a study than the CV and – especially – the GMR. The order is sample size < CV ≪ GMR. I prepared some R-code (given at the end) which allows a quick comparison. THX to Detlew for reminding me about R’s uniroot() .Two examples showing the influence of actual values deviating from assumptions (I have chosen a minimum power of 70% for the actual study):
In real studies all potential influences occur simultaneously. In the first example both GMR=0.94, CV=23%, n=24 or GMR=0.93, CV=20%, n=23 will result in ~80% power. I know some CROs whose PIs notoriously try to “convince” volunteers not to leave the study being afraid of loosing power. Such a practice is not only unethical but completely unjustified! R-code (developed in R 3.1.6 64-bit, PowerTOST 1.1-12 ):CV <- 0.20 — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2014-08-12 11:12 (3912 d 15:30 ago) @ Helmut Posting: # 13365 Views: 19,807 |
|
Dear Helmut! ❝ R-code (developed in R 3.1.6 64-bit, PowerTOST 1.1-12 ):AFAIK R3.1.6 is not out yet. Or are you using a development version? For the rest of the post: ![]() BTW: Todays scientific quote (Ronald G. Marks) link does not function. — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-12 12:49 (3912 d 13:53 ago) @ d_labes Posting: # 13367 Views: 19,720 |
|
Dear Detlew! ❝ AFAIK R3.1.6 is not out yet. Or are you using a development version?
❝ For the rest of the post: THX. In the meantime I developed code covering all designs; I will post it later. I also played around with ABEL and RSABE – looks funny. ❝ BTW: Todays scientific quote (Ronald G. Marks) link does not function. Oops. Corrected. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-12 19:21 (3912 d 07:20 ago) @ d_labes Posting: # 13370 Views: 20,451 |
|
Dear Detlew and all, below the code for all designs covered in PowerTOST() , conventional (unscaled) ABE. You have to paste the functions only once to the R-console and call the function Sens() with values defined in the end or call it directly, e.g.,Sens(CV=0.24, GMR=1.07, pwr.target=0.8, pwr.min=0.7, des="3x6x3") ![]() Note that the code tries to keep the degree of imbalance as close as possible between sequences. In real life other combinations may occur. As already noted by Detlew in this post the more imbalanced a study is, the lower the power.* Example (3 drop-outs from planned 18 ⇒ 15) power2.TOST(CV=0.2, theta0=0.95, n=c(3, 3, 3, 2, 2, 2), design="3x6x3") require(PowerTOST) @R-Freaks: The construction of imbalanced sequences is both lengthy and clumsy. Any suggestions are welcome. ![]() Too busy to develop the code for RSABE and ABEL at the moment.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2014-08-13 11:30 (3911 d 15:12 ago) @ Helmut Posting: # 13372 Views: 19,775 |
|
Dear Helmut! ❝ below the code for all designs covered in ❝ THX for that code. Maybe there is a better solution than pasting the function into the R-console: Incorporating it into PowerTOST ![]() May need also some thinking about the return value(s) of the function. BTW: What did you mean by your sentence in your post above "This little exercise is not a substitute for the sensitivity analysis which should be performed in study planning ..."? Sens() not useful in real life?❝ Note that the code tries to keep the degree of imbalance as close as possible between sequences. In real life other combinations may occur. As already noted by Detlew in this post the more imbalanced a study is, the lower the power.* This is surely a minor shortcoming of your code. But if we assume drop-outs by random its a reasonable attempt I think in the planning phase where we don't have 'real life' data. And at least I have no "better" idea. ❝ ... ❝ @R-Freaks: The construction of imbalanced sequences is both lengthy and clumsy. Any suggestions are welcome. Will try it if time allows ![]() Idea: use number of sequences = steps from known.designs() and make a general solution of the n's in sequence groups - looping over sequences.❝
If this is truly correct? Don't think so. Simply the design changes to another one. Maybe also to one not implemented. In your example it changes to "3x5x3". Not in known.designs() !This can't handled properly in PowerTOST at moment. No df's, no design constant ... Eventually power2.TOST() should throw an error/warning if it encounters some n's = zero.— Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-13 16:49 (3911 d 09:53 ago) @ d_labes Posting: # 13374 Views: 19,846 |
|
Dear Detlew! ❝ Maybe there is a better solution than pasting the function into the R-console: Incorporating it into PowerTOST Up to you. My primary aim was an educational one – such questions regularly pop up at my workshops and end up in eyes wide opened in disbelief. ![]() ❝ May need some additional effort for checking the input and some effort for documenting. ❝ May need also some thinking about the return value(s) of the function. Yessir. A tabular output would be nice. The plots are less than ideal yet. ❝ BTW: What did you mean by your sentence in your post above "This little exercise is not a substitute for the sensitivity analysis which should be performed in study planning ..."? Only as a “first look” since one of the parameters is varied in each case and the respective others are still “carved from stone”. My understanding of a sensitivity analysis is to explore their combinations, i.e., playing games like: “What happens to power if we have x drop-outs, the CV increases to y% and the GMR shifts by z?” In my protocols I use to come up with a couple of “plausible” ones.Let’s think about my first example, planning for 90% and aiming at a minimum acceptable power for each parameter of 80% this time. That would translate into CV 20⇒24.25%, GMR 0.95⇒0.9208, and n 26⇒19. Although the worst case is possible, it is not realistic. power2.TOST(CV=0.2425, theta0=0.9208, n=c(10, 9)) I don’t think that there is an easy (automatic) solution. One could set a lower limit for the power of individual parameters (say 70%) and with a shotgun-simulation explore combinations which will prevent the power to drop below 80%. Such a simulation must also allow parameters to vary towards “better” values. Tricky. BTW, I knew that the GMR is such a “Prinzessin auf der Erbse”, but was surprised how sensitive (compared to the others) she is. Maybe we could consider to throw the upper CL of the CV into the waste-bin and work with the CI of the GMR (with a high α = the producer’s risk here)? In my first example we get:
Interesting. ❝ ❝ Note that the code tries to keep the degree of imbalance as close as possible between sequences. In real life ❝ ❝ This is surely a minor shortcoming of your code. But if we assume drop-outs by random its a reasonable attempt I think in the planning phase where we don't have 'real life' data. And at least I have no "better" idea. Agree. ❝ ❝ @R-Freaks: The construction of imbalanced sequences is both lengthy and clumsy. Any suggestions are welcome. ❝ ❝ Will try it if time allows ❝ Idea: use number of sequences = steps from Yes, I thought along these lines as well. No spare time at the moment… On the other hand if the code is capsuled in your famous package who cares how elegant it might be? Bringing the execution time down by a fraction of second does not really matter. ❝ ❝ * If a sequence is completely lost, ❝ If this is truly correct? Don't think so. Me not either. ![]() ❝ Simply the design changes to another one […] Not in ❝ Eventually Would make sense, IMHO. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2014-08-13 18:32 (3911 d 08:10 ago) @ Helmut Posting: # 13375 Views: 19,760 |
|
Dear Helmut, here comes my programming gem ![]() #################################### — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-13 18:42 (3911 d 08:00 ago) @ d_labes Posting: # 13376 Views: 19,807 |
|
Dear Detlew, that’s really a gem! ![]() I wasn’t aware that 1.1-13 is already on CRAN. See also some remarks I just added to my previous post about a CL of the GMR. ![]() Edit: Some additions… ❝ ❝ Unavoidable. Sometimes people don’t know which version they (or their IT guys) installed. Maybe it’s safer to replace # can use allways power2.TOST() if PowerTOST V1.1-13 is used by if(packageVersion("PowerTOST") < "1.1.13" & des == "paired") { Most input errors are already trapped by PowerTOST. At the beginning of Sens() add: if(pwr.min < 0.05) stop("Minimum acceptable power must not be <0.05!") … otherwise uniroot() throws an error.Consider adding ... to transfer optional parameters (theta1,…).Tested: R 3.0.2 (2013-09-25), PowerTOST 1.1-12 (2014-07-02) on 32bit Vista Business SP2 R 3.1.1 (2014-07-10), PowerTOST 1.1-13 (2014-08-12) on 64bit Win7 Pro SP1 Sneak Preview ![]() ![]() To do: Explore the loss of power caused by lower variability. Very important for scaling. In the plots I have set the lower limit to 50% of the expected CV. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2014-08-15 11:01 (3909 d 15:41 ago) @ Helmut Posting: # 13382 Views: 19,517 |
|
Dear Helmut, ❝ Sometimes people don’t know which version they (or their IT guys) installed. Maybe it’s safer to replace ❝ ... ❝ by ❝ ❝ } else { ❝ pwrN <- power2.TOST(CV=CV, n=n, theta0=GMR, design=des) ❝ } This is only necessary for a function definition outside package. Within the next release of PowerTOST, which will introduce the sensitivity analysis, the correct version of power2.TOST() is of course available.❝ Most input errors are already trapped by PowerTOST. At the beginning of ❝ ❝ … otherwise Done. Do you know the real reason behind that error? ❝ Consider adding Very good idea ![]() ... '.❝ Sneak Preview Funny ![]() Especially the deviation for the CV! Seems we don't have to care for any deviation here. In the first look surprising. Will send you the code of the package version of your function Sens() soon.Some adaption of the argument names seems necessary to me to be in accordance with the naming within PowerTOST up to know (f.i. pwr.target=targetpower, pwr.min=minpower, GMR=theta0 ...). Although I must confess that I find your naming more appealing. In the output you name the achieved power 'expected' power. But this term is already exhausted for the output of exppower.TOST() .I would suggest the function name to be something like psa.ABE() (power sensitivity analysis for ABE) or psa.TOST() to account for the upcoming things seen in your Sneak Preview.— Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-15 14:02 (3909 d 12:39 ago) @ d_labes Posting: # 13386 Views: 19,533 |
|
Dear Detlew, ❝ ❝ Maybe it’s safer to replace […] ❝ This is only necessary for a function definition outside package. Within the next release of PowerTOST, which will introduce the sensitivity analysis, the correct version of Yes, sure. ❝ ❝ Most input errors are already trapped by PowerTOST. […] otherwise ❝ Done. Do you know the real reason behind that error? 0.05 works and 0.049 doesn’t. Especially looking at the output-vector and the plots not the slightest idea why. ❝ ❝ Consider adding ❝ But ran into trouble doing so with some inexplicable (for me) errors. Seems have to study how to work with the argument ' I’ve tried it myself before. I was hoping that you know better than I do. Not important at the moment. ❝ ❝ Sneak Preview ❝ ❝ Funny ❝ Especially the deviation for the CV! ❝ Seems we don't have to care for any deviation here. In the first look surprising. That’s really nice to show the idea behind scaling. The CV is “dealt with” and the other parameters behave like in TOST. ❝ Some adaption of the argument names seems necessary to me to be in accordance with the naming within PowerTOST […] Sure. ❝ I would suggest the function name to be something like I would suggest to avoid the term “sensitivity analysis”. Looking at the plots novices might get the false impression that effects may occur simultaneously “CV can go up to x%, the GMR to y, and with y drop-outs I have still 80% power”. We should make that very clear in the man-page. My suggestion is pa.TOST() (power analysis for TOST). In my code for scaling I have included both RSABE and ABEL, therefore, pascABEl() and paRSABE() – which would be inline with PowerTOST’s naming conventions is not an option. It should be easy to include FDA’s NTID-scaling as well. Maybe something like paScaledBE() ?— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2014-08-15 13:27 (3909 d 13:14 ago) @ Helmut Posting: # 13383 Views: 19,571 |
|
Dear Helmut! Just tried to mehl you. Answer: "helmut.schuetz ![]() SMTP error from remote server in greeting: host: mx.sil.at: rejected because 212.227.15.15 is in a black list at zen.spamhaus.org Any fall-back? Forget it! Second try was success. — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2014-08-15 13:42 (3909 d 12:59 ago) @ d_labes Posting: # 13384 Views: 19,433 |
|
Dear Detlew! ❝ SMTP error from remote server in greeting: ❝ host: mx.sil.at: ❝ rejected because 212.227.15.15 is in a black list at zen.spamhaus.org Fantastico! I cannot confirm that. I received one 11:33 from your gmx-account. Did you send the one which bounced back from your company’s address? ❝ Any fall-back? helmut.schuetz ![]() I will send you my RSABE/ABEL-code soon. I will post some comments on your previous post as well. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |