Helmut ★★★ Vienna, Austria, 2023-12-18 12:20 (357 d 13:54 ago) Posting: # 23794 Views: 4,993 |
|
Dear all, I stumbled across this (goody ?): An adaptive trial design for testing the I don’t have the paper yet (behind a paywall) but I have some doubts that the method is well thought out:
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2023-12-19 12:10 (356 d 14:05 ago) @ Helmut Posting: # 23796 Views: 4,319 |
|
Dear all, I could not resist and had a closer look (a lengthy -script at the end). Say, we have a 2-sequence 4-period (full) replicate design and start the study in 16 subjects (n1). We observe a CVwR of 0.30. Since swR < 0.294, we have to go with ABE (no scaling). Power based on a fixed GMR 0.95 is below the target of 0.80. Hence, we initiate a second stage. With Pocock’s adjusted α 0.0294 we recruit 6 subjects (n2). We observe a CVwR of 0.30 again and GMR 0.92 in the final analysis of pooled data. Power will be only 0.7093 since the GMR is worse than assumed. However, the Type I Error will be significantly inflated (0.0861 > α). We would have needed at least an adjusted α of 0.0149 (which is substantially lower than the one we used) in order to control the Type I Error. Call the script with the example’s data:
Try a fixed GMR of 0.90 – which is more realistic for HVD(P)s – and you will be surprised. Note also that in the RSABE-branch (swR ≥ 0.294) the empirical Type I Error drops to the adjusted α for CVwR infinitesimal greater than 30%. Example above changed to:
Of course, increasing the sample size in stage 1 does not help in the ABE-branch (swR < 0.294).
Let’s go fully adaptive, i.e., use the observed stage 1 GMR rather than a fixed one. In the final analysis the GMR is worse than in the first stage and the CVwR lower. We use some of the defaults. We have to increase our target power. That’s a guessing game because in the interim we don’t know what will happen in the second stage.
Only if you are a devout follower of the FDA church and believe in the ‘desired consumer risk model’,1 run the first example with
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2023-12-20 14:27 (355 d 11:48 ago) @ Helmut Posting: # 23798 Views: 4,252 |
|
Dear all, I was wrong in this post; the adjusted α can be specified indeed. By trial and error (the documentation is not particularly helpful) I discovered that only a 2×2×4 full replicate design is implemented. Contrary to the functions of PowerTOST unequal variances of T and R are not supported. The total sample size (N) can be an odd number and has to be rounded up in order to obtain balanced sequences.I could reproduce the examples of my yesterday’s post. -script at the end. The first example:
However, it is not possible to estimate power and the empirical Type I Error. By ‘hand’ with yesterday’s GMR 0.92 in the final analysis:
Verdict:
I discovered that some tweaks are needed. It’s bad coding practice in simulations that the seed of the PRNG is not fixed (obviously it’s random). The user should have the option to use a random seed to assess reproducibility (hence, in the functions of
Test of reproducibility
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2023-12-24 14:01 (351 d 12:14 ago) @ Helmut Posting: # 23804 Views: 3,825 |
|
Dear all, something is rotten in the state of Denmark… For the test case you need the script of the previous post and the library PowerTOST .
First the functions of PowerTOST :
Now the function with its defaults. Additionally I set details = FALSE to return the data.frame of results, inter = TRUE to show the intermediate stage 2 sample sizes, and increased the number of calls for the workaround to a crazy number:
Another try with a higher nmax :
sampleN.RSABE() .— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Naksh ☆ India, 2023-12-25 05:16 (350 d 20:59 ago) @ Helmut Posting: # 23805 Views: 3,730 |
|
Hi Helmut, Should't we calculate stage-2 sample size with GMR of 0.95?? I am trying to understand if this will be helpful in case of drugs which are very highly variable e.g. Mesalamine. Hypothetically, if study conducted with N=24, fully replicate design but T/R ratio observed worse like 0.72. And if calculate power with reduced alfa in study using actual result of T/R=0.72, ISCV=0.6978 and N=24 using powerTOST package. power.RSABE(alpha = 0.0294, theta0 = 0.72, CV = 0.6978, n = 24, design = "2x2x4") (which is less than 80%).To recalculate the sample size with actual variability and assumed T/R of 95% and reduced alfa. rss(n = 24, r = 2, S_WR = 0.630, params = list(sig_level=0.0294)) this N=21 is total sample size right?? not the additional sample size (plz correct me if i am wrong) Since, we have already started study with N=24, we cant go ahead with stage-2. Am i missing something? Edit: Subject line changed; see also this post #2. [Helmut] |
Helmut ★★★ Vienna, Austria, 2023-12-25 11:54 (350 d 14:21 ago) @ Naksh Posting: # 23806 Views: 3,821 |
|
Namaste Naksh, ❝ Should't we calculate stage-2 sample size with GMR of 0.95?? ❝ I am trying to understand if this will be helpful in case of drugs which are very highly variable e.g. Mesalamine. First of all let’s attach the libraries and define some variables.
❝ Hypothetically, if study conducted with N=24, fully replicate design but T/R ratio observed worse like 0.72. ❝ And if calculate power with reduced alfa in study using actual result of T/R=0.72, ISCV=0.6978 and N=24 using powerTOST package. ❝ ❝
alpha and targetpower you like):
❝ To recalculate the sample size with actual variability and assumed T/R of 95% and reduced alfa. ❝ ❝ ❝ ❝ this N=21 is total sample size right??
N < n1 is yet another bug.❝ not the additional sample size (plz correct me if i am wrong) GMR , the function’s default 0.95 is used – which is much better than the 0.72 you expect.❝ Since, we have already started study with N=24, we cant go ahead with stage-2. ❝ Am i missing something? rss() is buggy. Though the PE-constraint is implemented (m = 1.25 ) in the list of parameters, the function should throw an error like sampleN.RSABE() if you specify a T/R-ratio outside 1/m … m . Instead it returns nmax , which defaults to 100.
LALA mesalamine is a nasty drug. It’s not the high CVwR which is problematic but the T/R-ratio. If you know already (say, from a pilot study or a failed one) that it’s outside 0.8000 – 1.2500, forget it. Even with a ‘better’ one:
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Naksh ☆ India, 2023-12-25 12:42 (350 d 13:32 ago) @ Helmut Posting: # 23807 Views: 3,757 |
|
Namaste Helmut ! Thanks for the reply and I agree that PE outside is not possible but it has been observed in pilot studies of mesalamine where many BLQs are observed in a subject which skewed the T/R to the extreme. 95% is a very bold to assume in HVD. but should actual T/R ratio for re-estimation of sample size? somehow it should be according to potvin method B? This wonders me with T/R ratio of 85% as well. power.RSABE(alpha = 0.0294, theta0 = 0.85, CV = 0.6978, n = 24, design = "2x2x4") !! |
Helmut ★★★ Vienna, Austria, 2023-12-25 14:15 (350 d 12:00 ago) @ Naksh Posting: # 23808 Views: 3,706 |
|
Namaste Naksh, ❝ […] I agree that PE outside is not possible but it has been observed in pilot studies of mesalamine where many BLQs are observed in a subject which skewed the T/R to the extreme. ❝ 95% is a very bold to assume in HVD. but should actual T/R ratio for re-estimation of sample size? somehow it should be according to potvin method B? nmax = 100 ). That means any re-estimated total sample size >nmax will be considered a failure in stage 1 and reported as nmax . For 2×2×2 and parallel designs that’s implemented in the functions of the package Power2Stage (argument Nmax ).As in the Potvin methods a fixed GMR (not the observed one in stage 1) is used. The authors discussed that in the paper and argued against it (i.e., not going fully adaptive). Implemented in the functions of Power2Stage (argument usePE = TRUE ), though its use requires a careful selection of futility criteria in order not to compromise power.❝ This wonders me with T/R ratio of 85% as well. ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ !! GMR = 0.95 is used (the function’s default). Of course a total sample size less the one in stage 1 is nonsense (read my previous post again).In the second case nmax is reported. By chance? Increase nmax and – surprise, surprise:
Try the script of this post.
However, if you really expect a T/R-ratio of 0.72, it would be both economically and ethically more than questionable to perform a study designed for 0.85. That’s a recipe for disaster!
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Naksh ☆ India, 2023-12-26 05:49 (349 d 20:26 ago) @ Helmut Posting: # 23809 Views: 3,632 |
|
Hi Helumt, Thank you for the nmax info, didnt know. BTW your RSABE.TSD script seems very useful !! Thanks again for sharing with us. |
Helmut ★★★ Vienna, Austria, 2023-12-26 13:50 (349 d 12:25 ago) @ Naksh Posting: # 23811 Views: 3,635 |
|
❝ Hi Helumt, ❝ Thank you for the nmax info, didnt know. ❝ BTW your RSABE.TSD script seems very useful !! I don’t think that a TSD is useful at all, unless you use the observed GMR of stage 1 (argument usePE = TRUE ) and a reasonable Nmax . But (‼) if your final GMR is really that bad, you will fail – even if the CVwR is larger (which means more scaling).
We could observe the PE-constraints according to the guidances: Stop for futility in the interim if \(\small{GMR\notin\left\{>0.80\;\wedge<1.25\right\}}\). However, for such an approach the GMR has to be reasonably accurate, i.e., the stage 1 sample size sufficiently large. Don’t ask me what would be sufficient… Larger deviations between GMRs are a direct consequence of higher variability. The FDA requires at least 24 enrolled subjects in replicate designs intended for reference-scaling. If the CVwR is expected to be large, I would opt for substantially more in order to get a reliable estimate.With the results of your first stage with 24 subjects:
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |