Helmut ★★★ Vienna, Austria, 2006-12-28 22:33 (6558 d 12:39 ago) Posting: # 422 Views: 55,953 |
|
Dear all, following this post and just out of curiosity I implemented SAS-code by Jones and Kenward (2000) [1] in R (v2.4.1).To quote the beginning of Jones'/Kenward's SAS-Code (which is copyrighted material and therefore not given here; but you can afford buying a book for sure): /*** WARNING : PROGRAM OFFERED FOR USE WITHOUT ANY GUARANTEES ***/ Their example (pp 336-337) yields from a within-subject standard deviation of 0.355 in 58 subjects and a point estimate of 1.00 power=90.4%. Following R-code gives 90.3988%. # significance level The following code reproduces the power-curves for n=8/12/18/24/30/36/40/60 and CV=20% given in Fig.1c by Diletti et al. (1991) [2]
a <- 0.05 # alpha Values calculated by the R-code for n=24 match Diletti et al.'s Fig.1c: +------+-------+ Please note, this code is not validated, but only a toy you can play around with! With some ambition you can modify the code (essentially varying the point estimate instead of the sample size) in order to match also Diletti et al.'s Fig.2c. Now it should be only a small step to re-calculate their table of sample sizes...
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2007-01-02 00:59 (6554 d 10:13 ago) (edited on 2007-06-04 12:41) @ Helmut Posting: # 428 Views: 45,756 |
|
Dear all, I performed some comparisons of the code based on Jones/Kenward implemented in R, Diletti's table and results obtained from StudySize (v2.0.1). Quite interesting... All comparisons were done for CV=20% with ratios of 0.85–1.20. Dilleti reported sample sizes to obtain ≥80% power, calculated odd sample sizes were reported rounded up to the next even number ( underlined ):
+======+=====+===========+===========+ Power with sample sizes given by Diletti et al. at a GMR of 1.05 were below 80%, calculated both with R and StudySize… A Monte Carlo Simulation (1000000 runs) for GMR=1.05 and 18 subjects in StudySize resulted in: Power 0.8006 (95% coverage probability: 0.7998-0.8013 ).Differences may be due to the implementation of the algorithm to obtain the value of the noncentral t-distribution by numeric integration... Maybe somebody of you has access to SAS or software specialized in power analysis (e.g., PASS or nQuery Advisor) and would like to check these results? — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2007-01-06 16:17 (6549 d 18:55 ago) @ Helmut Posting: # 441 Views: 45,644 |
|
Dear all, Dieter Hauschke informed me that Jones’/Kenward’s code uses Satterthwaite’s approximation for degrees of freedom (for heterogenous variances), which was corrected in a recent book. [1] In power calculations degrees of freedom are fixed by the number of subjects (and sequences). If you want to play around again, please replace code line #22
Power (CV=20%) by the (old) new codes for n=24 are:
+======+=======+=======+ Power >80% (CV=20%) by the (old) new codes are:
+======+=====+===========+===========+ Power is >80% for all combinations tested. ❝ Maybe somebody of you has access to SAS or software specialized in power analysis (e.g., PASS or NQuery) and would like to check these results? According to Dieter Hauschke Owen’s exact method is implemented in nQuery Advisor. Since there is still some confusion about different methods for sample size estimation in bioequivalence (software, tables, approximations) an entire chapter of a new book [2] will be devoted to this issue.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
mathews ● 2008-04-03 15:19 (6096 d 20:53 ago) @ Helmut Posting: # 1757 Views: 44,367 |
|
Dear all, Please tell me how to calculate power for a replicate crossover design (2 treatment 2 sequence 4 period). Regards Matz |
Helmut ★★★ Vienna, Austria, 2008-04-03 15:49 (6096 d 20:23 ago) @ mathews Posting: # 1758 Views: 45,389 |
|
Dear Matz, you will get the same power in ~½N subjects (TRRT|RTTR) as compared to a 2×2×2 design in N subjects (half the sample size - but same number of treatments!). If you perform a study in a 2-treatment 2-sequence 3-period design (e.g., TRT|RTR) the same power is obtained in ~¾N subjects. 1)
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
mathews ● 2008-04-04 15:21 (6095 d 20:50 ago) @ Helmut Posting: # 1763 Views: 43,850 |
|
Dear HS, Thanks for your reply. How we can modify the above R code for a replicate crossover design. The above code used within subject s.d for a 2x2 design, but in a replicate crossover design we will get within subject s.d for both test and reference. So which value should be used?... Regards, Matz |
Helmut ★★★ Vienna, Austria, 2009-09-25 21:20 (5556 d 14:52 ago) @ Helmut Posting: # 4255 Views: 42,953 |
|
Dear all, I made an investment and was able to compare results from R with the current version 7 of nQuery Advisor:*
+======+=====+===========+========+==========+ nQuery requires sqrt(MSE) rather than CV as input = sqrt(ln(CV²+1)). The maximum precision is 6 decimal places. Therefore 0.1980422 rather than the value in full precision (0.1980422004353650284627675024887) like in R is used. But you see this value only if clicking the respective cell; both displayed and printed is 0.198042. nQuery gives power in percent with a maximum number of two decimal digits. That's a pity and values in the first column were the best one could get (display/print). If clicking in each cell power is given in percent to 5 decimal digits (7 significant digits); second column. It's clear that results are not rounded, but truncated. This looked weird first, but at least for power it makes sense. Values agree nicely with the exception of a small discrepancy at GMR=1 (83.31980% vs. 83.32001%). If I recall it correctly Dieter Hauschke wrote something about numerical problems in power estimation at T/R=1 in one of his papers. I love validating software.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2009-09-28 01:27 (5554 d 10:45 ago) @ Helmut Posting: # 4256 Views: 42,511 |
|
Hi HS, ❝ [Blahdeeblah] Therefore 0.1980422 rather than the value in full precision (0.1980422004353650284627675024887) [more blah] Try to put yourself in the shoes of the software writer. There are two known options to getting the calculation of power. One is the bruteforce approach, the other goes via probabilities of the non-central t distribution. The former can be cumbersome and is generally not preferred (takes time but converges asymptotically to the true power with the number of resamples if the random number generator works (and trust me, it is not that hard to make one that does!)), but there is still preference for the latter. Now, actually programming a stable Al Gore Rhythm which get the probs from the nct dist is a nightmare, so the developer of course googles it and finds some really neat source code. This is what the developers of R did, and a dollar gets ten that this is what others do as well. And here's the deal: If you look at that source you can easily see that it is based on some (necessary) helper constants and short-cuts and convergence criteria and what not. In other words, whatever the algo does it is not exact. It is approximate, but how good that approximation actually is is unknown (unknown error will be a function of the conditions, by the way). We know R's power is an approximation since it uses an adapted ASA243. So when two or more applications get the same result it could be either because they are both right or because they both make the same errors (as would be the case if they are based on mods of the same algo etc), or it could be pure chance. And we have absolutely no way of telling which one is the right one until some clever guy works out the integrals exactly. I'll buy you a Mozartkugel if you do it. ❝ I love validating software. Decompile your recent acquisition and check if they one way or another use the ASA243. If they do, why validate? EM. |
Helmut ★★★ Vienna, Austria, 2009-09-28 17:21 (5553 d 18:50 ago) @ ElMaestro Posting: # 4260 Views: 43,528 |
|
Hi ElMaestro! ❝ Try to put yourself in the shoes of the software writer. Oh, I took these boots on many years ago. And they got muddy almost immediatelly. I tried to get a numerical approximation of the cdf of the t-distribution which lead me to the gamma-function and headaches. Finally I ended up with numerical approximations.[1,2] ❝ […] whatever the algo does it is not exact. It is approximate, but how good that approximation actually is is unknown. Sure. ❝ […] we have absolutely no way of telling which one is the right one until some clever guy works out the integrals exactly. Definitely not me. I’m getting more stupid every day. ❝ I'll buy you a Mozartkugel if you do it. THX, no needs. ❝ Decompile your recent acquisition and check if they one way or another use the ASA243. I guess the SW was built using the Borland C++ compiler. You know that reverse engineering of software is a breach of the license? BTW, according to the manual the ‘Illinois method’ (Algorithm AS 184, FORTRAN source) is used. I assume AS 243[3] is ‘better’ than AS 184.[4] But to which extent? Is it my job to go through pros and cons of algorithms? ❝ If they do, why validate? First, different algos are used. Second, I have to validate software - or not? At the product's site I found a nice statement: These sets of solutions were reviewed by Janet Elashoff, who checked for consistency, face validity, and for computational accuracy against other sources. Face validity?What I actually wanted to do was to check my R code (which is also part of bear) with a piece of SW of high reputation. Checking the sample size I wasn’t satisfied (only integers), therefore I opted to do it the other way 'round (power). It made me angry that there is no (easy) way to obtain more than 4 significant digits. By this external validation is effectively prevented. Another example from the dirt track of SW validation: Diletti et al.[5] published exact samples size tables for BE ranges of 0.9-11 and 0.7–1.43. When trying to validate my code against these tables I found small discrepancies (only at low CVs and close to the acceptance range). By trial-and-error I got the solution: I could reproduce tables only if setting the upper acceptance limit not to the reciprocal of the lower AL (0.9-1, 0.7-1) but to exactly 1.1111 or 1.4286… BTW, nitpicking Diletti’s Table 1[6] (power 70%, T/R=1, CV=7.5%) and nQuery 7 give n=4, whilst my R code, StudySize 2.0.1, and FARTSSIE 1.6 come up with n=6. Now what? Go with a democratic vote of 3:2 for 6?
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2009-09-28 21:12 (5553 d 15:00 ago) @ Helmut Posting: # 4261 Views: 42,599 |
|
Dear HS, ❝ Oh, I took these boots on many years ago. And they got muddy almost immediatelly. I tried to get a numerical approximation of the cdf of the t-distribution which lead me to the gamma-function and headaches. ❝ Finally I ended up with numerical approximations.[1,2] Let me highfive you for this. I did the same, in C, and it took me quite some time, only to find out later that C actually comes with a beautiful function called lgamma which does it and returns the logarithm. I wasted a good part of my youth reinventing the wheel it seems ❝ BTW, nitpicking Diletti’s Table 1[6] (power 70%, T/R=1, CV=7.5%) and nQuery 7 give n=4, whilst my R code, StudySize 2.0.1, and FARTSSIE 1.6 come up with n=6. Now what? Go with a democratic vote of 3:2 for 6? Two solutions:
|
Helmut ★★★ Vienna, Austria, 2009-09-28 21:52 (5553 d 14:20 ago) @ ElMaestro Posting: # 4263 Views: 42,322 |
|
Hi ElMaestro! ❝ Let me highfive you for this. BTW, great subject line! 42? (or look what has to say). ❝ I wasted a good part of my youth reinventing the wheel Welcome to the club of wheel re-inventors. ❝ ❝ BTW, nitpicking Diletti's Table 1 […] ❝ Two solutions: ❝ 1. Ask dlabes Ha! Have you read his post? With that code you can reproduce the block for power=80% […] with some minor deviations for CV=5% and 7.5%. ❝ 2. Get your power from a program that bases its evaluation on brute force rather than on numerical integration. Do you have one handy? — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2009-09-28 22:18 (5553 d 13:54 ago) @ Helmut Posting: # 4264 Views: 42,562 |
|
Hi HS, ❝ ❝ 2. Get your power from a program that bases its evaluation on brute force rather than on numerical integration. ❝ ❝ Do you have one handy? I think I can make you an elmaestrolophystic piece of software for this; will take "some time"* to write it, I guess. EM. *: Remember Alan Turing? He proved that there is no way of knowing if an algorithm converges until it converges. If it converges. Unfortunately it works the same way when I fire up my C compiler. In spite of good intentions and fancy plans I may get personally stuck in an infinite loop. By the way, it is pretty much the same with women and telephones, but that's a completely different story. And another forum, I guess. |
yjlee168 ★★★ Kaohsiung, Taiwan, 2009-09-28 23:30 (5553 d 12:41 ago) @ Helmut Posting: # 4265 Views: 42,444 |
|
Dear Helmut, I just tried PASS and got the following (for power 70%, T/R=1, CV=7.5%). Power Analysis of 2x2 Cross-Over Design for Testing Equivalence Using Ratios N = 4. Sorry for my previous message (I set power= 0.8, not 0.7!). ❝ [...] ❝ Do you have one handy? — All the best, -- Yung-jin Lee bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee Kaohsiung, Taiwan https://www.pkpd168.com/bear Download link (updated) -> here |
Helmut ★★★ Vienna, Austria, 2009-09-29 01:18 (5553 d 10:54 ago) @ yjlee168 Posting: # 4266 Views: 42,702 |
|
Good morning, bears! Thanks for the PASS-run. Now the game is tied at 3:3 for sample sizes of 4 or 6 (target power 70%, T/R=1, CV=7.5%). power % n Underlined sample sizes are rounded up to the next even integer (calculated results in parentheses). From the manual I would say that PASS uses also AS 243. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2009-09-29 15:37 (5552 d 20:35 ago) @ Helmut Posting: # 4267 Views: 42,561 |
|
Dear guys, let me take part in your hairsplitting party . Hope you have already waited for me. Here are the results of the SAS Proc Power. Code can be found here, of course appropriately adapted. The POWER Procedure The power is given with 3 decimals only by default. No simple way to change. Seems the power is calculated in this case totally wrong! This is astonishing enough, because the same Proc Power code can reproduce most of the sample sizes given in Diletti et.al., at least for power 80%. Ok, Proc Power is called experimental by SAS, usually means the user shall do the testing and report bugs. Others call such software beta. My own rolled code (relying on Owen's Q-functions, undocumented but available in SAS) gives:
Coeff. of — Regards, Detlew |
Helmut ★★★ Vienna, Austria, 2009-09-29 16:52 (5552 d 19:20 ago) @ d_labes Posting: # 4269 Views: 42,899 |
|
Dear D Labes! ❝ Hope you have already waited for me. Yesss! ❝ Here are the results of the SAS Proc Power. ❝ Seems the power is calculated in this case totally wrong! Fascinating. ❝ My own rolled code (relying on Owens Q-functions, undocumented but available in SAS) gives: That’s the more interesting part (homebrew)! n % power n % power n=4 takes the lead by 4:3! Observations: #5 (based on Algorithm AS 243) and #6 give identical results. This is not surprising, because the VBA-routine within FARTSSIE was written by Russell V. Lenth, the author of AS 243. #2/3 agree; for n=6 power is comparable with #5/6. #4 calculates lower power than #1-3, but still would suggest n=4. The difference between #1-3 and #5/6 might be due to different algorithms for estimating the noncentral t-distribution (#1/3: ?, #2: AS 243?, #4: AS 184, #5/6: AS 243). #7 uses approximations (Ref. 2 in this post); however, yields conservative results. Maybe I should contact Dieter Hauschke to join the party… Edit: Many, many years later.
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2009-09-29 20:14 (5552 d 15:58 ago) @ Helmut Posting: # 4270 Views: 42,345 |
|
Dear HS, ❝ n=4 takes the lead by 4:3! Eighth observation: I made a little application that goes under the codename EFG (which is short for ElF#ckupGrande) and which implements brute force in conjunction with Mersenne Twister and some very ugly elmaestrolophystic code clutter. At N=4, CV=0.075 and T/R=1.0 I get power = 72.7191% with 2000000 iterations. At N=6 power is 98.97435%. EM. |
yjlee168 ★★★ Kaohsiung, Taiwan, 2009-09-30 01:55 (5552 d 10:17 ago) @ Helmut Posting: # 4272 Views: 42,377 |
|
Dear Helmut, Please read my message of this thread. Sorry for my mistake. I find that PASS can reach the results that are pretty close to D Labes's SAS run. ❝ That's the more interesting part (homebrew)! n % power n % power — All the best, -- Yung-jin Lee bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee Kaohsiung, Taiwan https://www.pkpd168.com/bear Download link (updated) -> here |
yjlee168 ★★★ Kaohsiung, Taiwan, 2009-09-30 01:49 (5552 d 10:23 ago) @ yjlee168 Posting: # 4271 Views: 42,210 |
|
Dear Helmut, I made some correction with PASS-run for the power when N = 6. Please help to update your collection data. Thanks. Power Analysis of 2x2 Cross-Over Design for Testing Equivalence — All the best, -- Yung-jin Lee bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee Kaohsiung, Taiwan https://www.pkpd168.com/bear Download link (updated) -> here |
KDA ☆ India, 2019-01-23 05:24 (2150 d 05:48 ago) @ Helmut Posting: # 19800 Views: 18,874 |
|
Dear all, I am a amateur in BA/BE area and trying to learn from this forum. I would appreciate if anyone kindly answer few of my queries for the r-code used to reproduce Fig.1c by Diletti et al. (1991). Q1) If it is 2x2 cross-over study, test and ref drug would be given only once. Can we get true intra-subject co-efficient of variance (CV)from 2x2 cross-over study? Q2) If it is possible to calculate the intra-subject co-efficient of variance (CV)from 2x2 cross-over study, is this value coming from ref drug only? Q3) What is the equation for calculating this intra-subject co-efficient of variance (CV)? Q4) In the equation "s = sqrt(2)*sigmaW", what 's' stands for? why '2' is square-rooted? Q5) I have used the following r-code for calculating point estimate and CI limit of AUC. A1 <- data$AUC[which(data$TRT=="R1" & data$PER==1)]
Can I calculate CV by the following equation? CV=SD/(exp(mD)) Based on my understanding, this my calculated CV is not the same as the 'CV' that represent intra-subject co-efficient of variance used to reproduce Fig.1c by Diletti et al. (1991) what code/formula should I use if I want to calculate intra-subject co-efficient of variance? Regards KDA |
Ohlbe ★★★ France, 2019-01-27 19:24 (2145 d 15:48 ago) @ KDA Posting: # 19814 Views: 18,471 |
|
Dear KDA, ❝ Q1) If it is 2x2 cross-over study, test and ref drug would be given only once. Can we get true intra-subject co-efficient of variance (CV)from 2x2 cross-over study? No. You need a replicate design study. I'll let others answer Q5. — Regards Ohlbe |
Brus ★ Spain, 2021-12-02 16:00 (1105 d 19:12 ago) @ Helmut Posting: # 22683 Views: 8,670 |
|
Dear Helmut, I am not an expert in R, and I am trying to adapt your example of power calculation with plot replesentation, but for the calculation of the sample size. My objetive is to plot a graphicala representation with ratio in X axis and sample size in Y axis and setting the CV and power as a fixed value. Can you or someone help me? Best regards, |
ElMaestro ★★★ Denmark, 2021-12-02 16:15 (1105 d 18:57 ago) @ Brus Posting: # 22684 Views: 8,801 |
|
Hello Brus, ❝ I am not an expert in R, and I am trying to adapt your example of power calculation with plot replesentation, but for the calculation of the sample size. My objetive is to plot a graphicala representation with ratio in X axis and sample size in Y axis and setting the CV and power as a fixed value. Here's something basic to get you started. I assume you already have the PowerTOST package on your system. Try.this=function(CV=0.2, tpow=0.8) Can be refined with designs and multipanel options and what not. PS: Correct, I am not entirely comfy with the tapply, sapply, apply family of functions.— Pass or fail! ElMaestro |
Helmut ★★★ Vienna, Austria, 2021-12-02 18:30 (1105 d 16:42 ago) @ ElMaestro Posting: # 22686 Views: 8,639 |
|
Hi ElMaestro, ❝ A suggestion. ❝ PS: Correct, I am not entirely comfy with the You are not alone. To complicate things, in PowerTOST many functions are not or only partly vectorized. See the script in this article for a remedy (click on to show it).— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2021-12-02 17:57 (1105 d 17:14 ago) @ Brus Posting: # 22685 Views: 8,675 |
|
Hi Brus, ❝ I am trying to adapt your example of power calculation with plot replesentation, but for the calculation of the sample size. My objetive is to plot a graphicala representation with ratio in X axis and sample size in Y axis and setting the CV and power as a fixed value. I suggest this goody first. May sound picky but we can’t get the exact desired power, only one which is at least that high.* Try this one:
The sample size is a staircase function and due to the symmetry in \(\small{\log_{e}}\)-scale we require the same sample size for any given \(\small{\theta_0}\) and \(1/\small{\theta_0}\) (see also there).
Diving deeper into the matter: Power is a hypersurface depending on \(\small{\theta_0}\), \(\small{n}\), and \(\small{CV}\). We can only show spatial projections at certain \(\small{CV}\text{-}\)values (the panels) with \(\small{\log_{e}\theta_0}\) (x-axis), \(\small{n}\) (y-axis), and power (z-axis). \(\small{n=12-292}\) and \(\small{\theta_0=0.80-1.25}\) to demonstrate that at the limits of the acceptance range power (which is there the Type I Error) is ≤0.05. The intersections with the horizontal plane (the target power) show the U-shaped dependency of \(\small{n}\) on \(\small{\theta_0}\) as in the plot above. The slices at certain \(\small{\theta_0}\)- and \(\small{n}\)-values give the plots like in the very old post above. Don’t try to imagine four-dimensional objects. Even a simple hypercube (tesseract) may fry you brain. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |