ElMaestro ★★★ Denmark, 2010-02-06 18:10 (5550 d 11:25 ago) Posting: # 4724 Views: 11,413 |
|
Bong sure, referring to the other thread where the new BE guideline from EMEA is discussed and where HS proposed a simulation. As usual I am several steps behind as I am not sure I understand all details of the discussion. Nevertheless, the word anticonservative was right in my face. Can that claim be generalised in any way? Following the philosophy of HS' single simulations experiment I think we could simulate different 3-per, 3-trt datasets under various conditions, analyse them by a mixed model and record sigma = the residual error (this is the s used in calculation of the conficence interval). Next, prune the dataset so it's possible to analyse it as a 2,2,2-BE dataset with a normal linear model, and then obtain the residual and compare. The philosophy here is that if the pruning results in a sigma that is on average larger than the sigma obtained through the full dataset, then the approach is conservative; otherwise it is not. Here's some elmaestrolophystic code (probably bugged and wrong): load(nlme) On my machine the code above is a win for anticonservativism. But that is dependent on the settings of means and CV in the treatment groups. HS: I got the impression that you simulated your dataset on the ordinary scale, then log-transformed the data. At least, if I log-transform your data then I can reproduce your result (with the code above). Here I am simulating normal dist data on the log scale. Seems more right to me. Bugs galore.... EM. Update a few mins after posting: Seems escape characters do not go well on this forum?! The last two lines starting with "cat" was intended to finish with a letter n preceded by a down-right slash but I cannot paste that for some reason. Edit: Yeah, that's a bug (see here). Corrected in the v.1.7.7beta scripts - hopefully available here soon. [HS] |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-03 22:12 (4915 d 07:23 ago) @ ElMaestro Posting: # 7598 Views: 9,135 |
|
Hi to all simulants! I modified your code according to Martin's suggestion. I tested three scenarios (10 000 sims each):
require(nlme) Is this correct – or did I bungle the code? ![]()
T R1 R2 Cons. Anticons. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2011-11-03 22:43 (4915 d 06:51 ago) @ Helmut Posting: # 7599 Views: 8,967 |
|
Hi HS, 2 comments. 1: ❝ ![]() 2: I think we need to take degrees of freedom into consideration too. After all, the critical value of the t-dist (and thus the width of the CI which is the ultimate indicator) depends heavily on it. I think the df's differ between the two scenarios. I hope this does not lead into a discussion about how df's are calculated for a mixed model (R will not do Satterthwaite etc.), a discussion where I have absolutely nothing to contribute. I am sorry to question my own sanity. — Pass or fail! ElMaestro |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-04 02:56 (4915 d 02:38 ago) @ ElMaestro Posting: # 7601 Views: 9,213 |
|
Salute! ❝ I am not sure why ElMaestro used a mixed model here; doesn't this specification just give the same as a linear model with seq1, per1, trt1, and sub1 as fixed factors In terms of the residual variance – yes. Good ol’ days of subjects as random. Met last week another statistical pro who was upset upon the ‘all fixed’ story. ![]() ❝ I think we need to take degrees of freedom into consideration too. […] I think the df's differ between the two scenarios. Right. ❝ I hope this does not lead into a discussion about how df's are calculated for a mixed model (R will not do Satterthwaite etc.), No, no. Everything is balanced here. 20 dfs in the Williams’ design and 10 after R2 is thrown out. Another try: set.seed(12041958) # keep this line to compare results only Sorry for the lengthy code; couldn’t do better. T R1 R2 Cons. (CVintra) Liberal (CVintra) Hhm. Kannitverstan. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2011-11-04 09:41 (4914 d 19:53 ago) @ Helmut Posting: # 7602 Views: 8,932 |
|
Good morning HS, I might add: At initialisation:
RM.be <- 0 At evaluation: if ((FMlo>=80) && (FMhi<=125)) FM.be <- FM.be + 1 ...and then compare FM.be and RM.be at exit. And now it's time for the second cup of coffee. Have a great day. EM. |
d_labes ★★★ Berlin, Germany, 2011-11-04 10:40 (4914 d 18:54 ago) @ Helmut Posting: # 7603 Views: 9,005 |
|
Dear Helmut! One fundamental question: If you simulate with rlnorm you get log-normal distributed values. Like those for AUC and Cmax in BE studies were also a log-normal distribution is assumed. The logs of the values simulated with rlnorm will be normally distributed. So long so good. Nothing new told here. But now how to analyse them ![]() Your code f.i. the line ❝ FullModel <- lm(log(y1) ~ 0 + seq1 + sub1 %in% seq1 + per1 + trt1) — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-04 17:05 (4914 d 12:30 ago) @ d_labes Posting: # 7607 Views: 9,250 |
|
Dear simulants! ❝ or what I would do ❝ Oh THX! Affects also the T-R. Latest version (flexible sample sizes, corrected T/R1 and T/R2): set.seed(12041958) # keep this line to compare results only Results: T R1 R2 Full BE (CVintra) Red. BE (CVintra) I have no idea how to simulate CVintra; CVtotal is not what I really want (to see if reference’s different variances have an influence on the result). — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
ElMaestro ★★★ Denmark, 2011-11-04 17:22 (4914 d 12:13 ago) (edited on 2011-11-04 22:36) @ Helmut Posting: # 7610 Views: 9,014 |
|
Hi Hs, ❝ I have no idea how to simulate CVintra; CVtotal is not what I really want (to see if reference’s different variances have an influence on the result). Neither had I when I started this thread, but now I have and I don't think it is difficult. Our model (simplest case but not satisfying d_labes' concern): y~Seq+Subj+Treat+Per+error There is one error in play unless we do a mixed model. So we sample the error (intrasubject) from a normal distribution and add it to the fixed effects. We can set the fixed effects to zero for Per and Seq, and tweak the Treat's as per our wishes for T/R ratios. Edit: The following code possibly shows what I suggest (and I am as always probably wrong):
##..and analyse from here I have not debugged so it is prolly full of errors. — Pass or fail! ElMaestro |
d_labes ★★★ Berlin, Germany, 2011-11-07 12:16 (4911 d 17:19 ago) @ ElMaestro Posting: # 7631 Views: 9,100 |
|
Dear Ol' Pirate, dear Helmut, ❝ ❝ I have no idea how to simulate CVintra; CVtotal is not what I really want (to see if reference’s different variances have an influence on the result). ❝ ❝ Neither had I when I started this thread, but now I have and I don't think it is difficult. Our model (simplest case but not satisfying d_labes' concern): ❝ ❝ y~Seq+Subj+Treat+Per+error ❝ ❝ There is one error in play unless we do a mixed model. So we sample the error (intrasubject) from a normal distribution and add it to the fixed effects. We can set the fixed effects to zero for Per and Seq, and tweak the Treat's as per our wishes for T/R ratios. Totally right if you think in terms of the EMA mantra 'all effects fixed'. But then you have to specify subjects effects in some reasonable way. What you do is to assume subjects effect of zero or Dolly-clones of a subject ![]() Don't know how this affects your goal. Don't also know what a reasonable way is to specify subjects fixed effects. At least you have many, many scenarios beside those you like to consider. The way out would be to consider subjects effects as random with a CV associated. Of course this is a mixed model. No one else than EMA has doubt about it. Refer f.i. to Chow, Liu "Design and Analysis of Bioavailability ..." page 42 for the model of the classical 2x2 crossover. To generalize this to the 3x3 study one could simulate the logs of a PK metric via a multivariate normal distribution with variance-covariance matrix
( varT+varS varS varS ) where varT, varR1 and varR2 are the intra-subject variances and varS the variability of the subjects effect (between-subjects variability). This model of course neglects any subject-by-treatment interaction. Together with a proper specification of the mean vector (µT, µR1, µR2) you get from the multivariate normal distribution (R package mvtnorm) vectors of simulated logs for T, R1 and R2. In this notation one neglects also period effects. If you like to deal with them you have to write down the above variance-covariance matrix for the period values for each sequence analogous to Chow, Liu for the 2x2 crossover. Another possibility would be the simulation with a zero mean vector and add the necessary fixed effects (treatment, period) afterwards. Hope this gibberish sounds reasonable to you and I have not made a big mistake. Code follows if I have tested it ![]() — Regards, Detlew |
ElMaestro ★★★ Denmark, 2011-11-08 12:02 (4910 d 17:33 ago) (edited on 2011-11-08 17:16) @ d_labes Posting: # 7634 Views: 8,906 |
|
All hands on deck! I hope this illustrates the issue of no treament differences when fixed effects between subjects are zerod or some scatter in fixed effects between subjects exists: Per=c(1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3, 1,2,3) Have a good day. As always it is prolly wrong and bugged. ![]() EM — Pass or fail! ElMaestro |
d_labes ★★★ Berlin, Germany, 2011-11-25 14:54 (4893 d 14:41 ago) @ Helmut Posting: # 7723 Views: 8,635 |
|
Dear Helmut! Quite a time ago I had promised to come out with a simulation including subject variability and intra-subject correlation. Due to spare time I could not until now consider this. Before posting my results I would like to compare it to your code. Inspecting it I have some doubts: ❝ ❝ ❝ ❝ ❝ ❝ Must the EMA guidance interpreted in this way, i.e. analysing the dataset with 'pseudo' periods and sequences? Or should it interpreted the way which is employed in the Q&A for the replicate design (intra-subject variability of the reference). That means only massacre out the data concerning R2 but retaining the original periods and sequences. ![]() ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ At least the code for FMMSE has stolen my sleep ![]() lm.summary() "... sigma - the square root of the estimated variance of the random error" and IMHO then the MSE is sigma^2 without the division by the degrees of freedom!Check this out via anova(FullModel)["Residuals","Mean Sq"]==summary(FullModel)$sigma^2 .Or did I miss sumfink here? — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-25 15:23 (4893 d 14:12 ago) @ d_labes Posting: # 7724 Views: 8,600 |
|
Dear Detlew! ❝ Inspecting it I have some doubts: ❝ ❝ […] some nonsense code ❝ ❝ Must the EMA guidance interpreted in this way, i.e. analysing the dataset with 'pseudo' periods and sequences? Or should it interpreted the way which is employed in the Q&A for the replicate design (intra-subject variability of the reference). That means only massacre out the data concerning R2 but retaining the original periods and sequences. Exactly! Should forget copypasting old code when new ‘knowledge’ became available in the meantime. ![]() ❝ ❝ ❝ At least the code for FMMSE has stolen my sleep Oh s**t! ❝ Check this out via ❝ Or did I miss sumfink here? No, no. Sorry! — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2011-11-04 16:46 (4914 d 12:48 ago) @ ElMaestro Posting: # 7606 Views: 8,914 |
|
Dear EM, ❝ nAntiConservative=0 ... for (iter in 1:100) { ## T is 1, R1 is 2 and R2 is 3 lny1 = rnorm(n=36, mean=95, sd=0.15*95) for (i in 1:36) if (trt1[i]==2) lny1[i]=rnorm(n=1, mean = 90, sd = 0.10*90) for (i in 1:36) if (trt1[i]==3) lny1[i]=rnorm(n=1, mean = 110, sd = 0.15*110) ... Really? I would suggest: If you consider 95, 90 and 110 as the arithmetic means on the untransformed scale use log(x)-sd2/2 as the mean of the logs. If your numbers 0.15, 0.10 and 0.15 are the CV's user sd = sqrt(log(CV*CV+1)) as sd of the logs. The manner you have written the simulation for the three treatments suggest you consider the log(metric) for the 3 treatments as independent. Is this really the case for a cross-over where the 3 treatment were applied to each subject ![]() — Regards, Detlew |
ElMaestro ★★★ Denmark, 2011-11-04 17:06 (4914 d 12:28 ago) @ d_labes Posting: # 7608 Views: 8,970 |
|
Hi d_labes, ❝ Really? ❝ I would suggest: ❝ If you consider 95, 90 and 110 as the arithmetic means on the untransformed scale use log(x)-sd2/2 as the mean of the logs. ❝ If your numbers 0.15, 0.10 and 0.15 are the CV's user sd = sqrt(log(CV*CV+1)) as sd of the logs. I think Martin raised this before. I don't understand this aspect. Feel free to educate me, you great scientist! ❝ The manner you have written the simulation for the three treatments suggest you consider the log(metric) for the 3 treatments as independent. ❝ Is this really the case for a cross-over where the 3 treatment were applied to each subject Probably a good point. But let's take the simple 2,2,2-BE scenario as example. Treatmeant T and R is probably correlated within patients; but we still only work with a covariance matrix with a single sigma squared on the diagonal and zeros elsewhere. I don't know how to take any such correlation into effect, but if you wish to do it a way forward might be a mixed model. As you can see above the original code was something I wrote 20 months ago, and I am today questioning my sanity back then. Actually I am still bonkers, but I guess that's another matter ![]() EM. |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-04 17:09 (4914 d 12:26 ago) @ d_labes Posting: # 7609 Views: 8,985 |
|
Dear Detlew! ❝ The manner you have written the simulation for the three treatments suggest you consider the log(metric) for the 3 treatments as independent. ❝ Is this really the case for a cross-over where the 3 treatment were applied to each subject Exactly. Saw your post after submitting mine. See its end… ![]() — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2011-11-08 12:31 (4910 d 17:03 ago) @ ElMaestro Posting: # 7635 Views: 9,088 |
|
Moin moin Simul-Ants! Sitting here, being bored ![]() ![]() I have found this online resource and this one on the topic. I think this has a direct influence on the scenarios to simulate here, namely we have to simulate ratios on the acceptance borders (H0: inequivalence) to get an estimate of the type I error alpha. Otherwise we simulate some sort of power at the set ratios (near 1) I think. And higher power may be an indication of a liberal test but must not. It can also be that the alpha is <5% (conservative) but the test has more power. BTW: post 499 ![]() — Regards, Detlew |
martin ★★ Austria, 2011-11-08 21:50 (4910 d 07:45 ago) @ d_labes Posting: # 7640 Views: 8,768 |
|
Dear d_labes! You may find the following paper of interest where the terms "liberal" and "conservative" are used for evaluation of the simulation results. Jaki T, Wolfsegger MJ, Lawo J-P (2010). Establishing bioequivalence in complete and incomplete data designs using AUCs. Journal of Biopharmaceutical Statistics, 20(4):803–820. DOI: 10.1080/10543401003618835. A pre-print of this article can be found here. hope this helps Martin |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-08 23:56 (4910 d 05:39 ago) @ martin Posting: # 7641 Views: 8,819 |
|
Dear Martin! THX for reminding me on this paper – I remember your presentation at the Inst. for Biostat. also quite well. ![]() The total sample size in the complete data design is smaller than for the serial sampling design, with the advantage disappearing with increasing intra-subject correlation with little effect of the number of measured time points to estimate the AUC. (my emphasis) I looked into my database and evaluated the within-subject correlation in 130+ studies and close to 3,000 subjects. I considered only studies showing BE (I can hear you shout: “Selection bias!!”). Not surprisingly correlations were quite high. Forgot to bring the data to Berlin. I will post some results ASAP. ![]() — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
martin ★★ Austria, 2011-11-09 10:01 (4909 d 19:34 ago) @ Helmut Posting: # 7642 Views: 8,969 |
|
Dear HS! This is extremely interesting. I would by happy if you can you provide some summary regarding the correlation between time points. (e.g. was the correlation constant over time, ...). best regards Martin PS.: As far as I know, Thomas will give a presentation on flexible batch designs at the Inst. for Biostat in the near future. I hope that you will find some time to participate as your input is highly appreciated!! I suppose that there will be also a debriefing at the old AKH. ![]() |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2011-11-25 18:16 (4893 d 11:18 ago) @ martin Posting: # 7725 Views: 8,879 |
|
Dear Martin! ❝ This is extremely interesting. I would by happy if you can you provide some summary regarding the correlation between time points. (e.g. was the correlation constant over time, …). As I told you last Wednesday at the ‘debriefing’ I would need a month of spare time to retrieve them (~100,000 data pairs in various formats). I can only serve with an overview. 133 cross-over studies, 2843 subjects, AUCt or AUCτ: ![]() min 2.5% med 97.5% max R² of my sample has an interesting distribution… Of course, correlation depends on the sample size* – but you get an impression. Studies with low correlations contained one ‘outlier’ each but were still large enough to show BE (well, that was my selection bias). Sponsors ![]()
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |