yjlee168 Senior Kaohsiung, Taiwan, 20081026 20:46 (edited by yjlee168 on 20081026 21:00) Posting: # 2584 Views: 22,926 

Dear Elmaestro (EM) & Helmut, I almost forget how to initiate a new thread in this Forum. have got used to click "Post reply" to send a replied message. In previous message, I said that "...bear v2.0.1 is not designed to analyze the imbalanced data, but the balanced data, obtained from BE studies..." because of this replied message by EM. In that replied message, EM has suggested an lme method should be used first. Currently, we have not implemented lme method yet with bear. Right now, we use lm to do anova only. We still work on lme. Helmut also had a comment about analyzing unbalanced (or imbalanced) sequence BE data. So we got the book and studied MVUE. Please let me know if I am wrong about this. In this case, can bear still be used to analyze the imbalanced data of a BE study only with lm? Thanks. — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
ElMaestro Hero Denmark, 20081027 20:36 @ yjlee168 Posting: # 2585 Views: 21,306 

» In this case, can bear still be used to analyze the imbalanced data of a BE study only with lm? Dear Dr Lee, yes, you can easily do it with lm or glm as far as a 2sequence, 2period, 2treatment study goes. Do it this way: MyFunkyFit=lm(LnParam~Subj+Seq+Pe+Treatm) (You can use glm is stead of lm here, no problem) The for the anova... Anova with type I SS goes something like this: A_typeI=anova(MyFunkyFit) Anova with type III goes SS something like this: A_typeIII=drop1(MyFunkyFit, test="F") It is correct that I earlier suggested the use of lme in stead. The result wil be exactly the same, it needs to be said. I proposed that because I saw a thread in another forum which indicated that lm might be unstable or incorrect for imbalanced data. This seems not to be the case in practice; I have really tried hard to find a dataset that shows this phenomenon butI can't. Thus lm or lme or glm, they will all work, the choice is yours. Good luck. EM. 
yjlee168 Senior Kaohsiung, Taiwan, 20081028 08:59 (edited by Jaime_R on 20081028 10:38) @ ElMaestro Posting: # 2587 Views: 21,342 

dear EM, Thank you for your message. No wonder that I cannot find anything about the method to analyze imbalanced sequence, BE data from USFDA guideline. Yes, we've already been able to calculated Type I SS and Type III SS with bear right now. Thus, bear can analyze imbalanced or balanced sequence, BE data using lm method. In your message of previous thread, you said it might be "unsafe" if we used lm to analyze imbalanced BE data. And in the following replied message, you said it might be "unstable or incorrect" (from another forum) if we do so. Could you please give us more details about why you said so? What have you seen if one uses lm to analyze an imbalanced sequence BE? It sounds like lme or SAS PROC MIXED is preferred when analyzing imbalanced sequence BE. Am I correct? — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
ElMaestro Hero Denmark, 20081028 09:49 (edited by ElMaestro on 20081028 13:04) @ yjlee168 Posting: # 2588 Views: 21,449 

Dear Dr Lee, I am sorry if my signals have been wobbly. I think at some point I read a thread on the mailing list for R http://tolstoy.newcastle.edu.au/R/ about the simple linear model, that it in R's implementation might be problematic or some other negative thing (will give the specific link if I find it again, promise) in case of imbalance. I could easily have misunderstood it, no doubt. Due to this I suggested lme, which has some potential advantages:
Finally, in some *AHEM COUGH*iers I have seen for 2,2,2BE studies, the protocol has explicitly stated that proc glm would be used if the dataset was balanced, and that proc mixed would be used in case of imbalance. I am not a statistician so I cannot qualify this in any way, I only relay it as an observation. Best regards EM. 
d_labes Hero Berlin, Germany, 20081028 17:14 @ ElMaestro Posting: # 2592 Views: 21,477 

Dear Yungyin, dear ElMaestro » [...] the protocol has explicitly stated that proc glm would be used if the dataset was balanced, and that proc mixed would be used in case of imbalance. May be the confusion comes from the meaning of imbalance. Usually imbalance in the context of 2x2 crossover means unequal number of subjects in the sequence groups. But sometimes incomplete data are meaned, i.e. dropouts with data for the first period. This is of course imbalance to number of data in periods. In the first case Proc GLM and Proc MIXED give identical results. Only in the case of missings Proc MIXED can recover some information from the subject with incomplete data and will then give different results to GLM. Let us take an example: Lets use Helmuts data here on the forum. This is a balanced study. The results (without subject effects, wich are dealt different in GLM or mixed effects analysis and wich are seldom from interest): Proc GLM ANOVA typeIII Ftests Proc MIXED Fixed effects Ftests (type III) As you see, identical results. Now let us deal with imbalance in sequence. Let's drop subject 24. The results: Proc GLM ANOVA Ftests Proc MIXED
Fixed effects Ftests As you see, again complete identical results. And now with incomplete data. Set subject 24, period 2 to missing. The results: Proc GLM ANOVA Ftests Proc MIXED Fixed effects Ftests Here we have differences in the results. Proc GLM gives the same confidence interval as with subject 24 dropped from the analysis, whereas Proc MIXED yields some result near the analysis of the complete data. Now its up to you to figure out what the Rophylistic analysis will give. Hopefully the same . Then this could be a validation example for BEAR. — Regards, Detlew 
yjlee168 Senior Kaohsiung, Taiwan, 20081029 07:36 @ d_labes Posting: # 2594 Views: 21,308 

dear DLabes, We will follow your suggestions and examples given here to validate bear and will report the test result here. One more question: do we have to include all cases wuth incomplete data (only R or T), or we can treat these cases as dropouts? Which one is more acceptable based on regulatory guidelines (if any)? Thank you so much. — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
d_labes Hero Berlin, Germany, 20081029 09:24 @ yjlee168 Posting: # 2595 Views: 21,259 

Dear Yungyin, » [...] One more question: do we have to include all cases wuth incomplete data (only R or T), or we can treat these cases as dropouts? Which one is more acceptable based on regulatory guidelines (if any)?[...] By the way: wuth was a brewery in Wiesbaden with good German beer . Nowadays an institute for learning advertising (buy things you do not need), no beer. But let us be unsmiling scientific: Generally spoken: Include all data you have. But in the context of a classical 2x2 design and the use of Proc GLM or equivalent software it does not make a difference with respect to the bioequivalence test (90% confidence interval) as we have seen from our example analysis (3rd analysis) discussed just. Therefore my attitude up to now was to exclude subjects with incomplete data from a 2x2 crossover (reporting the data, if any, but not including in the statistical analysis). But see Helmut's excerpt of SENN here. In the new EMEA Draft it is stated under 4.1.8 Evaluation/Subject accountability: "All treated subjects should be included in the statistical analysis, with the exception of subjects in a crossover trial who do not complete at least one period receiving each of the test and reference products (or who fail to complete the single period in a parallel group trial)." A little bit confusing (at least to me ) but according to Ohlbe it can be interpreted as not to include subjects with data for only one period (... receiving each of the test and reference products ...) from a 2x2 xover. Other regulatory recommendations I am not aware in the moment. Any other out there with a comment? — Regards, Detlew 
yjlee168 Senior Kaohsiung, Taiwan, 20081030 07:12 @ d_labes Posting: # 2610 Views: 21,193 

dear DLabes, Thank you for your message. » By the way: wuth was a brewery in Wiesbaden with good German beer . » Nowadays an institute for learning advertising (buy things you do not need), no beer. German beer? I've already tasted some during August when attending useR!2008 Annual Conference at Dortmund, Germany. Here was one that I had in Kron. Don't know it's exact name. probably homemade. But it tasted great. — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
ElMaestro Hero Denmark, 20081029 09:27 @ d_labes Posting: # 2596 Views: 21,377 

Dear Dlabes, excellent input. I learn a lot from these discussions. Allow me a little digression here.... I have never seen a *AHEM COUGH*ier for a 2,2,2BE study in which subjects with missing period data are included in the study. In fact, it is sort of standard to simply specify that the analysis will be based on data from subjects completing both periods (in some cases it is even written as 'the first 28 completers' etc etc). I think this stems from the fact that the EMEA has been challenged on missing data (missing periods can be considered such a thing, I guess). For this reason they produced some documents with guidance. That guidance implicitly deals mainly with progressiontypes of data (i.e. intended measurements of the degree of syphilis/parkinsonism/crohn/hemorrhoids/tuberculosis/survival/whatever at week 4, 8, 12, 16 after treatment initiation but data from week 12 goes missing, and how do you then do the KaplanMeier etc), but equivalence/noninf. studies are not excluded per se. Among realistic alternatives, EMEA says, is imputation. Who can imagine imputation for missing 2,2,2BE period data*? I think it is wise to simply rule those subjects out. In R this would correspond to "na.omit" if my memory serves me correct. Please, please more discussions of this type. EM. *: For example "Analyses that investigate missing data imbalance in all relevant factors and whether patients with and without missing values have different characteristics at baseline might also be conducted" and that sort of thinking [http://www.emea.europa.eu/pdfs/human/ewp/177699EN.pdf, under revision] 
d_labes Hero Berlin, Germany, 20081029 09:56 @ ElMaestro Posting: # 2597 Views: 21,208 

Dear ElMaestro, full ACK with your points. At least for the classical 2x2 crossover. See my two coins in the comment above. A further justification: In Equivalence trials (and BE studies are a sort of, really ) the primary population usually is the per protocol population. We already had it here in this thread. And missing data are mostly the consequence of protocol deviations/violations. » Please, please more discussions of this type. » EM. I also wish this. Really! I know, time is spare with us. But YOU out there, do not follow "Talk is silver, silence is golden". Take silver. Gold is spare in the nowadays time of financial disasters over the world. — Regards, Detlew 
Helmut Hero Vienna, Austria, 20081029 13:03 @ d_labes Posting: # 2602 Views: 21,134 

Dear all! I haven’t performed analyses of incomplete data of a 2×2×2 study until now – and haven’t seen one in my work as a consultant performed by anybody else. Another quote* (referring to their example 3.2 in 49 subjects): Before we can continue to fit a linear model to the (logtransformed) data, we must decide what to do with the data from those subjects who did not provide a value in both periods. Such a comparison is not possible for those subjects with only a single value. If however, there are two such subjects and has a value only on T and the other has a value only on R, then a betweensubject comparison of T and R is possible by taking the difference of these two single values. However, the precision of such a comparison will be low because the betweensubject variation […] is much higher than the withinsubject variability. Because we have assumed the subject effects are random variables, these betweensubject comparisons can be recovered in the analysis if we fit what is known as a mixed model. […] However, the recovery of betweensubject information on the comparison of T and R is unlikely to make much difference to the results, and so nothing of significance will be lost by ignoring the data on those subjects that provided only a single value. To justify this assertion, we will also report the results of fitting the mixed model to the complete data set […] Personally I think it’s a little bit ambiguous to perform an analysis of the complete data set (simple model) as primary and ‘justify’ it by running a second one on the full (mixed) model. For our example the width of the confidence interval is0.198015 (simple model) and 0.189859 (full model). As DLabes pointed out in this post the mixed model is scientifically valid, but I would be wary to present it as the only one – since without suitable software it’s not possible to reproduce the results. If done the other way ’round (from Patterson/Jones), I would expect a lot of discussions (imagine if model #1 shows BE, and #2 does not)…
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Helmut Hero Vienna, Austria, 20081029 12:12 @ d_labes Posting: # 2601 Views: 21,428 

Dear all, let’s continue the example in WinNonlin 5.2.1 (which uses also linear mixed effects modeling). Identical results to SAS are obtained both for the complete data set, and for the data set with #24 missing in both periods. If data of #24 are missing in period 2 only, I obtained: Denominator df option : residual Partial Tests of Model Effects Denominator df option : satterthwaite Partial Tests of Model Effects The second table is in agreement with SAS’ Proc MIXED. Cave: although Satterthwaite degrees of freedom are default in WinNonlin, it’s possible to change this behaviour in the program’s preferences. This doesn’t make a difference if complete subjects are missing, but should be avoided in this case. @DLabes: I gave the PE/CI to 6 decimal places in order to see a difference (if any); Satterthwaite dfs were 21.760741 (Sequence), 21.049560 (Formul), and 21.059500 (Period); can you give us your results also?— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20081029 16:38 @ Helmut Posting: # 2605 Views: 21,123 

Dear HS, dear all, Here the results of SAS9.2 Proc MIXED Denominator df option : residual Type 3 Tests of fixed effects Denominator df option : satterthwaite Type 3 Tests of fixed effects Seems there is a very distinct definition, what residual df option is. The SAS standard method is Containment (whatever this is) and this was what I reported earlier in this thread. I think I must delve into my SAS/Stat users guide and struggle again with the dragon "The power to know". — Regards, Detlew 
ElMaestro Hero Denmark, 20081030 08:57 @ d_labes Posting: # 2611 Views: 21,180 

Hey hey hey, Having thought a little about this discussion, I almost feel for initiating a dry study investigating some of the issues in this thread. It would be interesting to find out which approach performs better (if an objective measure of that can be defined) when there are missing period data for one or more subjects. In particular it would be great FUN (and of potential interest to the PK oracles at EMEA) to play around with imputation methods and see if it is possible to find an imputation approach that outperforms linear mixed effect models. Comments? EM. 
d_labes Hero Berlin, Germany, 20081030 15:54 @ d_labes Posting: # 2612 Views: 21,146 

Dear all, after RTFM and reading tons of Inet places dealing with SAS Proc MIXED I am more confused as before. For German speakers: "Da steh ich nun, ich armer Tor und bin genauso klug als wie zuvor!" (Johann W. von Goethe  Faust). The only thing I could figure out was that DDFM=residual creates degrees of freedom which neglect totally the covariance structure of the random effects in the model. Not a good idea in case of data from xover trials, I think. But I couldn't figure out yet what this option is good for. Interesting enough this DDFM was the default in SAS before Version 8. Thus users not aware of this had gotten inappropriate Ftest! For our data the consequence is: Simply forget it. The degrees of freedom are not appropriate anyhow. Full stop. @for HS: Could it be that the DDF for the sequence effect was 22 instead of 21? Then WinNonlin method could be compared to SAS DDFM=contain or DDFM=betweenwithin. — Regards, Detlew 
Helmut Hero Vienna, Austria, 20081030 16:30 @ d_labes Posting: # 2613 Views: 21,138 

Dear DLabes! » after RTFM… » Interesting enough this DDFM was the default in SAS before Version 8. Thus users not aware of this had gotten inappropriate Ftest! Wow! » @for HS: Could it be that the DDF for the sequence effect was 22 instead of 21? Then WinNonlin method could be compared to SAS DDFM=contain or DDFM=betweenwithin. Checked: it’s given with 21 indeed… For other WinNonlinUsers I uploaded a workbook containing the full data set, the imbalanced one (#24 both periods missing, and the incomplete one (#24 2^{nd} period missing) in separate sheets. Feel free to fiddle around with it… — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
yjlee168 Senior Kaohsiung, Taiwan, 20081107 05:12 @ d_labes Posting: # 2640 Views: 21,039 

Dear DLabes, In order to validate bear with your SAS outputs, could you please provide detailed anova output obtained from your SAS runs? Such as that we got anova output obtained from bear with Helmut's data (of balanced seq.) as follow: Dependent Variable: Cmax Thank you. — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
d_labes Hero Berlin, Germany, 20081111 08:35 @ yjlee168 Posting: # 2648 Views: 20,931 

Dear Yungjin, sorry I'm a bit late because I missed your post. Here are the complete ANOVA tables (the Ftests were already contained in the post above) from Proc GLM: Complete dataset: Dependent Variable: logCmax Inbalanced dataset (Subject 24 excluded): Dependent Variable: logCmax Type I sum of squares are not contained by default. If you need them, let me know. In case of balanced data they are same as type III, isn't it? — Regards, Detlew 
yjlee168 Senior Kaohsiung, Taiwan, 20081111 08:54 (edited by yjlee168 on 20081111 11:39) @ d_labes Posting: # 2649 Views: 21,048 

Dear DLabes, Thank you for your replied message. We've done the part of balanced and unbalanced data with SAS PROC GLM and PROC MIXED when we're waiting for your replied message. Fortunately, bear can produce the exactly same results as SAS. I will post both results at this Forum later. However, we need the part of incomplete data (obtained from both PROC GLM and PROC MIXED). Actually, we did that part a little with beat with lme package, too, and found there were some differences between SAS (with PROC MIXED and PROC MIXED) and bear. Thanks. — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
d_labes Hero Berlin, Germany, 20081111 13:13 @ yjlee168 Posting: # 2651 Views: 20,960 

Dear Yungyin, here comes the incomplete data
The GLM Procedure As you see, there is no such thing like the ANOVA table in Proc MIXED. — Regards, Detlew 
yjlee168 Senior Kaohsiung, Taiwan, 20081111 13:35 @ d_labes Posting: # 2653 Views: 20,988 

Dear DLabes, Thanks for your quick response. Also, Helmut's good tip though I've used that way for a while. » (...) » The GLM Procedure » Tests of Hypotheses for Mixed Model Analysis of Variance » (...) » As you see, there is no such thing like the ANOVA table in Proc MIXED. still has ANOVA Table with PROC MIXED? BTW, in your previous post, you showed a 90%CI as well as a point estimate for each run with SAS. What 90%CI method did you use? We got the results from bear and there were some different from yours after the second decimal digits. — All the best, Yungjin Lee bear v2.8.33: created by Hsinya Lee & Yungjin Lee Kaohsiung, Taiwan http://pkpd.kmu.edu.tw/bear Download link (updated) > here 
d_labes Hero Berlin, Germany, 20081111 14:02 @ yjlee168 Posting: # 2655 Views: 21,031 

Dear Yungying, » still has ANOVA Table with PROC MIXED? The heading you cited is from Proc GLM . Under Proc MIXED there are no SSq nor MSq. This is because the REML estimation method does not rely on SSq's and their decomposition. » What 90%CI method did you use? The shown point estimates and 90% CI's are from the difference between Least square means. Here is my code for Proc GLM / mixed (to play around with, it seems you have access to "The power to know"): ODS output LSMeandiffCL=ratios; *< this saves the 90% CIs; — Regards, Detlew 
Helmut Hero Vienna, Austria, 20081111 12:24 @ d_labes Posting: # 2650 Views: 20,977 

Dear DLabes, » sorry I'm a bit late because I missed your post. Just some hints: You may bookmark the Latest Posts instead of the forum for a quick overview what's going on… Or you may use the RSSFeed… Or you may opt in for email notification about new posts in your personal profile… — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20081111 13:24 @ Helmut Posting: # 2652 Views: 20,923 

Dear HS, thanx for the hints. But all that has not helped me in that case. I think I need some more time or money, or both . — Regards, Detlew 
Helmut Hero Vienna, Austria, 20081111 13:39 @ d_labes Posting: # 2654 Views: 20,966 

Dear DLabes, » I think I need some more time or money, or both . I was once told by a very famous biostatistician that there are only three things which actually count:
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20081111 14:09 @ Helmut Posting: # 2656 Views: 20,952 

Dear HS, in the old days:
— Regards, Detlew 