Eva ☆ Germany, 20080506 12:04 Posting: # 1817 Views: 20,137 

Dear all, I have recently analysed a simple crossover bioequivalence study in SAS and detected a significant sequence effect (p=0.0425). To verify my results, I have run the same analysis using the WinNonlin Bioequivalence Wizard (WNL v5.1.1). The results of the analysis were completely identical except for the sequence effect which was not significant (p=0.0634). The Pharsight support claims that this is due to a slightly different model used by the Bioequivalence Wizard and recommended me to read the online help on linear mixed effects modelling. I can not follow this argumentation, because in my opinion my model specification is the same. Besides I get identical results for all other effects: both my SAS program and the BE Wizard produce the same SS, MS and DF for sequence effect, period effect, treatment effect and subject within sequence, the MSE is identical in both programs and (consequently) also the confidence limits. The F statistics and the pvalues are the same for period effect, treatment effect and subject within sequence. I have repeated the comparison with other data with the same result, all values identical, but the F statistic of the sequence effect always lower in WNL compared to SAS. Has any of you ever come across this discrepancy before? I wonder if somebody could help me to find out if there is a mistake in the Bioequivalence Wizard calculation or if I am simply too stupid to model my data correctly... Thanks in advance & kind regards, Eva 
Ohlbe ★★★ France, 20080506 13:56 @ Eva Posting: # 1821 Views: 16,095 

Dear Eva, I am not a SAS or WinNonlin specialist, but I would have a basic question. Where do you read the SAS F and p values from ? Usually the printouts I see from SAS present the results in two parts. You first have the full ANOVA with sequence, subject(seq), period and treatment (with F and p values), then a specific test for sequence using the MS for subject(seq) as an error term, (again with a F and p value). Only the results of this specific test are supposed to be considered. Could this be the origin of the difference you found ? Regards Ohlbe 
Helmut ★★★ Vienna, Austria, 20080506 14:13 @ Eva Posting: # 1822 Views: 16,060 

Dear Eva! One example in SAS… SUM OF … and WinNonlin 5.2 Partial Sum of Squares Obviously sequence effects are identical (p 0.187). Maybe you had some dropouts in your study, which resulted in imbalanced sequences. In such a case SAS’ SS Type III and WinNonlin’s Partial Sum of Squares may differ (although they ‘coincide in most situations’). SAS SS III is a strange pot of tea anyhow… It would be nice, if you would come up with a simple dataset (12 subjects?), where you find differences. It should be possible to track it down! — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Eva ☆ Germany, 20080506 15:42 @ Helmut Posting: # 1824 Views: 15,906 

Dear Ohlbe and Helmut, thanks for your fast replies! Here is a sample data set of 8 subjects, maybe this can help to illustrate my problem...
Subject Period Sequence Treatment AUC My SAS results (PROC GLM) for these data are
EFFECT DF TYPE III SS MEAN SQUARE FVALUE PVALUE and this is what WinNonlin calculated:
EFFECT DF TYPE III SS MEAN SQUARE FVALUE PVALUE Dear Ohlbe, I think I have picked the right F and pvalues from my output, I am using PROC GLM (and I just checked, the output is identical with PROC ANOVA) and picked the Sequence, Subject(Sequence), Treatment and Period results from the bottom part and the Error results from the top part. Dear Helmut, could this be a problem with my WNL version? (I noticed that you used v5.2, whereas I have v5.1.1) As you can see above, the sums of squares are not the problem, it is only the F and pvalues that differ. I understood that the Fvalue is the ratio of the effect mean square and the mean square error. This is correct for the SAS results (sequence: 0.6105/0.2481=2.46), but not for WinNonLin... Any ideas? Thanks again, for your help and for sharing your precious time with my "little" problem... Kind regards, Eva Edit: Treatment in the table changed to Test to avoid confusion with the heading. [Helmut] 
Ohlbe ★★★ France, 20080506 17:25 @ Eva Posting: # 1825 Views: 15,914 

Dear Eva, F is the ratio of the mean square of your factor, divided by the mean square of what you consider as the error factor. SAS divided the MS of the sequence effect (0.6105) by the MS of the residual error (0.2481). WinNonlin divided the sequence MS by the MS of subject(sequence) (0.2623). This is what SAS does in the second step I mentioned previously. Helmut, I will let you explain why subject(sequence) should be used as an error term Regards Ohlbe 
Helmut ★★★ Vienna, Austria, 20080506 19:33 @ Ohlbe Posting: # 1826 Views: 16,154 

Dear Eva & Ohlbe! » Helmut, I will let you explain why subject(sequence) should be used as an error term OK, sequences are a between subject effect and therefore need to be compared to the between subject residual error (or F=0.6105/0.2623=2.327). In other words we compare (the group of) subjects in sequence TR to (the group of) subjects in sequence RT. If you compare the sequence against the within subject residual error (0.2481) you certainly will get a different – and wrong! – answer. Using the within subject residual actually overestimates the effect of sequence leading to a greater chance of concluding a sequence effect. Since in my first post I showed identical results for SAS and WinNonlin, I would expect a coding issue in the SAS program you are using. Results in WinNonlinversions don't differ. Unfortunately it’s not possible to have different versions running on the same machine – I rerun ‘old’ evaluations everytime I update to a newer version – and did not get differences since v4.0.1 (3.3 and earlier versions used ANOVA instead of linear mixed effects modeling). Just to be more comfortable about the results – a little roundup of your dataset: WinNonlin5.2 (2007)  Chow & Liu (2001) – coded in Excel  EquivTest/PK (2006)  Ref [3] (2001) from this post; coded in R2.7.0 (2008) NCSS (2001) (CrossOver Analysis)  Kinetica 4.4.1 (2007)  Besides your SAS code only Kinetica tests against the withinsubject residual error; this a well known bug since v1.1 (‼) – and I would expect Thermo_{Scientific} never to correct it… — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Helmut ★★★ Vienna, Austria, 20080507 16:18 @ Helmut Posting: # 1830 Views: 15,939 

» … Unfortunatelly it’s not possible to have different versions running on the same machine… OK, I remembered WinNonlin4.1 to be still installed on an old machine; again same results for the sequence effect: WinNonlin4.1 (2003)
Partial Tests of Model Effects — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes ★★★ Berlin, Germany, 20080508 09:16 @ Helmut Posting: # 1834 Views: 16,007 

Dear Eva & Ohlbe & Helmut » Since in my first post I showed identical results for SAS and WinNonlin, I would expect a coding issue in the SAS program you are using. That is totally right and wrong. The SAS Procedure GLM has the "standard" of treating all effects as fixed effects. Thus if you use the code: Proc GLM data=test; you get Eva's result. This result is correct if all effects are considered fixed. That means regarding the subject effect we are making inference about the subjects which are actually under study. But the goal of our BE study is to broaden the scope and make inference about some "population" of subjects (healthy volunteers f.i. from which we have randomly sampled our subjects actually under study). This leads to subject as a random factor (effect) in our model. The code in Proc GLM for that is: Proc GLM data=test; Proc GLM is resistant to our wishes and first displays the fixed effect analysis again (this is because according to SAS GLM handles random effects in a "post hoc manner", in direct words this feature was invented later and tinkered into GLM "mit heißer Nadel"). But then the sun is shining bright (emphasis by me): The GLM Procedure Hope this helps a little bit in "the power to know". SAS is an outstanding beast . Side note: The annoying fixed effects analysis displayed again is not needed and can be suppressed using the "famous" ODS (whatever this is). Final code: ODS exclude ExpectedMeanSquares ModelANOVA; — Regards, Detlew 
Helmut ★★★ Vienna, Austria, 20080508 15:18 @ d_labes Posting: # 1837 Views: 15,957 

Dear DLabes! » SAS is an outstanding beast… » […] using the "famous" ODS (whatever this is). Though I don’t have any idea about SAS, Google is my friend – coming up with ODS (Output Delivery System)… — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes ★★★ Berlin, Germany, 20080509 07:35 @ Helmut Posting: # 1840 Views: 15,709 

Dear Helmut, » Google is my friend – coming up with ODS = Output Delivery System… but this is only for THE INITIATED dragon fighter struggling with the beast . — Regards, Detlew 
Nirali ★ India, 20080509 09:00 (edited by Nirali on 20080510 02:30) @ d_labes Posting: # 1842 Views: 15,828 

Dear Friend, Yes, WinNonlin & SAS are giving same outputs for all effects of ANOVA all the time, just need to modify the SAS programm according to the requirement. Below programm will definitely help you . PROC GLM DATA=TEST; CLASS SEQUENCE SUBJECT PERIOD TREATMENT; MODEL AUC=SEQUENCE SUBJECT(SEQUENCE) TREATMENT PERIOD /SS3; TEST H=SEQUENCE E=SUBJECT(SEQUENCE); RUN; QUIT; » SAS is an outstanding beast . Now, no need to struggle with beast just read below book to get better knowledge of SAS PROGRAMS with basic statistical methods, which is published by SAS INSTITUTE ONLY. Book: "Common Statistical Methods for Clinical Research with SAS examples" (2nd eddition) Author: Glenn A. Walker Thanks & Regards, Nirali 
d_labes ★★★ Berlin, Germany, 20080516 06:54 (edited by d_labes on 20080516 07:26) @ Nirali Posting: # 1850 Views: 15,592 

Dear Nirali, » Now, no need to struggle with beast just read below book to get better knowledge of SAS PROGRAMS ... But what to do if I am not able to read? I have done my scientific work all the time over years (approx. 35 up to now) without reading any page, book or paper . Especially for struggling with the beast it was not necessary for me to have a look into any book or even not into the the documentation. I am an INITIATED! Thus, don't hurt me with a newbies book at the end of my career. » Below programm will definitely help you . In doing what? Your mentioned code is a little bit outstanding and has its origin in SAS versions lacking the RANDOM statement. The version of GLM code above in this thread with the RANDOM statement has the benefit to make clear the nature of effects (fixed and / or random) under analysis. It opens the door in understanding the evaluation of more complex designs like replicate xover using PROC MIXED in SAS or other "Mixed model" ANOVA software. — Regards, Detlew 
kevan ☆ 20090525 13:46 @ d_labes Posting: # 3749 Views: 14,934 

» This result is correct if all effects are considered fixed. Hi d_labes, Possible all effects are considered fixed especially in nowadays population? Thanks. 
ElMaestro ★★★ Belgium?, 20090525 20:25 (edited by ElMaestro on 20090525 20:37) @ kevan Posting: # 3753 Views: 15,148 

Oi, this was an old thread. » Possible all effects are considered fixed especially in nowadays population? The standard 2,2,2BE model can be fit with a linear model or a mixed model (equivalently glm, lm, lme etc in R) and you can obtain the same results. When one uses the RANDOM statement in connection with PROC GLM, it means.......very little of relevance to the 2,2,2B situation, as I see it. It can be omitted and we're still happy. A model with additional random effects other than the good old epsilon residuals is a mixed model, and mixed models are not linear models (they are LMS = "Linear Models on Steroids"). PROC GLM fits a linear model (only!) and in the fiting process it treats all effects as fixed regardless of the presence of a RANDOM statement. The SAS documentation has the power to confuse: " ...the random effects are treated in a post hoc fashion after the complete fixed effect model is fit. This distinction affects other features in the GLM procedure, such as the results of the LSMEANS and ESTIMATE statements. These features assume that all effects are fixed, so that all tests and estimability checks for these statements are based on a fixed effects model, even when you use a RANDOM statement. Standard errors for estimates and LSmeans based on the fixed effects model may be significantly smaller than those based on a true random effects model; in fact, some functions that are estimable under a true random effects model may not even be estimable under the fixed effects model. Therefore, you should use the MIXED procedure to compute tests involving these features that take the random effects into account." Eh...."True random effects model"? OK, so PROC MIXED is for true random effects models, while PROC GLM is for ...what?....false random effect models, perhaps? (which presumably are fixed because they don't contain truly random effects)? Aha, now things are very clear. Does anyone know if there are some handy statements for nonlinear mixed effects models which are not truly linear but also not nonlinear but only sort of a little bit possibly nonlinear on a foggy day, and which can be included or omitted without consequences? EM. 
d_labes ★★★ Berlin, Germany, 20090527 06:57 @ ElMaestro Posting: # 3773 Views: 15,016 

Dear ElMaestro, » The standard 2,2,2BE model can be fit with a linear model or a mixed model (equivalently glm, lm, lme etc in R) and you can obtain the same results. » When one uses the RANDOM statement in connection with PROC GLM, it means.......very little of relevance to the 2,2,2B situation, as I see it. » It can be omitted and we're still happy. Out of a walnut shell you are right. As long as you don't talk about a test of sequence effects. The RANDOM statement produces the correct test automatically. See above in this thread to notice that this is of relevance for a number of users (also with other software ), which are not so happy as you. And as long as you don't talk about the educational benefit of specifying which of your effects you consider fixed or random, which opens you the way to mixed models software necessary for the evaluation of more complicated study designs. I must confess that I never understand this SASophylistic "... post hoc fashion ...". But what is actually done in Proc GLM is nowadays known as "method of moments" in the mixed model context. BTW: Could you explain your "Linear model on Steroids" a bit more in detail? I couldn't get the the joke. — Regards, Detlew 
ElMaestro ★★★ Belgium?, 20090528 17:12 (edited by ElMaestro on 20090529 15:38) @ d_labes Posting: # 3784 Views: 14,885 

Hi dlabes, » BTW: Could you explain your "Linear model on Steroids" a bit more in detail? I couldn't get the the joke. In matrix notation, the normal linear model is written y = Xb + e, and the normal linear mixed model is y = Xb + Zg + e, i.e. the mixed one is just the linear one plus the Zgterm (model matrix for the random effects and the corresponding random effects vector). "On steroids" is a phrase used to indicate that something has been extended/inflated/improved/enlarged/soupedup etc. Finally, your explanation about the bogus term is very fine  I did not know that it results in an anova table with the correct F/Pvalue for the sequence effect. But I think it misses my point slightly. To me, manuals/documentation is where I look and read when I am faced with a problem. How do I etc and why. Let's say you are in a situation where you need to analyse a 2treatment, 4period BE study. OK, so you need a mixed model. But should you go for PROC MIXED or PROC GLM in combo with the bogus statement? Does the documentation reffed above help chosing here? I think it doesn't; in fact I think it adds more confusion than it helps. One of the old jokes about the Delphi programming language was that you needed to be an expert Delphi programmer in order to understand the Delphi manual. You see a parallel? Best regards EM. PS: I would use MIXED. 
d_labes ★★★ Berlin, Germany, 20090527 07:03 @ kevan Posting: # 3774 Views: 14,837 

Dear Kevan, » Possible all effects are considered fixed especially in nowadays population? Sorry, but I can not get your point? What is the question? What is your "nowadays population"? — Regards, Detlew 
Helmut ★★★ Vienna, Austria, 20081231 15:42 @ Helmut Posting: # 2995 Views: 15,916 

Dear all! » […] only Kinetica tests against the withinsubject residual error; this a well known bug since v1.1 (‼) – and I would expect Thermo_{Scientific} never to correct it… Expectation fulfilled. The bug remained uncorrected in the new release (v5.0 of 19 Dec 2008)… — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 