Log in
Register|
Eva ● 2008-05-06 16:04 (5825 d 23:16 ago) Posting: # 1817 Views: 26,173 |
|
|
Dear all, I have recently analysed a simple crossover bioequivalence study in SAS and detected a significant sequence effect (p=0.0425). To verify my results, I have run the same analysis using the WinNonlin Bioequivalence Wizard (WNL v5.1.1). The results of the analysis were completely identical except for the sequence effect which was not significant (p=0.0634). The Pharsight support claims that this is due to a slightly different model used by the Bioequivalence Wizard and recommended me to read the online help on linear mixed effects modelling. I can not follow this argumentation, because in my opinion my model specification is the same. Besides I get identical results for all other effects: both my SAS program and the BE Wizard produce the same SS, MS and DF for sequence effect, period effect, treatment effect and subject within sequence, the MSE is identical in both programs and (consequently) also the confidence limits. The F statistics and the p-values are the same for period effect, treatment effect and subject within sequence. I have repeated the comparison with other data with the same result, all values identical, but the F statistic of the sequence effect always lower in WNL compared to SAS. Has any of you ever come across this discrepancy before? I wonder if somebody could help me to find out if there is a mistake in the Bioequivalence Wizard calculation or if I am simply too stupid to model my data correctly... Thanks in advance & kind regards, Eva |
|
Ohlbe ★★★ France, 2008-05-06 17:56 (5825 d 21:24 ago) @ Eva Posting: # 1821 Views: 20,865 |
|
|
Dear Eva, I am not a SAS or WinNonlin specialist, but I would have a basic question. Where do you read the SAS F and p values from? Usually the printouts I see from SAS present the results in two parts. You first have the full ANOVA with sequence, subject(seq), period and treatment (with F and p values), then a specific test for sequence using the MS for subject(seq) as an error term, (again with a F and p value). Only the results of this specific test are supposed to be considered. Could this be the origin of the difference you found? Regards Ohlbe |
|
Helmut ★★★   Vienna, Austria, 2008-05-06 18:13 (5825 d 21:07 ago) @ Eva Posting: # 1822 Views: 20,789 |
|
|
Dear Eva! One example in SAS… SUM OF … and WinNonlin 5.2 Partial Sum of SquaresObviously sequence effects are identical (p 0.187). Maybe you had some drop-outs in your study, which resulted in imbalanced sequences. In such a case SAS’ SS Type III and WinNonlin’s Partial Sum of Squares may differ (although they ‘coincide in most situations’). SAS SS III is a strange cup of tea anyhow…  It would be nice, if you would come up with a simple dataset (12 subjects?), where you find differences. It should be possible to track it down! — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png) Helmut Schütz ![[image]](https://static.bebac.at/img/CC by.png) The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Eva ● 2008-05-06 19:42 (5825 d 19:39 ago) @ Helmut Posting: # 1824 Views: 20,694 |
|
|
Dear Ohlbe and Helmut, thanks for your fast replies! Here is a sample data set of 8 subjects, maybe this can help to illustrate my problem...
Subject Period Sequence Treatment AUCMy SAS results (PROC GLM) for these data are
EFFECT DF TYPE III SS MEAN SQUARE F-VALUE P-VALUEand this is what WinNonlin calculated:
EFFECT DF TYPE III SS MEAN SQUARE F-VALUE P-VALUEDear Ohlbe, I think I have picked the right F- and p-values from my output, I am using PROC GLM (and I just checked, the output is identical with PROC ANOVA) and picked the Sequence, Subject(Sequence), Treatment and Period results from the bottom part and the Error results from the top part. Dear Helmut, could this be a problem with my WNL version? (I noticed that you used v5.2, whereas I have v5.1.1) As you can see above, the sums of squares are not the problem, it is only the F- and p-values that differ. I understood that the F-value is the ratio of the effect mean square and the mean square error. This is correct for the SAS results (sequence: 0.6105/0.2481=2.46), but not for WinNonLin... Any ideas? Thanks again, for your help and for sharing your precious time with my "little" problem... Kind regards, Eva Edit: Treatment in the table changed to Test to avoid confusion with the heading. [Helmut] |
|
Ohlbe ★★★ France, 2008-05-06 21:25 (5825 d 17:55 ago) @ Eva Posting: # 1825 Views: 20,669 |
|
|
Dear Eva, F is the ratio of the mean square of your factor, divided by the mean square of what you consider as the error factor. SAS divided the MS of the sequence effect (0.6105) by the MS of the residual error (0.2481). WinNonlin divided the sequence MS by the MS of subject(sequence) (0.2623). This is what SAS does in the second step I mentioned previously. Helmut, I will let you explain why subject(sequence) should be used as an error term Regards Ohlbe |
|
Helmut ★★★ Vienna, Austria, 2008-05-06 23:33 (5825 d 15:48 ago) @ Ohlbe Posting: # 1826 Views: 20,932 |
|
|
Dear Eva & Ohlbe! ❝ Helmut, I will let you explain why subject(sequence) should be used as an error term OK, sequences are a between subject effect and therefore need to be compared to the between subject residual error (or F=0.6105/0.2623=2.327). In other words we compare (the group of) subjects in sequence TR to (the group of) subjects in sequence RT. If you compare the sequence against the within subject residual error (0.2481) you certainly will get a different – and wrong! – answer. Using the within subject residual actually overestimates the effect of sequence leading to a greater chance of concluding a sequence effect. Since in my first post I showed identical results for SAS and WinNonlin, I would expect a coding issue in the SAS program you are using. Results in WinNonlin-versions don't differ. Unfortunately it’s not possible to have different versions running on the same machine – I re-run ‘old’ evaluations everytime I update to a newer version – and did not get differences since v4.0.1 (3.3 and earlier versions used ANOVA instead of linear mixed effects modeling). Just to be more comfortable about the results – a little roundup of your dataset: WinNonlin5.2 (2007) ----------------------------------------------------------Chow & Liu (2001) – coded in Excel -------------------------------------------------------EquivTest/PK (2006) ------------------------------------------------Ref [3] (2001) from this post; coded in R2.7.0 (2008)-------------------------------------------------------NCSS (2001) (Cross-Over Analysis) -----------------------------------------------Kinetica 4.4.1 (2007) --------------------------------------------------------![[image]](img/uploaded/image4.gif) Besides your SAS code only Kinetica tests against the within-subject residual error; this a well known bug since v1.1 (‼) – and I would expect ThermoScientific never to correct it…  — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Helmut ★★★ Vienna, Austria, 2008-05-07 20:18 (5824 d 19:02 ago) @ Helmut Posting: # 1830 Views: 20,706 |
|
|
❝ … Unfortunatelly it’s not possible to have different versions running on the same machine… OK, I remembered WinNonlin4.1 to be still installed on an old machine; again same results for the sequence effect: WinNonlin4.1 (2003)
Partial Tests of Model Effects— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
d_labes ★★★ Berlin, Germany, 2008-05-08 13:16 (5824 d 02:04 ago) @ Helmut Posting: # 1834 Views: 20,788 |
|
|
Dear Eva & Ohlbe & Helmut ❝ Since in my first post I showed identical results for SAS and WinNonlin, I would expect a coding issue in the SAS program you are using. That is totally right and wrong. The SAS Procedure GLM has the "standard" of treating all effects as fixed effects. Thus if you use the code: Proc GLM data=test;you get Eva's result. This result is correct if all effects are considered fixed. That means regarding the subject effect we are making inference about the subjects which are actually under study. But the goal of our BE study is to broaden the scope and make inference about some "population" of subjects (healthy volunteers f.i. from which we have randomly sampled our subjects actually under study). This leads to subject as a random factor (effect) in our model. The code in Proc GLM for that is: Proc GLM data=test;Proc GLM is resistant to our wishes and first displays the fixed effect analysis again (this is because according to SAS GLM handles random effects in a "post hoc manner", in direct words this feature was invented later and tinkered into GLM "mit heißer Nadel"). But then the sun is shining bright (emphasis by me):The GLM ProcedureHope this helps a little bit in "the power to know". SAS is an outstanding beast  .Side note: The annoying fixed effects analysis displayed again is not needed and can be suppressed using the "famous" ODS (whatever this is). Final code: ODS exclude ExpectedMeanSquares ModelANOVA;— Regards, Detlew |
|
Helmut ★★★ Vienna, Austria, 2008-05-08 19:18 (5823 d 20:02 ago) @ d_labes Posting: # 1837 Views: 20,695 |
|
|
Dear DLabes! ❝ SAS is an outstanding beast…  ❝ […] using the "famous" ODS (whatever this is). Though I don’t have any idea about SAS, Google is my friend – coming up with ODS (Output Delivery System)… — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
d_labes ★★★ Berlin, Germany, 2008-05-09 11:35 (5823 d 03:45 ago) @ Helmut Posting: # 1840 Views: 20,490 |
|
|
Dear Helmut, ❝ Google is my friend – coming up with ODS = Output Delivery System… but this is only for THE INITIATED dragon fighter struggling with the beast  .— Regards, Detlew |
|
Nirali ★ India, 2008-05-09 13:00 (5823 d 02:20 ago) (edited by Nirali on 2008-05-10 04:30) @ d_labes Posting: # 1842 Views: 20,641 |
|
|
Dear Friend, Yes, WinNonlin & SAS are giving same outputs for all effects of ANOVA all the time, just need to modify the SAS programm according to the requirement. Below programm will definitely help you  .PROC GLM DATA=TEST; CLASS SEQUENCE SUBJECT PERIOD TREATMENT; MODEL AUC=SEQUENCE SUBJECT(SEQUENCE) TREATMENT PERIOD /SS3; TEST H=SEQUENCE E=SUBJECT(SEQUENCE); RUN; QUIT; ❝ SAS is an outstanding beast Now, no need to struggle with beast  just read below book to get better knowledge of SAS PROGRAMS with basic statistical methods, which is published by SAS INSTITUTE ONLY. Book: "Common Statistical Methods for Clinical Research with SAS examples" (2nd eddition) Author: Glenn A. Walker Thanks & Regards, Nirali |
|
d_labes ★★★ Berlin, Germany, 2008-05-16 10:54 (5816 d 04:27 ago) @ Nirali Posting: # 1850 Views: 20,322 |
|
|
Dear Nirali, ❝ Now, no need to struggle with beast But what to do if I am not able to read? I have done my scientific work all the time over years (approx. 35 up to now) without reading any page, book or paper  .Especially for struggling with the beast it was not necessary for me to have a look into any book or even not into the the documentation. I am an INITIATED! Thus, don't hurt me with a newbies book at the end of my career. ❝ Below programm will definitely help you In doing what? Your mentioned code is a little bit outstanding and has its origin in SAS versions lacking the RANDOM statement. The version of GLM code above in this thread with the RANDOM statement has the benefit to make clear the nature of effects (fixed and / or random) under analysis. It opens the door in understanding the evaluation of more complex designs like replicate x-over using PROC MIXED in SAS or other "Mixed model" ANOVA software. — Regards, Detlew |
|
kevan ☆ 2009-05-25 17:46 (5441 d 21:34 ago) @ d_labes Posting: # 3749 Views: 19,673 |
|
|
❝ This result is correct if all effects are considered fixed. Hi d_labes, Possible all effects are considered fixed especially in nowadays population? Thanks. |
|
ElMaestro ★★★ Denmark, 2009-05-26 00:25 (5441 d 14:55 ago) @ kevan Posting: # 3753 Views: 19,951 |
|
|
Oi, this was an old thread. ❝ Possible all effects are considered fixed especially in nowadays population? The standard 2,2,2-BE model can be fit with a linear model or a mixed model (equivalently glm, lm, lme etc in R) and you can obtain the same results. When one uses the RANDOM statement in connection with PROC GLM, it means.......very little of relevance to the 2,2,2-B situation, as I see it. It can be omitted and we're still happy. A model with additional random effects other than the good old epsilon residuals is a mixed model, and mixed models are not linear models (they are LMS = "Linear Models on Steroids"). PROC GLM fits a linear model (only!) and in the fiting process it treats all effects as fixed regardless of the presence of a RANDOM statement. The SAS documentation has the power to confuse: " ...the random effects are treated in a post hoc fashion after the complete fixed effect model is fit. This distinction affects other features in the GLM procedure, such as the results of the LSMEANS and ESTIMATE statements. These features assume that all effects are fixed, so that all tests and estimability checks for these statements are based on a fixed effects model, even when you use a RANDOM statement. Standard errors for estimates and LS-means based on the fixed effects model may be significantly smaller than those based on a true random effects model; in fact, some functions that are estimable under a true random effects model may not even be estimable under the fixed effects model. Therefore, you should use the MIXED procedure to compute tests involving these features that take the random effects into account." Eh...."True random effects model"? OK, so PROC MIXED is for true random effects models, while PROC GLM is for ...what?....false random effect models, perhaps? (-which presumably are fixed because they don't contain truly random effects)? Aha, now things are very clear. Does anyone know if there are some handy statements for nonlinear mixed effects models which are not truly linear but also not nonlinear but only sort of a little bit possibly nonlinear on a foggy day, and which can be included or omitted without consequences? EM. |
|
d_labes ★★★ Berlin, Germany, 2009-05-27 10:57 (5440 d 04:24 ago) @ ElMaestro Posting: # 3773 Views: 19,785 |
|
|
Dear ElMaestro, ❝ The standard 2,2,2-BE model can be fit with a linear model or a mixed model (equivalently glm, lm, lme etc in R) and you can obtain the same results. ❝ When one uses the RANDOM statement in connection with PROC GLM, it means.......very little of relevance to the 2,2,2-B situation, as I see it. ❝ It can be omitted and we're still happy. Out of a walnut shell you are right.As long as you don't talk about a test of sequence effects. The RANDOM statement produces the correct test automatically. See above in this thread to notice that this is of relevance for a number of users (also with other software ), which are not so happy as you.And as long as you don't talk about the educational benefit of specifying which of your effects you consider fixed or random, which opens you the way to mixed models software necessary for the evaluation of more complicated study designs. I must confess that I never understand this SASophylistic "... post hoc fashion ...". But what is actually done in Proc GLM is nowadays known as "method of moments" in the mixed model context. BTW: Could you explain your "Linear model on Steroids" a bit more in detail? I couldn't get the the joke. — Regards, Detlew |
|
ElMaestro ★★★ Denmark, 2009-05-28 21:12 (5438 d 18:09 ago) (edited by ElMaestro on 2009-05-29 17:38) @ d_labes Posting: # 3784 Views: 19,601 |
|
|
Hi dlabes, ❝ BTW: Could you explain your "Linear model on Steroids" a bit more in detail? I couldn't get the the joke. In matrix notation, the normal linear model is written y = Xb + e, and the normal linear mixed model is y = Xb + Zg + e, i.e. the mixed one is just the linear one plus the Zg-term (model matrix for the random effects and the corresponding random effects vector). "On steroids" is a phrase used to indicate that something has been extended/inflated/improved/enlarged/souped-up etc. Finally, your explanation about the bogus term is very fine - I did not know that it results in an anova table with the correct F/P-value for the sequence effect. But I think it misses my point slightly. To me, manuals/documentation is where I look and read when I am faced with a problem. How do I etc and why. Let's say you are in a situation where you need to analyse a 2-treatment, 4-period BE study. OK, so you need a mixed model. But should you go for PROC MIXED or PROC GLM in combo with the bogus statement? Does the documentation reffed above help chosing here? I think it doesn't; in fact I think it adds more confusion than it helps. One of the old jokes about the Delphi programming language was that you needed to be an expert Delphi programmer in order to understand the Delphi manual. You see a parallel? Best regards EM. PS: I would use MIXED. |
|
d_labes ★★★ Berlin, Germany, 2009-05-27 11:03 (5440 d 04:18 ago) @ kevan Posting: # 3774 Views: 19,644 |
|
|
Dear Kevan, ❝ Possible all effects are considered fixed especially in nowadays population?  What is your "nowadays population"? — Regards, Detlew |
|
Helmut ★★★ Vienna, Austria, 2008-12-31 17:42 (5586 d 20:38 ago) @ Helmut Posting: # 2995 Views: 20,721 |
|
|
Dear all! ❝ Expectation fulfilled. The bug remained uncorrected in the new release (v5.0 of 19 Dec 2008)…  — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Ing. Helmut Schütz