dmfisher
☆    

USA,
2015-06-09 05:55
(3215 d 15:23 ago)

(edited by Ohlbe on 2015-06-09 07:46)
Posting: # 14930
Views: 50,748
 

 R code for lm.mod does not make sense [🇷 for BE/BA]

The R code for lm.mod is (obtained by typing lm.mod at a prompt):

    cat("\n")
    target.lm <- lm(target ~ seq + subj:seq + prd + drug, data = inputdata)
    BearAnova = anova(target.lm)
    row.names(BearAnova)[4] = "subj(seq)"
    show(BearAnova[2:5, ])
    cat("\n")
    colnames(BearAnova) <- c(" DF", "   Type I SS", " Mean Square",
        " F Value", " Pr > F")
    show(BearAnova[2:4, ])
    cat("\n")
    colnames(BearAnova) <- c(" DF", " Type III SS", " Mean Square",
        " F Value", " Pr > F")
    show(BearAnova[2:4, ])
    cat("-----------------------------------------------------------\n")
    cat("\n")
    cat("Tests of Hypothesis using the Type I MS for \n")
    cat("SUBJECT(SEQUENCE) as an error term\n\n")
    show(summary(aov(target ~ seq, data = inputdata)))
    cat("\n")
    cat("Tests of Hypothesis using the Type III MS for\n")
    cat("SUBJECT(SEQUENCE) as an error term\n\n")
    show(summary(aov(target ~ seq, data = inputdata)))
    cat("---\n")
    cat("Sum Sq. = Type III SS\n\n")
    cat(" Tests for carryover effects and direct formulation effects\n")
    cat("------------------------------------------------------------")
    show(summary(aov(target ~ seq * drug + Error(subj), data = inputdata)))
    cat("------------------------------------------------------------\n")


There are several instances of
     show(BearAnova ...)
The column names are changed between these commands but the contents of the object are unchanged. This makes no sense — how could the values be the same for Type I and Type III SS?


Edit: category changed [Ohlbe]
Edit: Please follow the Forum’s Policy. [Helmut]
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2015-06-09 19:57
(3215 d 01:21 ago)

@ dmfisher
Posting: # 14935
Views: 13,624
 

 R code for lm.mod does not make sense

Dear dmfisher,

I guess that you talk about bear. Type I and Type III SS will be the same only when a 2x2x2 study is a balanced design. Otherwise, they are not the same. Doing this is just to mimic output results (Type I and Type III SS) obtained from SAS (PROC GLM). For more previous discussions of Type I and Type III SS with R, please search this Forum.

❝ [...] The column names are changed between these commands but the contents of the object are unchanged. This makes no sense — how could the values be the same for Type I and Type III SS?


All the best,
-- Yung-jin Lee
bear v2.9.1:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
ElMaestro
★★★

Denmark,
2015-06-09 23:07
(3214 d 22:11 ago)

@ yjlee168
Posting: # 14936
Views: 13,499
 

 R code for lm.mod does not make sense

Hi Yung-jin,


Looks like first you calculate a type I anova and print it with "type I" in the header. Next you print the same anova with "type III" in the header.
This would only make sense if this code snippet is guaranteed to execute for a balanced dataset only.

Pass or fail!
ElMaestro
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2015-06-10 13:34
(3214 d 07:44 ago)

@ ElMaestro
Posting: # 14939
Views: 13,544
 

 R code for lm.mod does not make sense

Dear Elmaestro and dmfisher,

No problem. I will remove Type III SS for next release. Many thanks.

❝ [...] This would only make sense if this code snippet is guaranteed to execute for a balanced dataset only.


All the best,
-- Yung-jin Lee
bear v2.9.1:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
ElMaestro
★★★

Denmark,
2015-06-10 16:22
(3214 d 04:56 ago)

@ yjlee168
Posting: # 14940
Views: 13,386
 

 R code for lm.mod does not make sense

Hi Yung-jin,

❝ No problem. I will remove Type III SS for next release. Many thanks.


wouldn't it perhaps be wiser to keep Type III SS for the next release; all you need to do is update the Bear Anova object with the type III information before printing it. Should be easy enough, I imagine this is already done in other parts of your code.

Keep up the good work with Bear for R :-)

Pass or fail!
ElMaestro
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2015-06-11 11:44
(3213 d 09:34 ago)

@ ElMaestro
Posting: # 14941
Views: 13,283
 

 R code for lm.mod does not make sense

Dear Elmaestro,

You mean using drop1() with manual corrections like your previous posts and this? I have done that part while ago. However, I did not have SAS to validate the codes for unbalanced dataset. Plus, recent WHM's post appeared. OK, I will see what I can do. Many thanks for your encouragement. :ok:

❝ Keep up the good work with Bear for R :-)


All the best,
-- Yung-jin Lee
bear v2.9.1:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2015-06-11 13:50
(3213 d 07:28 ago)

@ yjlee168
Posting: # 14942
Views: 13,374
 

 Reference datasets

Dear Yung-jin,

❝ […] However, I did not have SAS to validate the codes for unbalanced dataset.


There is no need to “validate” software against SAS. I’ve sent you our papers and the cross­over reference datasets on April 17th. ;-)

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2015-06-11 14:01
(3213 d 07:17 ago)

@ Helmut
Posting: # 14943
Views: 13,179
 

 Reference datasets

Hi Yung-jin,

❝ There is no need to “validate” software against SAS. I’ve sent you our papers and the crossover reference datasets on April 17th. ;-)


And in addition, I believe the code that is available nline as supplementary material to those papers may be useful for the purpose of producing type III SS anovas.

Have a good day.

Pass or fail!
ElMaestro
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2015-06-12 00:16
(3212 d 21:02 ago)

@ ElMaestro
Posting: # 14945
Views: 13,229
 

 Reference datasets

Dear Elmaestro,

Thanks again for your great information. I just run the codes and found there were slightly different from what I did before. They really did a great job.

All the best,
-- Yung-jin Lee
bear v2.9.1:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
d_labes
★★★

Berlin, Germany,
2015-06-12 11:04
(3212 d 10:14 ago)

@ ElMaestro
Posting: # 14947
Views: 13,270
 

 Type III kuddel muddel

Dear Helmut, dear ÖbersterGrößterMeister, dear Yung-jin,

❝ ❝ There is no need to “validate” software against SAS. I’ve sent you our papers and the crossover reference datasets on April 17th. ;-)


❝ And in addition, I believe the code that is available online as supplementary material to those papers may be useful for the purpose of producing type III SS anovas.


Do you remember that we have deliberately decided not to deal with the question(s) of Type I, Type III or whatever Type of sum of squares in our papers?

Do you remember also that the SAS Type III sum of squares may not exactly reproduced in R out of the box? We had a lot of discussions here in the forum about that topic.

WinNonlin/Phoenix uses a different form of F-Tests (not based on Type III sos) AFAIK.

So what can we suggest to Yung-jin w.r.t. the ANOVA?
SAS way, Phoenix way, R's drop1() (which doesn't work for sequence) or what else?

The attempt a well-known gangsta has done in the online supplementary material / code w.r.t. Type III sum of squares is not validated (against whatever, preferably SAS :-D since it was invented by the guys at SAS) and may prove as totally in error (except for balanced designs where it is sufficient to change only the header from Type I to Type III :cool:).

Ok, the whole kuddel muddel is only a side scene since the primary concern, the 90% CI as bioequivalence measure doesn't depend in any way from such considerations. Thank goodness!
But regulators like to see some sort of effect tests in the ANOVA. Which one is their secret

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2015-06-12 13:27
(3212 d 07:51 ago)

@ d_labes
Posting: # 14948
Views: 13,396
 

 Type III kuddel muddel

Dear Detlew & all,

❝ WinNonlin/Phoenix uses a different form of F-Tests (not based on Type III sos) AFAIK.


Yes. An excerpt of the manual here and an example there. There is no guaran­tee that Phoenix/WinNonlin and SAS will be that close in all cases.

❝ So what can we suggest to Yung-jin w.r.t. the ANOVA?

❝ SAS way, Phoenix way, R's drop1() (which doesn't work for sequence) or what else?


Duno. We explored package lsmeans in this lengthy thread without a final con­clusion. For the R-com­mu­nity’s “dogma” see Venables’ paper* (section 5).

❝ […] the primary concern, the 90% CI as bioequivalence measure doesn't depend in any way from such considerations. Thank goodness!


:ok:

❝ But regulators like to see some sort of effect tests in the ANOVA. Which one is their secret


I would rather say “[…] by which software and/or coding is their secret”. I never gave anything but the PE and CI in the main text of my reports (the complete eva­lu­ation only in the appendix). Although some assessors or inspectors routinely re­cal­cu­late studies (in SAS or in the semi-secret WinBE?) I never received a ques­tion. Maybe they compared only the CI, which – as you said – is the only relevant part.

When it comes to the type III stuff I gather that it will be practically impossible (e.g., Phoenix/WinNonlin) or difficult (R?) to exactly reproduce SAS’ results. AFAIK only SPSS uses the same algo as SAS.


  • Venables WN. Exegeses on Linear Models. S-PLUS User’s Conference Washington, DC, 8–9th October, 1998. [image] free resouce.

PS: Kuddelmuddel ~ hash, jumble, mess…


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2015-06-12 17:23
(3212 d 03:54 ago)

@ Helmut
Posting: # 14950
Views: 13,107
 

 ANOVA F-test results and regulators

Dear Helmut,

❝ ...

❝ ❝ But regulators like to see some sort of effect tests in the ANOVA ...


❝ ... I never gave anything but the PE and CI in the main text of my reports (the complete eva­lu­ation only in the appendix).


Here we are two, or more than 2 :cool:

❝ Although some assessors or inspectors routinely re­cal­cu­late studies (in SAS or in the semi-secret WinBE?) I never received a ques­tion. Maybe they compared only the CI, which – as you said – is the only relevant part.


May I call you Felix the Fortunate, the Happy? :-D

We received then and when (not so seldom!) questions about the ANOVA F-test results. Up to the request to comment significant results even for the treatment effects, of no relevance in case of BE studies of course.
Nasty the requests to comment the "validity" of the BE evaluation if period and/or sequence effects are found significant. As they many times come out if Type I tests will be applied, especially for sequence.

Regards,

Detlew
ElMaestro
★★★

Denmark,
2015-06-12 17:29
(3212 d 03:49 ago)

@ d_labes
Posting: # 14951
Views: 13,108
 

 ANOVA F-test results and regulators

Hmmmm,

❝ We received then and when (not so seldom!) questions about the ANOVA F-test results. Up to the request to comment significant results even for the treatment effects, of no relevance in case of BE studies of course.

❝ Nasty the requests to comment the "validity" of the BE evaluation if period and/or sequence effects are found significant. As they many times come out if Type I tests will be applied, especially for sequence.


Why not just always fit the model with no intercept and type I. The first term is almost guaranteed to have a significant P-value. You can then discuss it as much as you feel like at the nuisance level. Specify Sequence first, and voila here's a nice discussion brewing already :-D You can have a standard answer ready for all procedures.

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2015-06-12 17:44
(3212 d 03:34 ago)

@ ElMaestro
Posting: # 14953
Views: 13,154
 

 ANOVA F-test results and regulators

Huummmm,

❝ Why not just always fit the model with no intercept and type I. The first term is almost guaranteed to have a significant P-value. You can then discuss it as much as you feel like at the nuisance level. Specify Sequence first, and voila here's a nice discussion brewing already :-D You can have a standard answer ready for all procedures.


Didn't get your point. Can you elaborate in slow motion and simple words?

As I see it you can't discuss with regulators that their request is nuisance aka crap aka nonsense. Not if you have to discuss a submission.
Only if you meet one at conferences and have forgotten to take your pills ... :-D

Regards,

Detlew
ElMaestro
★★★

Denmark,
2015-06-12 17:52
(3212 d 03:26 ago)

@ d_labes
Posting: # 14954
Views: 13,040
 

 ANOVA F-test results and regulators

Hi d_labes,

❝ Didn't get your point. Can you elaborate in slow motion and simple words?


I think that when you fit without an intercept then you are basically asking if the levels of the first term differ from zero. When you fit with an intercept you are basically asking if the levels of the first term differ from the intercept (mean).
So when the intercept is not zero, the first term regularly comes out with a low p-value.

Pass or fail!
ElMaestro
ElMaestro
★★★

Denmark,
2015-06-12 14:18
(3212 d 07:00 ago)

@ d_labes
Posting: # 14949
Views: 13,121
 

 Type III again

Dear d_labes,

❝ So what can we suggest to Yung-jin w.r.t. the ANOVA?

❝ SAS way, Phoenix way, R's drop1() (which doesn't work for sequence) or what else?

  1. If the aim is to reproduce SAS then yes, we need to update the anova table with F= MS(Seq) / MS(Subj).
  2. The idea of testing MS(Seq) against the MS(Subj) is still just a postulate. One that of course makes people invest heart and sould in it, but still a postulate.
  3. Drop1 gives you type III SS, plain and simple with no questions asked. Type III is when the least term is left out and hence it can rightfully be claimed that by the shear defintion of type III the Seq effect has to be nil.
So, I am not so worried about drop1 scientifically. I am only worried about assessors who think that if it doesn't give the same result as SAS then it is wrong. Why don't those assessors instead ask why the sequence effect with type III is not nil with SAS. Much more relevant :-D

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2015-06-12 17:32
(3212 d 03:46 ago)

@ ElMaestro
Posting: # 14952
Views: 13,190
 

 SAS OT

Dear ElMaestro,

regarding your signature I have an addition:

Why? Because SAS says so. And SAS is validated.
Per se. Of course without any user attempts to show that it does what it should be.

Regards,

Detlew
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2015-06-11 21:56
(3212 d 23:22 ago)

@ Helmut
Posting: # 14944
Views: 13,268
 

 Reference datasets

Dear Helmut,

Thanks for reminding me. You have sent me your papers and the ref. datasets before. I just check that *.rtf file and find something interesting in there. I get back to you if I have any question. Great and thanks again.

❝ There is no need to “validate” software against SAS. I’ve sent you our papers and the cross­over reference datasets on April 17th. ;-)


All the best,
-- Yung-jin Lee
bear v2.9.1:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
yjlee168
★★★
avatar
Homepage
Kaohsiung, Taiwan,
2015-06-12 11:01
(3212 d 10:17 ago)

@ Helmut
Posting: # 14946
Views: 13,290
 

 Test hypothesis of using Type I/III MS...

Dear Helmut, detlew, Elmaestro, WHM and all other R users,

Do we also need to test hypothesis using Type I/III MS for SUBJECT(SEQUENCE) as an error term as SAS's proc glm? WHM's previous post mentioned that, but did not provide further validation information (vs. SAS). If yes, how should we do that with R? Thanks in advanced.

All the best,
-- Yung-jin Lee
bear v2.9.1:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here
dmfisher
☆    

USA,
2015-06-10 03:38
(3214 d 17:40 ago)

@ yjlee168
Posting: # 14937
Views: 13,532
 

 R code for lm.mod does not make sense

Please look at ElMaestro's comment — perhaps he explained the problem better than I did.


Edit: Full quote removed; delete everything from the text of the original poster which is not necessary in understanding your answer; see also this post! Please follow the Forum’s Policy. [Helmut]
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
96 visitors (0 registered, 96 guests [including 8 identified bots]).
Forum time: 20:18 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5