Bioequivalence and Bioavailability Forum

dmfisher
☆

USA,
2015-06-09 05:55
(3574 d 23:59 ago)

(edited on 2015-06-09 07:46)
Posting: # 14930
Views: 52,075

R code for lm.mod does not make sense [🇷 for BE/BA]

The R code for lm.mod is (obtained by typing lm.mod at a prompt):

    cat("\n")

    target.lm <- lm(target ~ seq + subj:seq + prd + drug, data = inputdata)

    BearAnova = anova(target.lm)

    row.names(BearAnova)[4] = "subj(seq)"

    show(BearAnova[2:5, ])

    cat("\n")

    colnames(BearAnova) <- c(" DF", "   Type I SS", " Mean Square", 

        " F Value", " Pr > F")

    show(BearAnova[2:4, ])

    cat("\n")

    colnames(BearAnova) <- c(" DF", " Type III SS", " Mean Square", 

        " F Value", " Pr > F")

    show(BearAnova[2:4, ])

    cat("-----------------------------------------------------------\n")

    cat("\n")

    cat("Tests of Hypothesis using the Type I MS for \n")

    cat("SUBJECT(SEQUENCE) as an error term\n\n")

    show(summary(aov(target ~ seq, data = inputdata)))

    cat("\n")

    cat("Tests of Hypothesis using the Type III MS for\n")

    cat("SUBJECT(SEQUENCE) as an error term\n\n")

    show(summary(aov(target ~ seq, data = inputdata)))

    cat("---\n")

    cat("Sum Sq. = Type III SS\n\n")

    cat(" Tests for carryover effects and direct formulation effects\n")

    cat("------------------------------------------------------------")

    show(summary(aov(target ~ seq * drug + Error(subj), data = inputdata)))

    cat("------------------------------------------------------------\n")

There are several instances of
show(BearAnova ...)
The column names are changed between these commands but the contents of the object are unchanged. This makes no sense — how could the values be the same for Type I and Type III SS?

Edit: category changed [Ohlbe]
Edit: Please follow the Forum’s Policy. [Helmut]

yjlee168
★★★

Kaohsiung, Taiwan,
2015-06-09 19:57
(3574 d 09:57 ago)

@ dmfisher
Posting: # 14935
Views: 14,763

R code for lm.mod does not make sense

Post reply

Dear dmfisher,

I guess that you talk about bear. Type I and Type III SS will be the same only when a 2x2x2 study is a balanced design. Otherwise, they are not the same. Doing this is just to mimic output results (Type I and Type III SS) obtained from SAS (PROC GLM). For more previous discussions of Type I and Type III SS with R, please search this Forum.

❝ [...] The column names are changed between these commands but the contents of the object are unchanged. This makes no sense — how could the values be the same for Type I and Type III SS?

—
All the best,
-- Yung-jin Lee
bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here

ElMaestro ★★★ Denmark, 2015-06-09 23:07 (3574 d 06:46 ago) @ yjlee168 Posting: # 14936 Views: 14,610	R code for lm.mod does not make sense Post reply
	Hi Yung-jin, Looks like first you calculate a type I anova and print it with "type I" in the header. Next you print the same anova with "type III" in the header. This would only make sense if this code snippet is guaranteed to execute for a balanced dataset only. — Pass or fail! ElMaestro

yjlee168 ★★★ Kaohsiung, Taiwan, 2015-06-10 13:34 (3573 d 16:20 ago) @ ElMaestro Posting: # 14939 Views: 14,679	R code for lm.mod does not make sense Post reply
	Dear Elmaestro and dmfisher, No problem. I will remove Type III SS for next release. Many thanks. ❝ [...] This would only make sense if this code snippet is guaranteed to execute for a balanced dataset only. — All the best, -- Yung-jin Lee bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee Kaohsiung, Taiwan https://www.pkpd168.com/bear Download link (updated) -> here

ElMaestro
★★★

Denmark,
2015-06-10 16:22
(3573 d 13:32 ago)

@ yjlee168
Posting: # 14940
Views: 14,495

R code for lm.mod does not make sense

Post reply

Hi Yung-jin,

❝ No problem. I will remove Type III SS for next release. Many thanks.

wouldn't it perhaps be wiser to keep Type III SS for the next release; all you need to do is update the Bear Anova object with the type III information before printing it. Should be easy enough, I imagine this is already done in other parts of your code.

Keep up the good work with Bear for R :-)

—
Pass or fail!
ElMaestro

yjlee168
★★★

Kaohsiung, Taiwan,
2015-06-11 11:44
(3572 d 18:09 ago)

@ ElMaestro
Posting: # 14941
Views: 14,401

R code for lm.mod does not make sense

Post reply

Dear Elmaestro,

You mean using drop1() with manual corrections like your previous posts and this? I have done that part while ago. However, I did not have SAS to validate the codes for unbalanced dataset. Plus, recent WHM's post appeared. OK, I will see what I can do. Many thanks for your encouragement. :ok:

❝ Keep up the good work with Bear for R :-)

—
All the best,
-- Yung-jin Lee
bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here

Helmut
★★★

Vienna, Austria,
2015-06-11 13:50
(3572 d 16:04 ago)

@ yjlee168
Posting: # 14942
Views: 14,492

Reference datasets

Post reply

Dear Yung-jin,

❝ […] However, I did not have SAS to validate the codes for unbalanced dataset.

There is no need to “validate” software against SAS. I’ve sent you our papers and the crossover reference datasets on April 17^th. ;-)

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro ★★★ Denmark, 2015-06-11 14:01 (3572 d 15:53 ago) @ Helmut Posting: # 14943 Views: 14,315	Reference datasets Post reply
	Hi Yung-jin, ❝ There is no need to “validate” software against SAS. I’ve sent you our papers and the crossover reference datasets on April 17^th. And in addition, I believe the code that is available nline as supplementary material to those papers may be useful for the purpose of producing type III SS anovas. Have a good day. — Pass or fail! ElMaestro

yjlee168 ★★★ Kaohsiung, Taiwan, 2015-06-12 00:16 (3572 d 05:38 ago) @ ElMaestro Posting: # 14945 Views: 14,360	Reference datasets Post reply
	Dear Elmaestro, Thanks again for your great information. I just run the codes and found there were slightly different from what I did before. They really did a great job. — All the best, -- Yung-jin Lee bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee Kaohsiung, Taiwan https://www.pkpd168.com/bear Download link (updated) -> here

d_labes
★★★

Berlin, Germany,
2015-06-12 11:04
(3571 d 18:50 ago)

@ ElMaestro
Posting: # 14947
Views: 14,395

Type III kuddel muddel

Post reply

Dear Helmut, dear ÖbersterGrößterMeister, dear Yung-jin,

❝ ❝ There is no need to “validate” software against SAS. I’ve sent you our papers and the crossover reference datasets on April 17^th. ;-)

❝

❝ And in addition, I believe the code that is available online as supplementary material to those papers may be useful for the purpose of producing type III SS anovas.

Do you remember that we have deliberately decided not to deal with the question(s) of Type I, Type III or whatever Type of sum of squares in our papers?

Do you remember also that the SAS Type III sum of squares may not exactly reproduced in R out of the box? We had a lot of discussions here in the forum about that topic.

WinNonlin/Phoenix uses a different form of F-Tests (not based on Type III sos) AFAIK.

So what can we suggest to Yung-jin w.r.t. the ANOVA?
SAS way, Phoenix way, R's drop1() (which doesn't work for sequence) or what else?

The attempt a well-known gangsta has done in the online supplementary material / code w.r.t. Type III sum of squares is not validated (against whatever, preferably SAS :-D

since it was invented by the guys at SAS) and may prove as totally in error (except for balanced designs where it is sufficient to change only the header from Type I to Type III :cool:

).

Ok, the whole kuddel muddel is only a side scene since the primary concern, the 90% CI as bioequivalence measure doesn't depend in any way from such considerations. Thank goodness!
But regulators like to see some sort of effect tests in the ANOVA. Which one is their secret

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2015-06-12 13:27
(3571 d 16:27 ago)

@ d_labes
Posting: # 14948
Views: 14,503

Type III kuddel muddel

Post reply

Dear Detlew & all,

❝ WinNonlin/Phoenix uses a different form of F-Tests (not based on Type III sos) AFAIK.

Yes. An excerpt of the manual here and an example there. There is no guarantee that Phoenix/WinNonlin and SAS will be that close in all cases.

❝ So what can we suggest to Yung-jin w.r.t. the ANOVA?

❝ SAS way, Phoenix way, R's drop1() (which doesn't work for sequence) or what else?

Duno. We explored package lsmeans in this lengthy thread without a final conclusion. For the R-community’s “dogma” see Venables’ paper* (section 5).

❝ […] the primary concern, the 90% CI as bioequivalence measure doesn't depend in any way from such considerations. Thank goodness!

❝ But regulators like to see some sort of effect tests in the ANOVA. Which one is their secret

I would rather say “[…] by which software and/or coding is their secret”. I never gave anything but the PE and CI in the main text of my reports (the complete evaluation only in the appendix). Although some assessors or inspectors routinely recalculate studies (in SAS or in the semi-secret WinBE?) I never received a question. Maybe they compared only the CI, which – as you said – is the only relevant part.

When it comes to the type III stuff I gather that it will be practically impossible (e.g., Phoenix/WinNonlin) or difficult (R?) to exactly reproduce SAS’ results. AFAIK only SPSS uses the same algo as SAS.

Venables WN. Exegeses on Linear Models. S-PLUS User’s Conference Washington, DC, 8–9^th October, 1998. free resouce.

PS: Kuddelmuddel ~ hash, jumble, mess…

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2015-06-12 17:23
(3571 d 12:30 ago)

@ Helmut
Posting: # 14950
Views: 14,208

ANOVA F-test results and regulators

Post reply

Dear Helmut,

❝ ...

❝ ❝ But regulators like to see some sort of effect tests in the ANOVA ...

❝

❝ ... I never gave anything but the PE and CI in the main text of my reports (the complete evaluation only in the appendix).

Here we are two, or more than 2 :cool:

❝ Although some assessors or inspectors routinely recalculate studies (in SAS or in the semi-secret WinBE?) I never received a question. Maybe they compared only the CI, which – as you said – is the only relevant part.

May I call you Felix the Fortunate, the Happy? :-D

We received then and when (not so seldom!) questions about the ANOVA F-test results. Up to the request to comment significant results even for the treatment effects, of no relevance in case of BE studies of course.
Nasty the requests to comment the "validity" of the BE evaluation if period and/or sequence effects are found significant. As they many times come out if Type I tests will be applied, especially for sequence.

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2015-06-12 17:29
(3571 d 12:25 ago)

@ d_labes
Posting: # 14951
Views: 14,241

ANOVA F-test results and regulators

Post reply

Hmmmm,

❝ We received then and when (not so seldom!) questions about the ANOVA F-test results. Up to the request to comment significant results even for the treatment effects, of no relevance in case of BE studies of course.

❝ Nasty the requests to comment the "validity" of the BE evaluation if period and/or sequence effects are found significant. As they many times come out if Type I tests will be applied, especially for sequence.

Why not just always fit the model with no intercept and type I. The first term is almost guaranteed to have a significant P-value. You can then discuss it as much as you feel like at the nuisance level. Specify Sequence first, and voila here's a nice discussion brewing already :-D

You can have a standard answer ready for all procedures.

—
Pass or fail!
ElMaestro

d_labes
★★★

Berlin, Germany,
2015-06-12 17:44
(3571 d 12:09 ago)

@ ElMaestro
Posting: # 14953
Views: 14,258

ANOVA F-test results and regulators

Post reply

Huummmm,

❝ Why not just always fit the model with no intercept and type I. The first term is almost guaranteed to have a significant P-value. You can then discuss it as much as you feel like at the nuisance level. Specify Sequence first, and voila here's a nice discussion brewing already :-D You can have a standard answer ready for all procedures.

Didn't get your point. Can you elaborate in slow motion and simple words?

As I see it you can't discuss with regulators that their request is nuisance aka crap aka nonsense. Not if you have to discuss a submission.
Only if you meet one at conferences and have forgotten to take your pills ... :-D

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2015-06-12 17:52
(3571 d 12:01 ago)

@ d_labes
Posting: # 14954
Views: 14,139

ANOVA F-test results and regulators

Post reply

Hi d_labes,

❝ Didn't get your point. Can you elaborate in slow motion and simple words?

I think that when you fit without an intercept then you are basically asking if the levels of the first term differ from zero. When you fit with an intercept you are basically asking if the levels of the first term differ from the intercept (mean).
So when the intercept is not zero, the first term regularly comes out with a low p-value.

—
Pass or fail!
ElMaestro

ElMaestro
★★★

Denmark,
2015-06-12 14:18
(3571 d 15:36 ago)

@ d_labes
Posting: # 14949
Views: 14,228

Type III again

Post reply

Dear d_labes,

❝ So what can we suggest to Yung-jin w.r.t. the ANOVA?

❝ SAS way, Phoenix way, R's drop1() (which doesn't work for sequence) or what else?

If the aim is to reproduce SAS then yes, we need to update the anova table with F= MS(Seq) / MS(Subj).
The idea of testing MS(Seq) against the MS(Subj) is still just a postulate. One that of course makes people invest heart and sould in it, but still a postulate.
Drop1 gives you type III SS, plain and simple with no questions asked. Type III is when the least term is left out and hence it can rightfully be claimed that by the shear defintion of type III the Seq effect has to be nil.

So, I am not so worried about drop1 scientifically. I am only worried about assessors who think that if it doesn't give the same result as SAS then it is wrong. Why don't those assessors instead ask why the sequence effect with type III is not nil with SAS. Much more relevant :-D

—
Pass or fail!
ElMaestro

d_labes ★★★ Berlin, Germany, 2015-06-12 17:32 (3571 d 12:21 ago) @ ElMaestro Posting: # 14952 Views: 14,342	SAS OT Post reply
	Dear ElMaestro, regarding your signature I have an addition: Why? Because SAS says so. And SAS is validated. Per se. Of course without any user attempts to show that it does what it should be. — Regards, Detlew

yjlee168
★★★

Kaohsiung, Taiwan,
2015-06-11 21:56
(3572 d 07:58 ago)

@ Helmut
Posting: # 14944
Views: 14,397

Reference datasets

Post reply

Dear Helmut,

Thanks for reminding me. You have sent me your papers and the ref. datasets before. I just check that *.rtf file and find something interesting in there. I get back to you if I have any question. Great and thanks again.

❝ There is no need to “validate” software against SAS. I’ve sent you our papers and the crossover reference datasets on April 17^th. ;-)

—
All the best,
-- Yung-jin Lee
bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here

yjlee168
★★★

Kaohsiung, Taiwan,
2015-06-12 11:01
(3571 d 18:53 ago)

@ Helmut
Posting: # 14946
Views: 14,415

Test hypothesis of using Type I/III MS...

Post reply

Dear Helmut, detlew, Elmaestro, WHM and all other R users,

Do we also need to test hypothesis using Type I/III MS for SUBJECT(SEQUENCE) as an error term as SAS's proc glm? WHM's previous post mentioned that, but did not provide further validation information (vs. SAS). If yes, how should we do that with R? Thanks in advanced.

—
All the best,
-- Yung-jin Lee
bear v2.9.2:- created by Hsin-ya Lee & Yung-jin Lee
Kaohsiung, Taiwan https://www.pkpd168.com/bear
Download link (updated) -> here

dmfisher ☆ USA, 2015-06-10 03:38 (3574 d 02:15 ago) @ yjlee168 Posting: # 14937 Views: 14,642	R code for lm.mod does not make sense Post reply
	Please look at ElMaestro's comment — perhaps he explained the problem better than I did. Edit: Full quote removed; delete everything from the text of the original poster which is not necessary in understanding your answer; see also this post! Please follow the Forum’s Policy. [Helmut]

R code for lm.mod does not make sense [🇷 for BE/BA]

R code for lm.mod does not make sense

R code for lm.mod does not make sense

R code for lm.mod does not make sense

R code for lm.mod does not make sense

R code for lm.mod does not make sense

Reference datasets

Reference datasets

Reference datasets

Type III kuddel muddel

Type III kuddel muddel

ANOVA F-test results and regulators

ANOVA F-test results and regulators

ANOVA F-test results and regulators

ANOVA F-test results and regulators

Type III again

SAS OT

Reference datasets

Test hypothesis of using Type I/III MS...

R code for lm.mod does not make sense