Helmut Hero Vienna, Austria, 20130416 17:40 Posting: # 10424 Views: 31,527 

Dear all, new experiences from an ongoing MRP. Two studies (first one in two groups due to logistic reasons), Method C (!), all metrics in both studies passed in stage 1 (>80% power in the interim, no αadjustment = 90% CIs). Studies’ models in stage 1:
“Statistical analysis should be GLM. Please justify. Please provide the NOL.” OK, I will repeat the analyses with all fixed effects (sigh; identical CIs…).BTW, does anybody know what “NOL” means? P.S.: All metrics would have passed with α_{adj} 0.0294 (Method B) as well. I did not report these results and — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro Hero Denmark, 20130417 02:31 @ Helmut Posting: # 10425 Views: 29,377 

Hi Hötzi, » new experiences from an ongoing MRP. Two studies (first one in two groups due to logistic reasons), Method C (!), all metrics in both studies passed in stage 1 (>80% power in the interim, no αadjustment = 90% CIs). Ahhh... MRP  a relic from the bygone ages when your grandad walked around in the snowy alps with bow and arrows looking for deer. » Studies’ models in stage 1:
» OK, I will repeat the analyses with all fixed effects (sigh; identical CIs…). Anything to please the assessor. I have a feeling the assessor means LM but might not know what that really means since every LM in SAS is done via PROC GLM. But honestly, group x treatment?? I'd rather not go down that road. I hope you do not make precedence. » BTW, does anybody know what “NOL” means? I am fairly sure it is either "No Objection Lertificate" or "Nixon's Obstructive Leprosy". I don't see what else it could possibly be. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
Helmut Hero Vienna, Austria, 20130417 12:23 @ ElMaestro Posting: # 10426 Views: 29,330 

My Capt’n! » MRP  a relic from the bygone ages when your grandad walked around in the snowy alps with bow and arrows looking for deer. » Anything to please the assessor. I have a feeling the assessor means LM but might not know what that really means since every LM in SAS is done via PROC GLM. Yep. » But honestly, group x treatment?? I'd rather not go down that road. I hope you do not make precedence. Not my invention. See this post for the reference. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20130417 13:10 @ Helmut Posting: # 10427 Views: 29,605 

Gents! » » But honestly, group x treatment?? I'd rather not go down that road. I hope you do not make precedence. » » Not my invention. See this post for the reference. I have another 'better' one (from a Letter of FDA, Barbara M. Davit)! " ... the following statistical model should be used when analyzing data from a multiple group bioequivalence study:
Subject(nested within Group*Sequence) is a random effect and all other factors are fixed effects. ..." Enormous effect(s) . BTW: Helmut, how had you planned the evaluation of the second stage, if it would be necessary? BTW2: In context of a 2Stage design, also a 2group design with groups here called stage, the mighty oracle prohibit us the GroupbyTreatment Interaction! From the EMA Q&A Rev.7 : "Conclusion ... A term for a formulation*stage interaction should not be fitted." BTW3: NOL = no objection letter ? — Regards, Detlew 
Helmut Hero Vienna, Austria, 20130417 16:37 @ d_labes Posting: # 10428 Views: 30,693 

Hi Detlew! » I have another 'better' one (from a Letter of FDA, Barbara M. Davit)! » […] » Group » Sequence » Treatment » Subject(nested within Group*Sequence) » Period(nested within Group) » GroupbySequence Interaction (sic!) » GroupbyTreatment Interaction » Subject(nested within Group*Sequence) is a random effect and all other factors are fixed effects. ..." » » Enormous effect(s) . Wow, this will send ElMaestro’s SillyoMeter into nirvana! I have seen some FDA code a good while ago: PROC GLM DATA = DATASET; Followed by: If the Groupbytreatment interaction test is not statistically significant (p ≥0.1), only the Groupbytreatment term can be dropped from the model. EMA & random effects… Phoenix gives me headaches if all effects are fixed (remember this thread?)… » BTW: Helmut, how had you planned the evaluation of the second stage, if it would be necessary? Probably too naïvely: fixed: sequence, treatment, stage, period, group, » BTW2: In context of a 2Stage design, also a 2group design with groups here called stage, the mighty oracle prohibit us the GroupbyTreatment Interaction! From the EMA Q&A Rev.7 : » "Conclusion ... A term for a formulation*stage interaction should not be fitted." Ahja! » BTW3: NOL = no objection letter ? Good idea. The reports contained a copy of the IECapproval and a section “Notification to Health Authorities”: All approvals needed for conducting the trial were obtained; the trial was performed in accordance with the German Medicines Act (AMG). The study was approved by the BfArM on DD Mon YYYY. The study notification to the BfArM [§ 42 (2) AMG,1 § 9 (1) GCPV] was performed by ███ on DD Mon YYY. Local authorities were notified according to § 67 AMG (‘█████’ for the clinical site ███ and ‘█████’ for the sponsor). On DD Mon YYYY end of the trial was reported by ███ to the BfArM. Will submit a copy of BfArM’s approvals (protocol, study notification).— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20130418 09:55 @ Helmut Posting: # 10431 Views: 29,338 

Dear Helmut! » I have seen some FDA code a good while ago: » PROC GLM DATA = DATASET; » CLASS SUB TRT PER SEQ GRP; » MODEL Y = GRP SEQ SEQ*GRP SUBJ(SEQ*GRP) PER(GRP) TRT TRT*GRP /SS3; » RANDOM SUBJ(SEQ*GRP); » TEST H = SEQ GRP E = SUBJ(SEQ*GRP)/HTYPE=3 ETYPE=3; » ESTIMATE 'T VS R' TRT 1 1; » LSMEANS TRT/STDERR PDIFF CL ALPHA=.1; » RUN; Yep. Implementation of what is written in the letter. But IMHO the RANDOM statement is superfluous since it doesn't do anything other than printing a symbolic representation of expected mean squares. If one is really interested in having a random effect for subj(seq*group) Proc MIXED should be used.» Followed by: » ... Same wording in the letter I have. But the most interesting part of my letter: "If all of the following criteria are met, it may not be necessary to test for group effects in the model:
Can't imagine BE studies where this is not true, except multicenter studies, but these are rare for BE evaluation. So what? Group effects are a Chimera? » » BTW: Helmut, how had you planned the evaluation of the second stage, if it would be necessary? » » Probably too naïvely: » fixed: sequence, treatment, stage, period, group, » group × treatment, stage × treatment » random: subject(sequence × stage) Too naïvely may be true . — Regards, Detlew 
Helmut Hero Vienna, Austria, 20130419 14:34 @ d_labes Posting: # 10438 Views: 29,475 

Dear Detlew! » But the most interesting part of my letter: » "If all of the following criteria are met, it may not be necessary to test for group effects in the model: » » • the clinical study takes place at one site; » • all study subjects have been recruited from the same enrollment pool; » • all of the subjects have similar demographics; and » • all enrolled subjects are randomly assigned to treatment groups at study outset » » In this latter case, the appropriate statistical model would include only the factors Sequence, Period, Treatment and Subject(nested within sequence." » » Can't imagine BE studies where this is not true, … Interestingly FDA’s Stat Guidance (Section VII.A.) states If a crossover study is carried out in two or more groups of subjects […], the statistical model should be modified to reflect the multigroup nature of the study. In particular, the model should reflect the fact that the periods for the first group are different from the periods for the second group. This applies to all of the approaches (average, population, and individual BE) described in this guidance. (my emphases)» … except multicenter studies, but these are rare for BE evaluation. Yep. » So what? Group effects are a Chimera? Probably. What about EMA’s BE GL (Section 4.1.8)? The statistical analysis should take into account sources of variation that can be reasonably [sic] assumed to have an effect on the response variable. Group = not a reasonable effect?» Too naïvely may be true . Alter schützt vor Torheit nicht. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
mittyri Senior Russia, 20140821 15:34 @ Helmut Posting: # 13416 Views: 27,130 

Dear Helmut, » Group = not a reasonable effect? As I understand correct, no one can answer on this question... I've read some topics about group effect in the statistical model, by the way I still have some questions 1. Is it obligatory EMA's requirement to use Group effect in the model in all cases when there is more than one group? Any experience of LoDs due to absence of Group effect in the statistical model? 2. What kind of statistical model is preffered for Europe submission? According to the Detlew's post » In context of a 2Stage design, also a 2group design with groups here called stage, the mighty oracle prohibit us the GroupbyTreatment Interaction! From the EMA Q&A Rev.7 : » "Conclusion ... A term for a formulation*stage interaction should not be fitted." ElMaestro wrotes that » Although I think the formulation x group interaction would steal a df and would be tested against the model residual (within). You are right that any step towards increasing the model residual translates into producer's risk. Did you ever use the model without group × treatment? Something like that: Fixed: sequence, period, treatment, group, subject(sequence) 3. Did you ever got the requests from EMA experts about calculating the CI into one of the groups as for FDA submision? Sorry for probable repeating some questions Edit: I’m off for a three weeks vacation. I’m sure others will have a lot to say in the meantime. [Helmut] — Kind regards, Mittyri 
ElMaestro Hero Denmark, 20140821 17:31 @ mittyri Posting: # 13417 Views: 27,125 

Hello Mittyri, » 1. Is it obligatory EMA's requirement to use Group effect in the model in all cases when there is more than one group? Any experience of LoDs due to absence of Group effect in the statistical model? Just include the term.... that can't be bad in a regulatory sense and it can't be a burden to you, right? » 2. What kind of statistical model is preffered for Europe .. » Something like that: » Fixed: sequence, period, treatment, group, subject(sequence) Sounds right to me, and I've had thumbs up in sc.advices. » 3. Did you ever got the requests from EMA experts about calculating the CI into one of the groups as for FDA submision? Not sure what you are actually asking. I have never heard a EU regulator ask for something the FDA want just because of FDA's request in itself. Hötzi: Enjoy your holidays. I am confident there will be good news in your mehlbox when you return — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
mittyri Senior Russia, 20141001 11:25 @ ElMaestro Posting: # 13629 Views: 26,577 

Hello ElMaestro and all, Thank you for clarification! 1. » Just include the term.... that can't be bad in a regulatory sense and it can't be a burden to you, right? Including the group effect to the statistical model leads to widening the CI. More effects in model mean wider intervals. Please correct me if I'm wrong. — Kind regards, Mittyri 
ElMaestro Hero Denmark, 20141001 11:31 @ mittyri Posting: # 13630 Views: 26,518 

Hi Mittyri, » More effects in model mean wider intervals. » Please correct me if I'm wrong. That's kind of incorrect. The more effects you have in your model, the more variation is accounted for by the totality of terms; the residual will decrease as will be residual's degrees of freedom. Therefore the width of the CI tends to shrink. It may happen that the inclusion of a term apparently does nothing. This is due to the design where some effect may contain the other; in such situation the residual variance, the residual df and the width of the CI remain unaffected. But you will never see the CI widened due to more factors. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
Helmut Hero Vienna, Austria, 20141001 13:24 @ mittyri Posting: # 13632 Views: 26,515 

Hi Mittyri, I agree with our Master. Simple example: group subject period sequence treatment response
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
VStus Regular Poland, 20161006 22:06 @ Helmut Posting: # 16701 Views: 23,283 

Hi Helmut, I have performed exercise on this dataset in R and have to support Mittyri: addition of more effects beyond group increases MSE and, therefore, CI:
Due to my limited R knowledge, I cannot figure out how to add other effects recommended by FDA e.g. subject(sequence x group) , but I'm afraid that at least in R, the more related effects  the wider confidence intervals...Regards, VStus 
ElMaestro Hero Denmark, 20161007 12:45 @ VStus Posting: # 16705 Views: 23,213 

Hi VStus, » I have performed exercise on this dataset in R and have to support Mittyri: addition of more effects beyond group increases MSE and, therefore, CI: (...) Hehe, this is a truly remarkable phenomenon. Inclusion of more factors or more crosses/nestings introduces new columns in the muddle matrix, so the residual variability measured as SS decreases. Nevertheless, the residual variance, MSE or SS divided by DF, may go up as the new columns in the muddle matrix steals a bit from the DF pool. You should see that if you do an anova on your model object. Quite possibly, you are right in theory the width of the CI could in such cases increase? I did not recalculate it in this case, perhaps that is really possible?!? I doubt it would happen with biological data, but I see now I should not have stated it like I did in the post above. I apologise for any confusion my post led to. » Due to my limited R knowledge, I cannot figure out how to add other effects recommended by FDA e.g. subject(sequence x group) , but I'm afraid that at least in R, the more related effects  the wider confidence intervals...You are ok as long as you code subjects uniquely, then it does not matter if you call them Subject or Subject in Sequence in Group etc. Search this forum for the famous Sillyometer and it will be crystal clear, of course. Not?!? A good weekend to all of you. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
mittyri Senior Russia, 20161007 15:06 (edited by mittyri on 20161007 15:32) @ ElMaestro Posting: # 16706 Views: 23,390 

Hi VStus & ElMaestro! This is really funny, we had 2 last days discussions with Helmut regarding Group effect during the workshop Yes, in case of just adding a Group you do not see any changes (Fixed: sequence, period, treatment, group, subject(sequence)) But in case of Period %in% Group the situation is not the same!! a little code for discussion: # the original code is from this perfect paper: > Analyse222BE(data, alpha=0.05, Group = FALSE) Even PE differs! What the heck?? Where am I wrong? I need to take some Schützomycin! PS: I see now, the model with Period:Group is wrong — Kind regards, Mittyri 
mittyri Senior Russia, 20161009 13:27 @ mittyri Posting: # 16708 Views: 23,029 

Dear All, Now I'm using the model from 2stage design (without Period term): muddle < lm(log(Var)~Group+Seq+Seq*Group+Subj%in%(Seq*Group)+Per%in%Group+Trt, data=data) BTW the question is still there, because the output is the same as above (PE and MSE are different) Also please note that the Group factor is extremely significant When I run this dataset for FDA Model I in Phoenix (don't know the code for R):
Dependent Hypothesis Numer_DF Denom_DF F_stat P_value The Group factor is faraway from significant level. It could be a nightmare for Russia, where the experts insist on the Group term in the model, but all other things should be like in EMA Guideline with all factors as fixed (does someone realize how many studies will be failed? The experts suggest not to pool the groups in this case like the FDA Guidance states!) — Kind regards, Mittyri 
mittyri Senior Russia, 20161010 18:33 @ mittyri Posting: # 16710 Views: 23,103 

Dear All, I cannot change the previous post, so please find my corrected code I've found my mistake regarding Fs As ElMaestro suggested, all beweensubject factors should be assessed vs. MSE_{subject}: # http://link.springer.com/article/10.1208/s1224801496610 Now Group factor seems to be more suitable. What bothers me is that Group effect is not equal in R and PHX, cannot figure out the reason — Kind regards, Mittyri 
ElMaestro Hero Denmark, 20161010 20:10 @ mittyri Posting: # 16711 Views: 22,895 

Hi there, » What bothers me is that Group effect is not equal in R and PHX, cannot figure out the reason Type III SS for Group may be somewhat challenging because is a betweenfactor like Sequence. I am OoO so cannot tests it well, but I reckon you'd need to play around with models without subject and sequence, in order to get something for Group. This post is untested and just reflecting my thoughts. It may not in anyway reflect realities at all (like most of my post in general). Actually if some of the R experts out there see this post, then I am hereby suggesting you to write a general extension to drop1 which produces meaningful (that means informative, if anyone wonders) SS's and Ftests for models where there are multiple betweens. The ideal candidate for this drop1 extension would purely hypothetically be a statistical expert with mahatma status who has plenty spare time because he recently went into retirement. But of course, such people are hard to find. Hötzi, do you think any such person exists on the planet or is it just wishful thinking?!?? — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
Helmut Hero Vienna, Austria, 20161011 00:17 @ ElMaestro Posting: # 16712 Views: 22,895 

Hi ElMaestro, » I am OoO so cannot tests it well, … So am I. » … a statistical expert with mahatma status who has plenty spare time because he recently went into retirement. And is away from home for the next two weeks enjoying delicious “Flens”. » Hötzi, do you think any such person exists on the planet or is it just wishful thinking?!?? Sounds like a Radio Yerewan question to me. A: “In principle yes, but…” — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20161011 13:17 (edited by d_labes on 20161011 13:30) @ ElMaestro Posting: # 16722 Views: 22,808 

Gentlemen (& Ladies), » The ideal candidate for this drop1 extension would purely hypothetically be a statistical expert with mahatma status who has plenty spare time because he recently went into retirement. But of course, such people are hard to find. » » Hötzi, do you think any such person exists on the planet or is it just wishful thinking?!?? I think a person with such skills couldn't be found within the whole galaxy. May be far, far away in the outer space . If I would be such a person I nevertheless would by no means delve into the mud of Type I, II, III or even type IV tests in ANOVA . There is some sort of Holy War out there with regard to such tests when it comes to unbalanced designs. This war is raging especially between the R folks and SAS adepts, the latter claimed to have invented "Type III" as I was told. Therefore it has to be seen as evil and must be abandoned by the R hordes under the virtual leadership of Douglas Bates. No person with a lucid mind or even with an expert or mahatma status would participate in such a hassle. Not for money (even much!) nor for good words. Especially if considering that these FTests and pvalue Kuddelmuddel are nothing good for within the context of BE, only being a description of nuisance terms in the ANOVA model. Better drink a good beer or a good glas of vine while contemplating about the meaning of life and that all. — Regards, Detlew 
mittyri Senior Russia, 20161011 15:07 (edited by mittyri on 20161011 17:02) @ d_labes Posting: # 16724 Views: 22,814 

Dear Detlew, Thank you very much, it clarifies a lot the R's model logic. I would explain the current situation in Russia and why we are so interested in ANOVA type III. Russian experts read your post with a letter from Barbara Davit. Now in ALL cases when the Protocol does not state the splitting to the groups, they are requesting the analysis with a group term, if the study was conducted in groups (irrespective of the reasons and time). As you know there is an algo in FDA guidance, if the Group term is significant, one cannot pool them and should proceed an analysis in one of the groups. (almost) End of Story. Many protocols state that all effects should be fixed (according to EMA GL)! So I tried to modify your code to get the significance level of group effect and I've found that I cannot reproduce ANOVA type III PHX results in R (using FDA approach and allfixedeffects approach). » Better drink a good beer or a good glas of vine while contemplating about the meaning of life and that all. Agree — Kind regards, Mittyri 
zizou Regular Plzeň, Czech Republic, 20161011 01:06 @ mittyri Posting: # 16713 Views: 22,953 

Dear mittyri, for the results consistency (R versus Phoenix as you reported), there should be one more term of marvelous effect x) ... you posted it as number 7: (I resorted it for comparison of F and p values, note: int and Trt are missing in the results from R code)
» Dependent Hypothesis Numer_DF Denom_DF F_stat P_value If you change the muddle: muddle < lm(log(Var)~Group+Seq+Seq*Group+Subj%in%(Seq)+Per%in%Group+Trt+Group*Trt, data=data) and run the mentioned code... Voilà! Type III sum of squares: The comparison is telling us that with groups it will be little bit more complicated. » Now Group factor seems to be more suitable. » What bothers me is that Group effect is not equal in R and PHX, cannot figure out the reason Good luck! zizou 
mittyri Senior Russia, 20161011 13:43 (edited by mittyri on 20161011 16:53) @ zizou Posting: # 16723 Views: 22,969 

Dear zizou, Thank you for suggestion! The FDA group model for PHX is described by Helmut here: fixed: Group+Sequence+Sequence(Group)+Period(Group)+Treatment+Treatment×Group I think in this case PHX easily switches from GLM to MIXED procedure and does not produce SS tables. We all know that the GLM procedure should be used for conventional FDA model. Could we use the same procedure (GLM) here? The relationship between GLM/MIXED is described very well in 'mustread' post by ElMaestro I executed the code with unequal sequences (removing Subj18 from the reference dataset): PHX Sequential tests: Dependent Hypothesis Numer_DF Denom_DF F_stat P_value PHX Partial tests: Dependent Hypothesis Numer_DF Denom_DF F_stat P_value R: muddle < lm(log(Var)~Group+Seq+Seq*Group+Subj%in%(Seq)+Per%in%Group+Trt+Group*Trt, data=data) Df Sum Sq Mean Sq F value Pr(>F) drop1(muddle,test="F")
Single term deletions and CI's are far from equality... PS: I tried to switch to the mixed model library(nlme) The ANOVA type I is the same as in PHX, but PE and CIs are not. I'm in stuck.... — Kind regards, Mittyri 
ElMaestro Hero Denmark, 20161012 01:07 @ mittyri Posting: # 16725 Views: 22,769 

Hi all, be careful: ?lm "The terms in the formula will be reordered so that main effects come first, followed by the interactions, all secondorder, all thirdorder and so on". So don't compare directly with any sequential anova from software that doesn't do exactly that. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
VStus Regular Poland, 20161017 15:32 @ mittyri Posting: # 16730 Views: 22,488 

Dear mittyri, Dear zizou, » The ANOVA type I is the same as in PHX, but PE and CIs are not. I'm in stuck.... Thank you very much for your help! I have checked proposed model formula in R's lm() versus SAS output of some old study report with unbalanced dataset. I found perfect much here for group*treatment! 1. Analysis started as FDA Group Model 1: > anovadata < lm(log(data2$Cmax)~group+seq+seq*group+subj%in%(seq)+prd%in%group+drug+drug*group, data=data2, na.action=na.exclude) F and P values for group:drug are exactly the same as obtained by SAS:
Param Source DF F Value Pvalue Model to be used: 2. I've got same F and P values (at least for some of the effects) while proceeding to FDA Group Model 2 (or EMA's only model with all effects fixed), not mentioning exactly the same outputs for MSE/ Intrasubject CV and PE / 90%CIs : > anovadata2 < lm(log(data2$Cmax)~group+seq+seq*group+subj%in%(seq)+prd%in%group+drug, data=data2, na.action=na.exclude) SAS output:
Param Effect DF F Value Pvalue Maybe lm() sometimes doing exactly the same as PROC GLM? Regards, VStus 
ElMaestro Hero Denmark, 20161017 18:58 @ VStus Posting: # 16732 Views: 22,531 

Hi VStus, » Maybe lm() sometimes doing exactly the same as PROC GLM? I actually lost a bit track of this discussion and I am not exactly sure what the comparisons try to achieve?!? However, lm() and PROC GLM are related but not the same and they do not necessarily achieve exactly the same result. I am very sure that their behavour corresponds to the specification, and I have good reason to think their output is correct cf. those specifications. I do not know how SAS handles crosses and interactions, but I know that anova on lm() in R puts main effects first. Drop1 is a good example of a source of differences in terms of results. Drop1 and type III is implemented differently in R and SAS. You can argue both are right, certainly, even though they are not identical results. Drop1 in R involves single term deletions which is how type III is defined, but that may give a zero SS (or SS decrease) for sequence. In SAS type III involves deletion of more than just single terms for betweenfactors in our BE studies so that sequence is not a zero SS effect. So, before you start comparing lm() (with or without anova or drop1) and SAS, perhaps you can formulate with your own words what you want calculated and how, when there is room for interpretation such as is the case with type I/III and lm()/drop1 versus PROC GLM. — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
VStus Regular Poland, 20161020 13:57 @ ElMaestro Posting: # 16745 Views: 22,295 

Dear Maestro, » I actually lost a bit track of this discussion and I am not exactly sure what the comparisons try to achieve?!? I'm sorry for confusion. I accept that I for sure may understood this topic completely wrong. I just wanted to address Myttiry's issue of nonsimilar results between R's lm() and WiNonlin in assessment of the FDA group model 1 and 2 for unbalanced dataset. I don't have access to SAS or WiNonlin, but I have some study reports assessed via SAS using FDA's group model 1 and 2. So, in regards to Point Estimates, 90% CIs, F and P values for: FDA Group 1 Group*Treat #Treat:Group in my R output FDA Group 2  Treat #Drug in my R output  Per(Group) # group:prd in my R output  Subj(Group*Seq) # seq:subj in R lm() provides same results as PROC GLM If Russian Authorities require an additional assessment using a group model, R output should be ok. So as PHX, if PE and 90%CI are not calculated using FDA Group Model 1. Regards, VStus 
VStus Regular Poland, 20161020 13:54 @ mittyri Posting: # 16744 Views: 22,328 

Dear mittyri, Getting a little back to possible nightmare for BE studies assessed in Russia: » It could be a nightmare for Russia, where the experts insist on the Group term in the model, but all other things should be like in EMA Guideline with all factors as fixed (does someone realize how many studies will be failed? The experts suggest not to pool the groups in this case like the FDA Guidance states!) Why do you proceed to calculation of Point Estimate and Confidence Intervals using FDA Group Model 1 (those with Treatment*Group interaction) if it's not seems to be intended for anything more than assessment of statistical significance of this particular effect? I've just gone through some historical studies (with groups) done at wellestablished CROs before EMA Q&A document regarding statistical model for TSDs became available. They assessed studies using FDA recommendations. Model 1 was used only to calculate significance of Treatment*Group interaction and nothing more! No MSE, PE and 90%CI printouts for this model. After confirmation that effect is not significant, data was processed under FDA Group Model 2 with bioequivalence assessment. Why not use the same strategy here? F and P seems to be equal between R and PNX, as well as between R and SAS PROC GLM. PS: FDA Group Model 1 produces weird results in regards to Point Estimate (if we get it directly from ANOVA table), potentially making a good trial useless. In one of examples PE traveled from 96% to 88% (sic!) for data where geometric means ratio was almost equal to LSMeans ratio. Regards, VStus 
zizou Regular Plzeň, Czech Republic, 20161231 02:54 @ ElMaestro Posting: # 16917 Views: 21,294 

Hi everybody and nobody! I am pointing to the 2x2 crossover studies in groups due to logistic reasons (capacity of clinical unit, etc.). Many discussions about groups were done here but recently I was asked to calculate effect of groups in one failed study (study acc. to EMA guidelines, in protocol/report without group effect testing  two groups within the week). (But no problem when "everything possible" is analysed in a failed study with getting more informations for sponsor's next decision.) Nevertheless there is nothing stated from EMA on testing of such groups (or did I miss something?). For FDA there is CDER or known letter mentioned many times dealing with groups*treatments. So everyone can follow it to analyse the model for FDA. For EMA groups are not intended to testing. ? I have opinion that group effect is not important in 2x2 crossover with proper planning/realization. (Moreover I have never seen group testing for EMA and I like the sentence "With one week between groups I would never ever thought a millisecond of setting up a group model." from this post.) Following model (from the post I replay to) sounds right but only sounds or is? » » 2. What kind of statistical model is preffered for Europe .. » » Something like that: » » Fixed: sequence, period, treatment, group, subject(sequence) » » Sounds right to me, and I've had thumbs up in sc.advices. Actually groups and sequences are nested in Subjects so there is no difference in results of BE from the standard model without groups (difference can be only in sequence effect and of course "in new" effects Groups and in Subjects(Groups*Sequences), so in betweensubject variability also). Moreover optional statement as: "If Groups differ significantly, they can't be pooled and BE must be calculated in one group only." smells fishy to me (I heard this bad idea somewhere, maybe misunderstanding due to inaccuracy). Group effect in such model has one property similar as period effect (not only that sometimes it happens significant). Group effect as well as Period effect is not able affect BE decision (in this simple model). You know if you calculate data from one period by 1000 nothing change in BE decision, only period significance (+intercept+total) is different. No change at Formulations, Subjects, Error. Not problem unless the study is extremely imbalanced. In the same way Group effect in such defined model is not able affect BE decision  if you multiply data from one group by 1000, there is no change in Formulations individual ratios, at ANOVA effects Formulations, Periods, Sequence and MSE. The difference is only in ANOVA group effect (will be significant) + Subjects (due to group is nested within the subjects). I checked that property in the next example. (The same size of groups and no dropouts is not required for this property I think, even though in my testing example below it is so. Multiplication by 1000 will be always used for same number of T and R values even in case of dropouts or whatever  sure imbalancing would change sequence effect, but not BE decision I expect.) I prepared example (no real data ... to get "unlucky" results). data=read.delim("http://tjprazdroj.lesyco.cz/download/groups.txt") Firstly I calculated classical GLM model without groups. (according to EMA) Effects = Formulations, Periods, Sequences, Subjects(Sequences). Results: CVintra = 27.4% PE (LL,UL) = 1.0127 (0.9236,1.1103) Great results in 80125%, I can congratulate myself! x) No change after adding effect Groups to the model: Effects = Formulations, Periods, Sequences, Groups, Subjects(Sequences*Groups). Of course everybody and nobody get the same results as above  due to Sequences and Groups are nested in Subjects, i.e. only effects Formulations, Periods, Subjects (as sum of betweensubjects effects) take role for intrasubject CV, GMR of T/R and CI. The effect Groups seems to be not significant as per ANOVA table (p=0.1230537). So it sounds good, everyone can write statement about Groups are not significant and data could be pooled (when using this simple model with Groups). The Phantom Menace for this example is to use another model. (according to FDA) Effects = Formulations, Periods(Groups), Sequences, Sequences*Groups, Subjects(Sequences*Groups), Groups, Formulations*Groups. (GLM with random statement for Subjects was used) Results: CVintra = 25.7% PE (LL,UL) = 1.0127 (0.9284,1.1045) But with significant Formulations*Groups interaction (p=0.008066805 < 0.1). There is recommendation to not to pool the data and calculate BE separately in one of the groups (if this model planned for the decision of pool or not to pool). Because the data are written by me randomly and then changing and changing to get the nice and not nice results at the same. The groups separately provide us with next BE results: Group 1: data1=subset(data,data[,"Groups"]==1) CVintra = 30.9% PE (LL,UL) = 0.8774 (0.7554,1.0191) Group 2: data2=subset(data,data[,"Groups"]==2) CVintra = 19.4% PE (LL,UL) = 1.1689 (1.0626,1.2858) I only want to make more clear and more visibile what can be hidden behind the "no significant group effect". If group testing was required by EMA I would prefer test for group*treatment interaction. But I am still convinced that group effect is not required to test in standard 2x2 crossover study. For me it seems similar to have 2x2 crossover study on 24 subjects in 1 group and test for Period*Treatment interaction (e.g. in model Period*Treatment, Sequence, Subjects(Sequence)  if Period*Treatment significant, use data from one period as in parallel design. It will be certainly significant if ratio T/R (i.e. ratio with means of different subjects) in period 1 will be around 1.25 and T/R in period 2 will be 0.80 (not probable to have opposite (reciprocal) ratio, but periods are often more separated in time than groups and who knows after several months in case of long washout). Best regards, zizou Note: All calculations were done using GLM (in R and/or SPSS). 
Helmut Hero Vienna, Austria, 20170603 14:57 @ zizou Posting: # 17442 Views: 18,117 

Hi zizou, sorry for reviving this old thread. See also this lengthy one. » […] there is nothing stated from EMA on testing of such groups (or did I miss something?). I don’t think so. I know only one case (biosimilar in 2016) where the statistician (“borrowed” from the MHRA) required the FDA’s model 2. » I have opinion that group effect is not important in 2x2 crossover with proper planning/realization. Agree – especially with this one. » Moreover I have never seen group testing for EMA and I like the sentence "With one week between groups I would never ever thought a millisecond of setting up a group model." from this post. Agree. » I prepared example (no real data ... to get "unlucky" results). THX! I could reproduce your results with my Rcode. » Because the data are written by me randomly and then changing and changing to get the nice and not nice results at the same. The groups separately provide us with next BE results: » » Group 1: » data1=subset(data,data[,"Groups"]==1) » CVintra = 30.9% » PE (LL,UL) = 0.8774 (0.7554,1.0191) » » Group 2: » data2=subset(data,data[,"Groups"]==2) » CVintra = 19.4% » PE (LL,UL) = 1.1689 (1.0626,1.2858) Nice example. My current thinking is: The decision scheme recommended by the FDA leads to nowhere. If there is a significant G×T interaction we are not allowed to pool. So far so good. If groups are not of the same size, everybody would present the results of the largest group evaluated by the conventional model. But: What if groups have equal sizes (like in your example)? Cherrypick and present the “better” one? I bet the assessor would ask for the other one as well. If I would be a regulator I would require that both pass or – if not – make a conservative decision: fail. Furthermore, if there is a true G×T interaction the treatment effect might be biased. To which extent is unknown (cannot be estimated from the data). It might well be that a smaller group is closer to the true treatment effect than a larger one. We simply don’t know. IMHO, the assumption that “size matters” is false. Your example is even more telling in another respect. Not only the PEs are different but also the variances. Should we blindly pool them? I know, the common model assumes equal variances anyhow, but… I have just limited data (always tried to avoid equal group sizes). These are the results of my data sets which show a significant G×T interaction in model 1 and I evaluated the equally sized largest groups by model 3:
» If group testing was required by EMA I would prefer test for group*treatment interaction. Disagree. What if you find one? Expected in 10% of studies by pure chance. Then you end up with the story from above. » But I am still convinced that group effect is not required to test in standard 2x2 crossover study. For me it seems similar to have 2x2 crossover study on 24 subjects in 1 group and test for Period*Treatment interaction (e.g. in model Period*Treatment, Sequence, Subjects(Sequence)  if Period*Treatment significant, use data from one period as in parallel design. Agree. Grizzle’s flawed testing for a sequence (or better: unequal carryover) effect… Cannot be handled statistically and only avoided by design. My current thinking for the EMA (maybe I’m wrong): Avoid a potential G×T interaction by design (i.e., comply with the FDA’s conditions for pooling, keep the interval between groups short, if possible use the staggered approach).
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
hiren379 Regular India, 20130822 14:32 @ Helmut Posting: # 11336 Views: 28,372 

Hi HS » Followed by: » If the Groupbytreatment interaction test is not statistically significant (p ≥0.1), only the Groupbytreatment term can be dropped from the model. » … provided that the group meets minimum requirements for a complete bioequivalence study. According to the above statement, what should be the minimum requirements of the group for a complete bioequivalence study? 
Helmut Hero Vienna, Austria, 20130823 11:53 @ hiren379 Posting: # 11338 Views: 28,414 

Hi Hiren, » » … provided that the group meets minimum requirements for a complete bioequivalence study. » » According to the above statement, what should be the minimum requirements of the group for a complete bioequivalence study? Please do your homework first. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 