Helmut ★★★ Vienna, Austria, 2024-04-19 03:26 (235 d 01:55 ago) Posting: # 23958 Views: 3,377 |
|
Dear all, maybe this article is of interest: Helmut Schütz , Divan A Burger , Erik Cobo , David D Dubins , Tibor Farkás, Detlew Labes , Benjamin Lang, Jordi Ocaña, Arne Ring , Anastasia Shitova , Volodymyr Stus, Michael Tomashevskiy. Group-by-Treatment Interaction Effects in Comparative Bioavailability Studies. AAPS J. 2024; 26(3): 50. doi:10.1208/s12248-024-00921-x. Open Access. Supplementary Material. ⅔ of the authors are members of the forum... I’m not sure whether the results of the article will change the respective section of ICH M13A, which in the draft states: Sample size requirements and/or study logistics may necessitate studies to be conducted with groups of subjects. The BE study should be designed to minimise the group effect in the study. The combination of multiple factors may complicate the designation of group. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ Vienna, Austria, 2024-09-20 15:30 (80 d 13:51 ago) @ Helmut Posting: # 24203 Views: 1,810 |
|
Dear all, the FDA was definitely not happy with our article. Why am I not surprised? Sun W , Alosh M, Schuirmann DJ, Grosser S. Letter to the Editor on “Group‑by‑Treatment Interaction Effects in Comparative Bioavailability Studies”. AAPS J. 2024; 26(5): 101. doi:10.1208/s12248-024-00972-0. Disclaimer The views expressed in this article represent the opinions of the authors, and do not represent the views and/or policies of the U.S. Food and Drug Administration. At least. Common procedure, e.g., in ‘Science’, ‘Statistics in Medicine’, ‘Journal of the American Statistical Association’, ‘Biometrics’, ‘British Medical Journals’ (a rant):
Edit: Rejoinder submitted on 6 October. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
BEQool ★ 2024-10-03 17:09 (67 d 12:12 ago) @ Helmut Posting: # 24211 Views: 1,482 |
|
Dear Helmut, thanks for the article. What should be used or what did you use as an error term (in the denominator) when testing group*treatment effect? MS Error (as for within-subject factors) or Subject(group×sequence) Error (as for between-subject factors)? Regards BEQool |
Helmut ★★★ Vienna, Austria, 2024-10-03 17:33 (67 d 11:48 ago) @ BEQool Posting: # 24212 Views: 1,472 |
|
Hi BEQool, ❝ … what did you use as an error term (in the denominator) when testing group*treatment effect? MS Error (as for within-subject factors) or Subject(group×sequence) Error (as for between-subject factors)? Here is the ANOVA of the first simulated study of our Scenario 1:
0.083870 . Did we screw up?— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
BEQool ★ 2024-10-03 18:08 (67 d 11:13 ago) @ Helmut Posting: # 24214 Views: 1,508 |
|
❝ Here is the ANOVA of the first simulated study of our Scenario 1: ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ ❝ From the table above if I am not mistaken I see that sequence effect was tested by using a Residuals (=MSE) as an error term. Shouldnt sequence effect be tested by using subject(sequence) or in this case subject(group*sequence) as the error term because it is a between subject factor? I am wondering if group*treatment is also a between-subject factor and should therefore be tested by using subject(group*sequence) as the error term and not Residuals (similarly as sequence effect)? Because based on this post and your reply (thanks!) sex, sequence, stage, group, site and also sex*treatment are between-subject factors and should therefore be tested by using subject(group*sequence) or subject(sequence) (if we dont have group in the model) as the error term. If sex*treatment is a between-subject factor, I assume group*treatment should also be? ❝ And then … ❝ ❝ … which is Probably not but based on my explanation above shouldnt then F value for group*treatment be 2.14516? So 0.2522449107 divided by 0.1175881872? PS Isn't MS Residuals (=MSE) generally always pretty smaller than MS subject(sequence) or subject(sequence*group) in case of groups? |
Helmut ★★★ Vienna, Austria, 2024-10-04 12:11 (66 d 17:10 ago) @ BEQool Posting: # 24215 Views: 1,448 |
|
Hi BEQool, you have a point!
However, I’m puzzled because then p(G×T) of our meta study are not uniformly distributed any more. Compare them to these plots. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
mittyri ★★ Russia, 2024-10-05 02:08 (66 d 03:13 ago) (edited on 2024-10-05 22:59) @ Helmut Posting: # 24216 Views: 1,417 |
|
Hi Helmut & BEQool, ❝ ❝ ❝ I really doubt that. In the article* referred there's a claim: For the second and third comparisons, models 1 and 2 were used. Sex-by-formulation interactions were expressed by comparing the ratio of the test and reference geometric means for women with that for men. For sex-by-formulation interactions, observed ratio differences of greater than or equal to ±20 percentage points or statistically significant differences at P < .05 were used to identify interactions of interest. It appears that the authors used the Residual Mean Squares (MS) as the denominator for the F-tests, rather than the Subject MS. This choice is not explicitly stated but seems likely based on the overall approach. Therefore, it's reasonable to assume that the interaction term Group*Formulation should also be tested against the Residual MS, not the Subject MS.
— Kind regards, Mittyri |
BEQool ★ 2024-10-06 22:06 (64 d 07:14 ago) @ mittyri Posting: # 24217 Views: 1,363 |
|
Hello Mittyri and Helmut, ❝ It appears that the authors used the Residual Mean Squares (MS) as the denominator for the F-tests, rather than the Subject MS. This choice is not explicitly stated but seems likely based on the overall approach. Therefore, it's reasonable to assume that the interaction term Group*Formulation should also be tested against the Residual MS, not the Subject MS. So you would assume that Residual Mean Squares (MS) as the denominator was used because nothing is explicitly stated? If Subject MS was used instead as the denominator, you think that they would probably mention it? Is there any source or literature what to use as denominator when assessing Group*Treatment interaction? I cant find anything relevant. Nothing is mentioned about it in the new ICH M13A guideline: The group x treatment interaction term should not be included in the model. However, applicants should evaluate potential for heterogeneity of treatment effect across groups and discuss its potential impact on the study data, e.g., by investigation of group x treatment interaction in a supportive analysis and calculation of descriptive statistics by group. I am wondering what to use as a denominator when testing the Group*Treatment interaction and how to support this decision with literature references when writing a report and answering to regulatory agencies. Additionally, what if the significance between the denominator used differ? Regards BEQool |
mittyri ★★ Russia, 2024-10-07 00:00 (64 d 05:21 ago) (edited on 2024-10-07 00:31) @ BEQool Posting: # 24219 Views: 1,351 |
|
Hi BEQool, ❝ So you would assume that Residual Mean Squares (MS) as the denominator was used because nothing is explicitly stated? If Subject MS was used instead as the denominator, you think that they would probably mention it? The regulators are not trying to make our life easier. Yes, the model to be used is rarely presented. There are not so many exceptions (i.e. replicate designs for EMA or FDA). ❝ Is there any source or literature what to use as denominator when assessing Group*Treatment interaction? I cant find anything relevant. Nothing is mentioned about it in the new ICH M13A guideline In any sources with Group*Treatment interaction the denominator is not mentioned directly, but I think that without additional RANDOM statement (or ANOVA postprocessing) which SHOULD be mentioned if in use, the Residual MS is still valid. So you should prove why you modified your ANOVA to get F against Subject MS. The `famous` Sun et al. * paper states: For each dataset, a Linear Mixed Model is fit with subgroup, sequence, subgroup-by-sequence, period nested within subgroup, treatment, treatment-by-subgroup as the fixed model and subject nested within subgroup and sequence as a random effect model (the same model can also be fit using the General Linear Model (GLM) with fixed effects only but with slight modification). ❝ Additionally, what if the significance between the denominator used differ?
PS: Are you equipped with SAS to check the ANOVA tables for this model (using some dataset without dropouts?) It would be interesting to see what PROC MIXED thinks about this interaction. From this paper I assume they should be the same as PROC GLM. But who knows! I could be wrong with my claim... — Kind regards, Mittyri |
BEQool ★ 2024-10-07 12:27 (63 d 16:54 ago) @ mittyri Posting: # 24220 Views: 1,293 |
|
❝ […] So you should prove why you modified your ANOVA to get F against Subject MS. […] I mean if the Group*Treatment is a between-subject factor, it should be, in my opinion, tested by using Subject MS as the denominator.❝ The `famous` Sun et al. paper states: ❝ For each dataset, a Linear Mixed Model is fit with subgroup, sequence, subgroup-by-sequence, period nested within subgroup, treatment, treatment-by-subgroup as the fixed model and subject nested within subgroup and sequence as a random effect model (the same model can also be fit using the General Linear Model (GLM) with fixed effects only but with slight modification). ❝ PS: Are you equipped with SAS to check the ANOVA tables for this model (using some dataset without dropouts?) It would be interesting to see what PROC MIXED thinks about this interaction. From this paper I assume they should be the same as PROC GLM. But who knows! I could be wrong with my claim... Regards BEQool |
mittyri ★★ Russia, 2024-10-07 15:11 (63 d 14:10 ago) @ BEQool Posting: # 24222 Views: 1,291 |
|
Hi BEQool, ❝ ❝ […] So you should prove why you modified your ANOVA to get F against Subject MS. […] ❝ I mean if the Group*Treatment is a between-subject factor, it should be, in my opinion, tested by using Subject MS as the denominator. <IMHO> Subject MS (nested within subgroup and sequence) is typically used to test other between-subject factors, like sequence or subgroup effects, not treatment-related effects. The subject MS usually captures variability between subjects but within their respective subgroups and sequences. Treatment-by-subgroup MS should generally be tested against the residual MS, because this residual error captures the variability within subjects that remains after accounting for treatment and subgroup effects. If the subject MS is used as the denominator, this could lead to inflated F-values because the subject MS reflects between-subject variability, which is not appropriate for testing interactions involving the treatment effect </IMHO> ❝ As already discussed, you think that if they had used MS Subject as the denominator, they would have probably written it here? yes ❝ Which dataset are you talking about? Some random dataset or a specific one? No preferences in exact dataset, but it would be good to test the dataset with unequal groups length ❝ And you are talking about testing Group*Treatment interaction with this dataset with both PROC MIXED and PROC GLM and to see if there is any difference? yes, if possible please — Kind regards, Mittyri |
BEQool ★ 2024-10-14 13:01 (56 d 16:19 ago) @ mittyri Posting: # 24227 Views: 1,037 |
|
Hello Mittyri, thank you for your comment regarding the denominator used for testing Group*Tretament effect, highly appreciated. ❝ ❝ Which dataset are you talking about? Some random dataset or a specific one? ❝ No preferences in exact dataset, but it would be good to test the dataset with unequal groups length ❝ ❝ And you are talking about testing Group*Treatment interaction with this dataset with both PROC MIXED and PROC GLM and to see if there is any difference? ❝ yes, if possible please In SAS I performed ANOVA with dataset with unequal group sizes (and with subjects that completed both periods) with both PROC MIXED and PROC GLM to test Group*Treatment effect and the p-values for all PK parameters do not differ between PROC MIXED and PROC GLM. So you were right BEQool |