Bioequivalence and Bioavailability Forum

Helmut
★★★

Vienna, Austria,
2024-04-19 03:26
(235 d 00:15 ago)

Posting: # 23958
Views: 3,375

Group-by-Treatment Interaction: the final word? [General Statistics]

Dear all,

maybe this article is of interest:

Helmut Schütz , Divan A Burger , Erik Cobo , David D Dubins , Tibor Farkás, Detlew Labes , Benjamin Lang, Jordi Ocaña, Arne Ring , Anastasia Shitova , Volodymyr Stus, Michael Tomashevskiy. Group-by-Treatment Interaction Effects in Comparative Bioavailability Studies. AAPS J. 2024; 26(3): 50. doi:10.1208/s12248-024-00921-x. [image] Open Access. [image] Supplementary Material.

⅔ of the authors are members of the forum... :-D

I’m not sure whether the results of the article will change the respective section of ICH M13A, which in the draft states:

Sample size requirements and/or study logistics may necessitate studies to be conducted with groups of subjects. The BE study should be designed to minimise the group effect in the study. The combination of multiple factors may complicate the designation of group.
BE should be determined based on the overall treatment effect in the whole study population. In general, the assessment of BE in the whole study population should be done without including the Group by Treatment interaction term in the model, but applicants may also use other pre-specified models, as appropriate. However, the appropriateness of the statistical model should be evaluated to account for the multi-group nature of the BE study. Applicants should evaluate potential for heterogeneity of treatment effect across groups, i.e., Group by Treatment interaction. If the Group by Treatment interaction is significant, this should be reported and the root cause of the Group by Treatment interaction should be investigated to the extent possible. Substantial differences in the treatment effect for PK parameters across groups should be evaluated. Further analysis and interpretation may be warranted in case heterogeneity across groups is observed.
(my emphasis)

If not, at least you can use the article in an argument with regulators… See also the meta-study in this article, which I regularly update with new data.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Helmut
★★★

Vienna, Austria,
2024-09-20 15:30
(80 d 12:10 ago)

@ Helmut
Posting: # 24203
Views: 1,808

Group-by-Treatment Interaction: FDA

Post reply

Dear all,

the FDA was definitely not happy with our article. Why am I not surprised?

Sun W , Alosh M, Schuirmann DJ, Grosser S. Letter to the Editor on “Group‑by‑Treatment Interaction Effects in Comparative Bioavailability Studies”. AAPS J. 2024; 26(5): 101. doi:10.1208/s12248-024-00972-0.

Disclaimer The views expressed in this article represent the opinions of the authors, and do not represent the views and/or policies of the U.S. Food and Drug Administration.

At least.

Common procedure, e.g., in ‘Science’, ‘Statistics in Medicine’, ‘Journal of the American Statistical Association’, ‘Biometrics’, ‘British Medical Journals’ (a rant):

An article is published.
Some don’t agree, draft a ‘Letter to the Editor’, and submit it for review.
If the editor deems it worthwhile, reviewers will be invited.
If it is accepted, the authors of the publication are notified, provided with the accepted Letter, and given the opportunity to write a Rejoinder.
The Letter is published together with the Rejoinder (not earlier).
In some journals, both have even the same DOI so that interested readers are aware of both.

The letter was submitted on May 31^st (#2). The editor invited me to review it on June 11^th (#3) and notified me six (‼) minutes later that “we have already received sufficient commitments to review this manuscript”. #4 and #5 not followed. Was I notified that the letter was published on 9 September? Nope. I checked the website of the journal regulary…
Is the letter Open Access like our article? Nope. The can’t afford that. Buy it for $ 39.95 or ask Wanjie for a copy.

Since Rejoinders are not possible in the AAPS J, we will submit a ‘Letter’ responding to a ’Letter’… :-D

Edit: Rejoinder submitted on 6 October.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool ★ 2024-10-03 17:09 (67 d 10:32 ago) @ Helmut Posting: # 24211 Views: 1,480	Group-by-Treatment Interaction: the final word? Post reply
	Dear Helmut, thanks for the article. What should be used or what did you use as an error term (in the denominator) when testing grouptreatment effect? MS Error* (as for within-subject factors) or Subject(group×sequence) Error (as for between-subject factors)? Regards BEQool

Helmut
★★★

Vienna, Austria,
2024-10-03 17:33
(67 d 10:08 ago)

@ BEQool
Posting: # 24212
Views: 1,470

ANOVA of Model 1

Post reply

Hi BEQool,

❝ … what did you use as an error term (in the denominator) when testing group*treatment effect? MS Error (as for within-subject factors) or Subject(group×sequence) Error (as for between-subject factors)?

Here is the ANOVA of the first simulated study of our Scenario 1:

Response: log(Y) Df Sum Sq Mean Sq F value Pr(>F) group 1 0.171667589 0.1716675892 2.12908 0.151630 sequence 1 0.339810342 0.3398103421 4.21444 0.046057 * treatment 1 0.349206911 0.3492069106 4.33098 0.043277 * group:period 2 0.224598347 0.1122991734 1.39277 0.259120 group:sequence 1 0.053695040 0.0536950403 0.66594 0.418865 group:treatment 1 0.252244911 0.2522449107 3.12843 0.083870 . group:sequence:subject 44 5.173880238 0.1175881872 1.45837 0.107359 Residuals 44 3.547717653 0.0806299467 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

And then …

p.GxT[sim] <- anova(model1)[["group:treatment", "Pr(>F)"]]

… which is 0.083870. Did we screw up?

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool
★

2024-10-03 18:08
(67 d 09:33 ago)

@ Helmut
Posting: # 24214
Views: 1,506

ANOVA of Model 1

Post reply

❝ Here is the ANOVA of the first simulated study of our Scenario 1:

❝ Response: log(Y)

❝ Df Sum Sq Mean Sq F value Pr(>F)

❝ group 1 0.171667589 0.1716675892 2.12908 0.151630

❝ sequence 1 0.339810342 0.3398103421 4.21444 0.046057 *

❝ treatment 1 0.349206911 0.3492069106 4.33098 0.043277 *

❝ group:period 2 0.224598347 0.1122991734 1.39277 0.259120

❝ group:sequence 1 0.053695040 0.0536950403 0.66594 0.418865

❝ group:treatment 1 0.252244911 0.2522449107 3.12843 0.083870 .

❝ group:sequence:subject 44 5.173880238 0.1175881872 1.45837 0.107359

❝ Residuals 44 3.547717653 0.0806299467

❝ ---

❝ Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

From the table above if I am not mistaken I see that sequence effect was tested by using a Residuals (=MSE) as an error term. Shouldnt sequence effect be tested by using subject(sequence) or in this case subject(group*sequence) as the error term because it is a between subject factor?

I am wondering if group*treatment is also a between-subject factor and should therefore be tested by using subject(group*sequence) as the error term and not Residuals (similarly as sequence effect)?
Because based on this post and your reply (thanks!) sex, sequence, stage, group, site and also sex*treatment are between-subject factors and should therefore be tested by using subject(group*sequence) or subject(sequence) (if we dont have group in the model) as the error term. If sex*treatment is a between-subject factor, I assume group*treatment should also be?

❝ And then …

❝ p.GxT[sim] <- anova(model1)[["group:treatment", "Pr(>F)"]]

❝ … which is 0.083870. Did we screw up?

Probably not but based on my explanation above shouldnt then F value for group*treatment be 2.14516? So 0.2522449107 divided by 0.1175881872?

PS Isn't MS Residuals (=MSE) generally always pretty smaller than MS subject(sequence) or subject(sequence*group) in case of groups?

Helmut
★★★

Vienna, Austria,
2024-10-04 12:11
(66 d 15:30 ago)

@ BEQool
Posting: # 24215
Views: 1,446

ANOVA of Model 1

Post reply

Hi BEQool,

you have a point!

anova1 <- anova(model1) num <- as.numeric(anova1["group:treatment", c(3, 1)]) denom <- as.numeric(anova1["group:sequence:subject", c(3, 1)]) cat(pf(num[1] / denom[1], num[2], denom[2], lower.tail = FALSE), "\n")

Gives for the first símulation above:

0.1501297

However, I’m puzzled because then p(G×T) of our meta study are not uniformly distributed any more. :confused:

Compare them to these plots.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

mittyri
★★

Russia,
2024-10-05 02:08
(66 d 01:33 ago)

(edited on 2024-10-05 22:59)
@ Helmut
Posting: # 24216
Views: 1,415

Between vs Within

Post reply

Hi Helmut & BEQool,

❝ anova1 <- anova(model1)

❝ num <- as.numeric(anova1["group:treatment", c(3, 1)])

❝ denom <- as.numeric(anova1["group:sequence:subject", c(3, 1)])

I really doubt that. In the article^* referred there's a claim:

For the second and third comparisons, models 1 and 2 were used. Sex-by-formulation interactions were expressed by comparing the ratio of the test and reference geometric means for women with that for men. For sex-by-formulation interactions, observed ratio differences of greater than or equal to ±20 percentage points or statistically significant differences at P < .05 were used to identify interactions of interest.

It appears that the authors used the Residual Mean Squares (MS) as the denominator for the F-tests, rather than the Subject MS. This choice is not explicitly stated but seems likely based on the overall approach. Therefore, it's reasonable to assume that the interaction term Group*Formulation should also be tested against the Residual MS, not the Subject MS.

Chen M-L, Lee S-C, Ng M-J, Schuirmann DJ, Lesko LJ, Williams RL. Pharmacokinetic analysis of bioequivalence trials: Implications for sex-related issues in clinical pharmacology and biopharmaceutics. Clin Pharm Ther. 2000; 68(5): 510–21. doi:10.1067/mcp.2000.111184

—
Kind regards,
Mittyri

BEQool
★

2024-10-06 22:06
(64 d 05:34 ago)

@ mittyri
Posting: # 24217
Views: 1,361

Between vs Within

Post reply

Hello Mittyri and Helmut,

❝ It appears that the authors used the Residual Mean Squares (MS) as the denominator for the F-tests, rather than the Subject MS. This choice is not explicitly stated but seems likely based on the overall approach. Therefore, it's reasonable to assume that the interaction term Group*Formulation should also be tested against the Residual MS, not the Subject MS.

So you would assume that Residual Mean Squares (MS) as the denominator was used because nothing is explicitly stated? If Subject MS was used instead as the denominator, you think that they would probably mention it?

Is there any source or literature what to use as denominator when assessing Group*Treatment interaction? I cant find anything relevant. Nothing is mentioned about it in the new ICH M13A guideline: The group x treatment interaction term should not be included in the model. However, applicants should evaluate potential for heterogeneity of treatment effect across groups and discuss its potential impact on the study data, e.g., by investigation of group x treatment interaction in a supportive analysis and calculation of descriptive statistics by group.

I am wondering what to use as a denominator when testing the Group*Treatment interaction and how to support this decision with literature references when writing a report and answering to regulatory agencies.

Additionally, what if the significance between the denominator used differ?

Regards
BEQool

mittyri
★★

Russia,
2024-10-07 00:00
(64 d 03:41 ago)

(edited on 2024-10-07 00:31)
@ BEQool
Posting: # 24219
Views: 1,349

Treatment-by-Group in PROC MIXED and PROC GLM?

Post reply

Hi BEQool,

❝ So you would assume that Residual Mean Squares (MS) as the denominator was used because nothing is explicitly stated? If Subject MS was used instead as the denominator, you think that they would probably mention it?

yes.
The regulators are not trying to make our life easier. Yes, the model to be used is rarely presented. There are not so many exceptions (i.e. replicate designs for EMA or FDA).

❝ Is there any source or literature what to use as denominator when assessing Group*Treatment interaction? I cant find anything relevant. Nothing is mentioned about it in the new ICH M13A guideline

You are right. There's nothing. Note that when FDA published the code for 2*2*2 studies (a while ago), they did not mention in the text that a sequence should be tested against Subject MS, but just added a RANDOM statement in PROC GLM. Anyway, this decision had some reason.

In any sources with Group*Treatment interaction the denominator is not mentioned directly, but I think that without additional RANDOM statement (or ANOVA postprocessing) which SHOULD be mentioned if in use, the Residual MS is still valid. So you should prove why you modified your ANOVA to get F against Subject MS. The `famous` Sun et al. ^* paper states:

For each dataset, a Linear Mixed Model is fit with subgroup, sequence, subgroup-by-sequence, period nested within subgroup, treatment, treatment-by-subgroup as the fixed model and subject nested within subgroup and sequence as a random effect model (the same model can also be fit using the General Linear Model (GLM) with fixed effects only but with slight modification).

❝ Additionally, what if the significance between the denominator used differ?

yes, that's why it should be mentioned if the model/ANOVA was modified to use Subject MS

Wanjie Sun, Don Schuirmann & Stella Grosser (31 Oct 2022): Qualitative versus Quantitative Treatment-by-Subgroup Interaction in Equivalence Studies with Multiple Subgroups, Statistics in Biopharmaceutical Research, doi:10.1080/19466315.2022.2123385

PS: Are you equipped with SAS to check the ANOVA tables for this model (using some dataset without dropouts?) It would be interesting to see what PROC MIXED thinks about this interaction. From this paper I assume they should be the same as PROC GLM. But who knows! I could be wrong with my claim...

—
Kind regards,
Mittyri

BEQool
★

2024-10-07 12:27
(63 d 15:14 ago)

@ mittyri
Posting: # 24220
Views: 1,291

Treatment-by-Group in PROC MIXED and PROC GLM?

Post reply

❝ […] So you should prove why you modified your ANOVA to get F against Subject MS. […]

I mean if the Group*Treatment is a between-subject factor, it should be, in my opinion, tested by using Subject MS as the denominator.

❝ The `famous` Sun et al. paper states:

❝ For each dataset, a Linear Mixed Model is fit with subgroup, sequence, subgroup-by-sequence, period nested within subgroup, treatment, treatment-by-subgroup as the fixed model and subject nested within subgroup and sequence as a random effect model (the same model can also be fit using the General Linear Model (GLM) with fixed effects only but with slight modification).

As already discussed, you think that if they had used MS Subject as the denominator, they would have probably written it here?

❝ PS: Are you equipped with SAS to check the ANOVA tables for this model (using some dataset without dropouts?) It would be interesting to see what PROC MIXED thinks about this interaction. From this paper I assume they should be the same as PROC GLM. But who knows! I could be wrong with my claim...

Yes I have access to SAS but I dont exactly understand what would you like to know? Which dataset are you talking about? Some random dataset or a specific one? And you are talking about testing Group*Treatment interaction with this dataset with both PROC MIXED and PROC GLM and to see if there is any difference?

Regards
BEQool

mittyri
★★

Russia,
2024-10-07 15:11
(63 d 12:30 ago)

@ BEQool
Posting: # 24222
Views: 1,289

Treatment-by-Group in PROC MIXED and PROC GLM?

Post reply

Hi BEQool,

❝ ❝ […] So you should prove why you modified your ANOVA to get F against Subject MS. […]

❝ I mean if the Group*Treatment is a between-subject factor, it should be, in my opinion, tested by using Subject MS as the denominator.

<IMHO>
Subject MS (nested within subgroup and sequence) is typically used to test other between-subject factors, like sequence or subgroup effects, not treatment-related effects. The subject MS usually captures variability between subjects but within their respective subgroups and sequences. Treatment-by-subgroup MS should generally be tested against the residual MS, because this residual error captures the variability within subjects that remains after accounting for treatment and subgroup effects.
If the subject MS is used as the denominator, this could lead to inflated F-values because the subject MS reflects between-subject variability, which is not appropriate for testing interactions involving the treatment effect
</IMHO>

❝ As already discussed, you think that if they had used MS Subject as the denominator, they would have probably written it here?

yes

❝ Which dataset are you talking about? Some random dataset or a specific one?

No preferences in exact dataset, but it would be good to test the dataset with unequal groups length

❝ And you are talking about testing Group*Treatment interaction with this dataset with both PROC MIXED and PROC GLM and to see if there is any difference?

yes, if possible please

—
Kind regards,
Mittyri

BEQool
★

2024-10-14 13:01
(56 d 14:39 ago)

@ mittyri
Posting: # 24227
Views: 1,035

Treatment-by-Group in PROC MIXED and PROC GLM?

Post reply

Hello Mittyri,

thank you for your comment regarding the denominator used for testing Group*Tretament effect, highly appreciated.

❝ ❝ Which dataset are you talking about? Some random dataset or a specific one?

❝ No preferences in exact dataset, but it would be good to test the dataset with unequal groups length

❝ ❝ And you are talking about testing Group*Treatment interaction with this dataset with both PROC MIXED and PROC GLM and to see if there is any difference?

❝ yes, if possible please

In SAS I performed ANOVA with dataset with unequal group sizes (and with subjects that completed both periods) with both PROC MIXED and PROC GLM to test Group*Treatment effect and the p-values for all PK parameters do not differ between PROC MIXED and PROC GLM. So you were right :ok:

BEQool

Group-by-Treatment Interaction: the final word? [General Sta­tis­tics]

Group-by-Treatment Interaction: FDA

Group-by-Treatment Interaction: the final word?

ANOVA of Model 1

ANOVA of Model 1

ANOVA of Model 1

Between vs Within

Between vs Within

Treatment-by-Group in PROC MIXED and PROC GLM?

Treatment-by-Group in PROC MIXED and PROC GLM?

Treatment-by-Group in PROC MIXED and PROC GLM?

Treatment-by-Group in PROC MIXED and PROC GLM?

Group-by-Treatment Interaction: the final word? [General Statistics]