Bioequivalence and Bioavailability Forum

GM ★ India, 2019-07-20 09:36 (2091 d 15:20 ago) Posting: # 20410 Views: 9,295	Possible reasons for group effect [Design Issues] Post reply
	Hello Bebac Friends, Hope all are doing well...!!! When I am reading about fixed effects used in general crossover studies and possible reasons for significant results of these fixed effects, I didn't found any relevant reasons for the significant group effect. Anybody know about the possible reasons for significant group effect...? Thanks in advance. — Best Regards, GM

Helmut
★★★

Vienna, Austria,
2019-07-20 16:02
(2091 d 08:54 ago)

@ GM
Posting: # 20413
Views: 8,374

Group “effect”

Post reply

Hi GM,

❝ […] fixed effects used in general crossover studies and possible reasons for significant results of these fixed effects, I didn't found any relevant reasons for the significant group effect.

The p-value itself tells you nothing. If you set your limit to 0.05, that means that you consider a p-value <0.05 denoting an effect which occurred not by pure chance. The reason for an effect is beyond the reach of statistics.
Think about the subject-term. It is always highly significant.* Recode the common effects to something neutral (say, response → Y, sequence → a, subject → b, period → c, treatment → d ) and provide the data to a statistician without telling the background to evaluate the linear model $$\ln(Y) \sim a+b+c+d$$ or – if you insist on the stupid over-specified one given in the guidelines – $$\ln(Y) \sim a+b(a)+c+d$$ Hey, $p(b)$ or $p(b(a)) = 0.00000314$. Now ask for a “reason”.
Answer: :ponder:

Science is wonderfully equipped to answer the question “How?”
but it gets terribly confused when you ask the question “Why?” Erwin Chargaff

Ask a physicist what gravity is. No, not how it is described in physics. You will be surprised.

❝ Anybody know about the possible reasons for significant group effect...?

Chance? IMHO, testing for it is futile (see there).

Unless you perform the study in cloned subjects, which would violate an assumption of the model, namely independence. If I would ever see a study with a p-value >0.05 for the subject effect it would ring my alarm bells.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro
★★★

Denmark,
2019-07-20 21:20
(2091 d 03:37 ago)

@ GM
Posting: # 20414
Views: 8,160

Possible reasons for group effect

Post reply

Hello GM,

there are always potential jokers in play:

1. Check if the volunteers within groups have something in common.
Were they recruited in a different manner, in spite of all subjects fulfilling the enrollment criteria? I have seen cases of that recently.
Eyeball if body weight, gender mix, age, or some other factor may differ a bit between groups. If it does then there is a likely -and I am not saying definite- reason, but it is not one associated with a ton of literature.

2. The use of groups is often a capacity issue, relating to the number of beds at the CRO.
Groups are then separated in time, for example by days or weeks, sometimes even months. And time has funky effects, not only on individuals but also on groups of individuals.
Heat wave in Mumbai? Some subjects will be borderline dehydrated when showing up, and they will not feel much like walking around.
Pollen season just set in in Winchester VA? Some subjects will be coughing and wheezing when showing up, even if they don't have a medical history of allergies.
I saw a TV add about vegan diets yesterday. Therefore, brainwashed as I am, the next two weeks I will be buying all the bran and celery soup available in the supermarket and I will be going full tilt into that thing until I realise it is just killing my quality of life, and all the while I will have a changing phenotype of sorts. Until I start living normally again. I find some comfort in knowing that there are others who saw the same TV add and who are suffering the same phenomenon (and the owner of the company selling celery is likely going on monthly vacations to Tonga or some such remote and exotic place, due to victims of his affairs like me, but this is another story).

And so forth.

Finally, bear in mind that phase III studies across centers often have significant center effects. I think this phenomenon is much comparable to the group effect, honestly. Such phase III trials get approved.

The group effect questions from regulators are often not too difficult to handle. They do not become a cause for rejection in my experience.

—
Pass or fail!
ElMaestro

GM
★

India,
2019-07-23 08:23
(2088 d 16:34 ago)

@ ElMaestro
Posting: # 20433
Views: 7,922

Possible reasons for group effect

Post reply

Hi Helmut and ElMaestro,

Thanks for reply. It helps me a lot.

I have seen one FDA BE review (see link) regarding group effect. Please see page 6 (Reviewer’s Comments) and page 52 (control document).

Here agency is asking for the BE of any one of the group, if group-by-treatment interaction is significant (p<0.05). Is it possible in studies conducted on HVD?

Please provide your thoughts on the same.

—
Best Regards,
GM

Helmut
★★★

Vienna, Austria,
2019-07-23 12:32
(2088 d 12:24 ago)

@ GM
Posting: # 20435
Views: 7,972

“Group-by-Treatment Interaction”

Post reply

Hi GM,

❝ I have seen one FDA BE review […].

Yep, the 1999 (!) infamous one. Groups separated by two weeks. See the FDA’ applicable guidance (2001), Section VII.A.

❝ Here agency is asking for the BE of any one of the group, if group-by-treatment interaction is significant (p<0.05).

0.1 not 0.05. BTW, did you bother reading the presentation I linked in my OP?

❝ Is it possible in studies conducted on HVD?

Do you have a replicate design in mind? For ABE, no problem. For the EMA’s ABEL, doable. For the FDA’s RSABE, difficult. Ask three statisticians only to get get four options. Possible that the FDA insists on the fifth nobody has thought about. Difficult terrain.

❝ Please provide your thoughts on the same.

Let’s assume that you plan the study in such a way that groups are not expected to differ (see ElMaestro’s post).

FDA, Gulf Cooperation Council, Russian Federation, Eurasian Economic Union
Give a justification (in the protocol!) that all criteria will be met and go full throttle with the pooled analysis.
FDA: If in doubt, start a controlled correspondence.
The others: There are no scientific advices, only private dinners. Consider ‘Model II’.
EMA
Thousand of studies accepted in multiple groups without problems (two exceptions in my presentation). If you prefer braces with suspenders, state ‘Model II’ in the protocol. Loss in power compared to the pooled analysis is negligible.

Do not (!) apply the framework suggested by the FDA in the dark ages!

If there is no true group-by-treatment interaction, in 10% of cases you will detect one by pure chance (false positive). No pooling, analysis of separate groups, power to show BE – even in the largest one – low.
Any pre-test can inflate the Type I Error.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool ★ 2024-10-22 14:39 (170 d 10:17 ago) @ Helmut Posting: # 24243 Views: 3,818	“Group-by-Treatment Interaction” Post reply
	Hello ❝ ❝ Here agency is asking for the BE of any one of the group, if group-by-treatment interaction is significant (p<0.05). ❝ 0.1 not 0.05. BTW, did you bother reading the presentation I linked in my OP? Why would the alpha for G*T interaction be 0.1 and not 0.05 like almost always? Regards BEQool

Helmut
★★★

Vienna, Austria,
2024-10-22 14:47
(170 d 10:10 ago)

@ BEQool
Posting: # 24244
Views: 3,812

“Group-by-Treatment Interaction”

Post reply

Hi BEQool,

❝ ❝ ❝ Here agency is asking for the BE of any one of the group, if group-by-treatment interaction is significant (p<0.05).

❝ ❝ 0.1 not 0.05. BTW, did you bother reading the presentation I linked in my OP?

❝

❝ Why would the alpha for G*T interaction be 0.1 and not 0.05 like almost always?

The original post is five years old… At the time being 0.05 indeed. The 0.1 was used by the FDA in analogy of Grizzle’s dreadful test¹ for unequal carryover.

“… a preliminary test should be made at some high level of significance, say α = .10 or α = .15”

It is evident that increasing the level of any test to offset inadequate power leads to an increased false positive rate.
BTW, the FDA’s 2022 guidance² does not specify a particular level but has Grizzle in the references. Even worse, Alosh³ as well:

“To improve the power when testing for an interaction, some suggest using a test size of 0.10 or in extreme cases 0.20, particularly when there is reason to suspect a specific interaction exists. While an increase in the test size makes it easier to detect an interaction, this would be at the expense of an increase in the chance of false positive findings. The choice of the test size can depend on the context and it would be inappropriate to say a specific test size is applicable in all scenarios.”

Make your pick.

Grizzle JE. The Two-Period Change-Over Design and Its Use in Clinical Trials. Biometrics. 1965; 21(2): 467–80. doi:10.2307/2528104.
FDA (CDER). Statistical Approaches to Establishing Bioequivalence. Guidance for Industry. Draft. Silver Spring, MD. December 2022. Download.
Alosh M, Fritsch K, Huque M, Mahjoob K, Pennello G, Rothmann M, Russek-Cohen E, Smith F, Wilson S, Yue L. Statistical Considerations on Subgroup Analysis in Clinical Trials. Stat Biopharm Res. 2015; 7(4): 286–304. doi:10.1080/19466315.2015.1077726.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

mittyri ★★ Russia, 2024-10-22 18:03 (170 d 06:53 ago) @ Helmut Posting: # 24245 Views: 3,752	OT: A choice between Post reply
	Dear Colleagues! cannot stop myself, sorry — Kind regards, Mittyri

BEQool
★

2024-10-23 13:02
(169 d 11:54 ago)

@ mittyri
Posting: # 24246
Views: 3,710

OT: A choice between

Post reply

Dear Helmut,

❝ The original post is five years old… At the time being 0.05 indeed. The 0.1 was used by the FDA in analogy of Grizzle’s dreadful test¹ for unequal carryover.“

❝ [...]Make your pick.

Thanks for a thorough answer.
Regarding the choice of alpha level of 0.05 or 0.1 - every sponsor would probbaly choose 0.05 in order to decrease the chance of detecting an interaction?

Dear Mittyri,
good one :ok:

(with alpha-value instead of p-value in the picture)
As written above, every sponsor would probably take the blue one :-D

Regards
BEQool

Helmut
★★★

Vienna, Austria,
2024-10-23 13:58
(169 d 10:59 ago)

@ BEQool
Posting: # 24247
Views: 3,666

OT: A choice between

Post reply

Hi BEQool,

❝ Regarding the choice of alpha level of 0.05 or 0.1 - every sponsor would probbaly choose 0.05 in order to decrease the chance of detecting an interaction?

Sure – though I still hold that the entire idea of such a test is crap.

Let’s see again what ICH M13A says:

The statistical model should take into account the multi-group nature of the BE study, e.g., by using a model including terms for group, sequence, sequence × group, subject within sequence × group, period within group and formulation. The group × treatment interaction term should not be included in the model. However, applicants should evaluate potential for heterogeneity of treatment effect across groups and discuss its potential impact on the study data, e.g., by investigation of group × treatment interaction in a supportive analysis and calculation of descriptive statistics by group.

What should we point out in the discussion? That a significant group × treatment interaction is to be expected at the level of test? What might be meant by »calculation of descriptive statistics by group«? Geometric means of PK metrics (irrespective of treatment), separately for the treatments, or PEs by model 3 (i.e., the conventional one with any group terms)? What would an assessor conclude from any of those? Even if they would calculate the CI of groups separately, likely they would overlap. So what?

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool
★

2024-10-23 15:17
(169 d 09:39 ago)

@ Helmut
Posting: # 24248
Views: 3,633

OT: A choice between

Post reply

Hello Helmut,

of course I unfortunately dont know the answers to your questions but

❝ [...] or PEs by model 3 (i.e., the conventional one with any group terms)? What would an assessor conclude from any of those? Even if they would calculate the CI of groups separately, likely they would overlap. So what?

Isnt it so that if we detect Group*Treatment interaction, the CIs of groups seperately very likely wouldnt overlap? That is why we detect an interaction?

BEQool

Helmut
★★★

Vienna, Austria,
2024-10-23 15:22
(169 d 09:34 ago)

@ BEQool
Posting: # 24249
Views: 3,655

OT: A choice between

Post reply

Hi BEQool,

❝ Isnt it so that if we detect Group*Treatment interaction, the CIs of groups seperately very likely wouldnt overlap? That is why we detect an interaction?

Not necessarily. See the example at the end of this post.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Helmut
★★★

Vienna, Austria,
2024-11-03 12:36
(158 d 11:20 ago)

@ BEQool
Posting: # 24255
Views: 3,006

Groups: Overlapping CIs

Post reply

Hi BEQool,

❝ Isnt it so that if we detect Group*Treatment interaction, the CIs of groups seperately very likely wouldnt overlap? That is why we detect an interaction?

The example I referred to in my previous post was not ideal because $\small{p(\text{G}\times \text{T})=0.080992}$ and you were asking for a case where the Group-by-Treatment interaction is significant.

Below simulated data for an extreme case – Wanjie would love – with $\small{\mu_1=0.80}$ and $\small{\mu_2=1.25}$.

subject group sequence treatment period Y 1 1 TR T 1 2.6786 1 1 TR R 2 2.6848 2 1 TR T 1 2.0931 2 1 TR R 2 3.5289 3 1 TR T 1 2.0567 3 1 TR R 2 3.4351 4 1 TR T 1 2.2692 4 1 TR R 2 2.7533 5 1 TR T 1 3.6712 5 1 TR R 2 2.3086 6 1 TR T 1 2.6790 6 1 TR R 2 3.3427 7 1 RT R 1 3.9382 7 1 RT T 2 2.9792 8 1 RT R 1 3.0783 8 1 RT T 2 3.8813 9 1 RT R 1 2.2989 9 1 RT T 2 2.8853 10 1 RT R 1 3.5939 10 1 RT T 2 2.0244 11 1 RT R 1 2.7416 11 1 RT T 2 1.7836 12 1 RT R 1 2.8352 12 1 RT T 2 1.7375 13 2 TR T 1 3.8682 13 2 TR R 2 3.8447 14 2 TR T 1 3.1741 14 2 TR R 2 3.4061 15 2 TR T 1 3.5032 15 2 TR R 2 3.4281 16 2 TR T 1 4.0822 16 2 TR R 2 2.1213 17 2 TR T 1 4.3318 17 2 TR R 2 2.1424 18 2 TR T 1 3.0408 18 2 TR R 2 3.3706 19 2 RT R 1 2.8236 19 2 RT T 2 2.8858 20 2 RT R 1 2.2274 20 2 RT T 2 3.5447 21 2 RT R 1 2.8156 21 2 RT T 2 2.7663 22 2 RT R 1 2.4314 22 2 RT T 2 1.8948 23 2 RT R 1 2.1932 23 2 RT T 2 2.7153 24 2 RT R 1 2.6504 24 2 RT T 2 2.8821 Model I (pooled data) p(G×T) = 0.024984 Model II (pooled data) PE = 97.64% 90% CI = 85.53 – 111.47% CVw = 27.13% Model III (group 1) PE = 82.54% 90% CI = 68.23 – 99.86% CVw = 26.16% Model III (group 2) PE = 115.50% 90% CI = 98.04 – 136.08% CVw = 22.43%

As to be expected, the Group-by-Treatment interaction is significant.
The study passes with flying colors by Model II. Also expected because the study was powered for $\small{\mu_2=1/\mu_1=1}$.
Both groups assessed by Model III fail. Of course, they do because we simulated them at the limits of the BE range. However, their confidence intervals do overlap.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool
★

2024-11-05 10:14
(156 d 13:42 ago)

@ Helmut
Posting: # 24259
Views: 2,878

Groups: Overlapping CIs

Post reply

Hello Helmut,

thank you for the illustration with specific example.

❝ Model I (pooled data)

❝ p(G×T) = 0.024984

❝ Model II (pooled data)

❝ PE = 97.64%

❝ 90% CI = 85.53 – 111.47%

❝ CVw = 27.13%

❝ Model III (group 1)

❝ PE = 82.54%

❝ 90% CI = 68.23 – 99.86%

❝ CVw = 26.16%

❝ Model III (group 2)

❝ PE = 115.50%

❝ 90% CI = 98.04 – 136.08%

❝ CVw = 22.43%

❝ As to be expected, the Group-by-Treatment interaction is significant.

❝ The study passes with flying colors by Model II. Also expected because the study was powered for $\small{\mu_2=1/\mu_1=1}$.

❝ Both groups assessed by Model III fail. Of course, they do because we simulated them at the limits of the BE range. However, their confidence intervals do overlap.

Yes they overlap slightly (upper limit of group 1= 99.86% vs. lower limit of group 2= 98.04%).
Like you said in the other post ("Even if an assessor would calculate the confidence interval of groups separately, likely they would overlap due to the limited sample sizes."), they most likely overlap because of the relatively small sample size (n=12 per group). Probably if we had larger total sample size (in this example it would maybe be reasonable as CVw is around 27%, e.g. total sample size of 32 or 36 subjects) and thus larger sample size per group then groups' 90% CI most likely wouldnt overlap.
I am just wondering if variabilty is moderate and the sample size is large enough (--> high power) then in most cases groups' 90% CI probably wouldnt overlap in case of significant G*T interaction?

BEQool

Helmut
★★★

Vienna, Austria,
2024-11-05 11:12
(156 d 12:44 ago)

@ BEQool
Posting: # 24260
Views: 2,860

Groups: Overlapping CIs

Post reply

Hi BEQool,

❝ Like you said in the other post ("Even if an assessor would calculate the confidence interval of groups separately, likely they would overlap due to the limited sample sizes."), they most likely overlap because of the relatively small sample size (n=12 per group). Probably if we had larger total sample size (in this example it would maybe be reasonable as CVw is around 27%, e.g. total sample size of 32 or 36 subjects) and thus larger sample size per group then groups' 90% CI most likely wouldnt overlap.

❝ I am just wondering if variabilty is moderate and the sample size is large enough (--> high power) then in most cases groups' 90% CI probably wouldnt overlap in case of significant G*T interaction?

You are right:

Simulated data n = 36 group 1: n = 18, mue = 0.80 group 2: n = 18, mue = 1.25 Model I (pooled data) p(G×T) = 0.014978 Model II (pooled data) PE = 112.86% 90% CI = 101.14 – 125.93% CVw = 28.01% Model III (group 1) PE = 96.75% 90% CI = 81.55 – 114.79% CVw = 30.02% Model III (group 2) PE = 131.64% 90% CI = 116.71 – 148.49% CVw = 20.91%

However, how realistic is $\small{\mu_1=0.80,\,\mu_2=1/\mu_1}$?

Let’s try a less extreme case with $\small{\mu_1=0.825,\,\mu_2=1/\mu_1}$:

Simulated data n = 32 group 1: n = 16, mue = 0.825 group 2: n = 16, mue = 1.2121 Model I (pooled data) p(G×T) = 0.040949 Model II (pooled data) PE = 103.96% 90% CI = 92.58 – 116.73% CVw = 27.80% Model III (group 1) PE = 90.57% 90% CI = 75.27 – 108.96% CVw = 30.37% Model III (group 2) PE = 119.33% 90% CI = 104.67 – 136.03% CVw = 21.28%

$\small{p(\text{G}\times\text{T})}$ is significant and CIs of groups overlap. :-D

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool
★

2024-11-06 08:44
(155 d 15:12 ago)

@ Helmut
Posting: # 24262
Views: 2,769

Groups: Overlapping CIs

Post reply

Hello Helmut

❝ Let’s try a less extreme case with $\small{\mu_1=0.825,\,\mu_2=1/\mu_1}$:

❝ Simulated data

❝ n = 32

❝ group 1: n = 16, mue = 0.825

❝ group 2: n = 16, mue = 1.2121

❝ Model I (pooled data)

❝ p(G×T) = 0.040949

❝ Model II (pooled data)

❝ PE = 103.96%

❝ 90% CI = 92.58 – 116.73%

❝ CVw = 27.80%

❝ Model III (group 1)

❝ PE = 90.57%

❝ 90% CI = 75.27 – 108.96%

❝ CVw = 30.37%

❝ Model III (group 2)

❝ PE = 119.33%

❝ 90% CI = 104.67 – 136.03%

❝ CVw = 21.28%

❝ $\small{p(\text{G}\times\text{T})}$ is significant and CIs of groups overlap. :-D

Yes true. The variability of group 1 is quite high - if we had similar variability than in group 2, then the 90% CIs wouldnt overlap :-D

But then again, it makes sense and we all know that with larger sample size and lower variability we get narrower CIs and lesser chance of them overlapping so yes as can be seen it depends on case-by-case basis. :-)

Helmut
★★★

Vienna, Austria,
2024-11-21 14:20
(140 d 09:36 ago)

@ BEQool
Posting: # 24286
Views: 2,430

Groups: (Hardly‽) overlapping CIs

Post reply

Hi BEQool,

an example. Study powered to 90%, n=65, n₁=31, n₂=34, groups separated by 1 (one!) day. Evaluation per protocol with group model II. C_max and AUC passed with ease. A deficiency letter one week before M13A was published:

Due to the significant Group × Formulation effect (p<0.05) for ln-transformed C_max observed in model I, a separate exploratory analysis of each group was performed and produced the following outcomes:

Group 1: The 90% CI for C_max (108.5–126.38%) fell outside the bioequivalence limits, while the CI for AUC (99.31–117.47%) remained within the limits.
Group 2: Both the 90% CIs for C_max (90.01–109.2%) and AUC (85.43–106.39%) fell within the bioequivalence limits.

The finding from model I suggests heterogeneity of treatment effect across groups, with 90% CIs for C_max hardly overlapped as demonstrated with model III submitted by the applicant. Therefore, the applicant should provide a justification for this difference and discuss its potential impact on the conclusion of bioequivalence.

A new term: Hardly overlapping CIs. Again: So what? Justification?

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool
★

2024-11-22 13:32
(139 d 10:24 ago)

@ Helmut
Posting: # 24291
Views: 2,283

Groups: (Hardly‽) overlapping CIs

Post reply

Hello Helmut,

thank you for the example provided. Unfortunately, these kind of deficiency questions will probably become inevitable when conducting a study in groups.

❝ […] a separate exploratory analysis of each group was performed and produced the following outcomes

Did you perform this exploratory analysis of each group or did the agency do it?

❝ Therefore, the applicant should provide a justification for this difference and discuss its potential impact on the conclusion of bioequivalence.

❝ A new term: Hardly overlapping CIs. Again: So what? Justification?

How do you plan to "justify" this difference?

Regards
BEQool

Helmut
★★★

Vienna, Austria,
2024-11-22 13:58
(139 d 09:59 ago)

@ BEQool
Posting: # 24292
Views: 2,289

Groups: (Hardly‽) overlapping CIs

Post reply

Hi BEQool,

❝ […] Unfortunately, these kind of deficiency questions will probably become inevitable when conducting a study in groups.

Quite likely, esp. since agencies think that $\small{p(G\times T)<0.05}$ is a signal of data manipulation.

❝ ❝ […] a separate exploratory analysis of each group was performed and produced the following outcomes

❝ Did you perform this exploratory analysis of each group or did the agency do it?

The applicant did.

❝ ❝ Therefore, the applicant should provide a justification for this difference and discuss its potential impact on the conclusion of bioequivalence.

↑ This was a request of the agency.

❝ ❝ A new term: Hardly overlapping CIs. Again: So what? Justification?

❝ How do you plan to "justify" this difference?

Well, the deadline for the response is today. I made only sarcastic comments like …

As long as confidence limits overlap, treatment effects estimated in the groups do not differ significantly. To question that is like saying “since the upper confidence limit is 124%, products are hardly bioequivalent”

… and suggested the applicant to translate it into a more diplomatic language.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Helmut
★★★

Vienna, Austria,
2024-11-23 14:05
(138 d 09:52 ago)

@ BEQool
Posting: # 24295
Views: 2,180

Multiplicity?

Post reply

Hi BEQool,

I forgot something. The GL states

… applicants should evaluate potential for heterogeneity of treatment effect across groups …

In my understanding »across groups« means all pairwise comparisons. Then their number increases quickly with the number of groups and PK metrics. Let $\small{n}$ be the number of groups and $\small{m}$ the number of PK-metrics. Then the number of pairwise comparisons is given by $\small{k=}\frac{n!}{2\,(n-2)!}$ per metric. The familywise error rate (here the chance to observe at least one false positive in any of the tests) is given by $\small{(1-(1-\alpha)^k})\times m$. With $\small{\alpha=0.05}$ in my example above we get 10%. In order to counteract that we should test with $\small{\alpha_\text{adj}=\alpha / (k\times m)}$.
If that is done, the G×T interaction of C_max would not be significant any more.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BEQool
★

2024-11-26 09:15
(135 d 14:41 ago)

(edited on 2024-11-26 13:40)
@ Helmut
Posting: # 24297
Views: 2,127

Groups

Post reply

Hello Helmut,

❝ Quite likely, esp. since agencies think that $\small{p(G\times T)<0.05}$ is a signal of data manipulation.

In cases where we have "good" PEs for PK parameters (let's say PEs around 100%) in the first group and "bad" PEs for PK parameters (let's now say around 85% or 115%) in the second group, there should be no reason to question the data integrity right?

❝ ❝ ❝ A new term: Hardly overlapping CIs. Again: So what? Justification?

❝ ❝ How do you plan to "justify" this difference?

❝ Well, the deadline for the response is today. I made only sarcastic comments like …As long as confidence limits overlap, treatment effects estimated in the groups do not differ significantly. To question that is like saying “since the upper confidence limit is 124%, products are hardly bioequivalent”… and suggested the applicant to translate it into a more diplomatic language.

❝ I forgot something. The GL states… applicants should evaluate potential for heterogeneity of treatment effect across groups …In my understanding »across groups« means all pairwise comparisons. Then their number increases quickly with the number of groups and PK metrics. Let $\small{n}$ be the number of groups and $\small{m}$ the number of PK-metrics. Then the number of pairwise comparisons is given by $\small{k=}\frac{n!}{2\,(n-2)!}$ per metric. The familywise error rate (here the chance to observe at least one false positive in any of the tests) is given by $\small{(1-(1-\alpha)^k})\times m$. With $\small{\alpha=0.05}$ in my example above we get 10%. In order to counteract that we should test with $\small{\alpha_\text{adj}=\alpha / (k\times m)}$.

❝ If that is done, the G×T interaction of C_max would not be significant any more.

Well to me that sounds like a good answer to a deficiency letter as well :-)

PS How do you get FWER=10%? Based on your example above m=2 (AUC and Cmax), n=2 (2 groups) and therefore k=2. With alpha=0.05 shouldnt FWER be 19.5%?
Or did you use m=1 as both AUC and Cmax should be okay (union-intersection principle) and got FWER=9.75%=10%?

BEQool

Possible reasons for group effect [Design Issues]

Group “effect”

Possible reasons for group effect

Possible reasons for group effect

“Group-by-Treatment Interaction”

“Group-by-Treatment Interaction”

“Group-by-Treatment Interaction”

OT: A choice between

OT: A choice between

OT: A choice between

OT: A choice between

OT: A choice between

Groups: Overlapping CIs

Groups: Overlapping CIs

Groups: Overlapping CIs

Groups: Overlapping CIs

Groups: (Hardly‽) overlapping CIs

Groups: (Hardly‽) overlapping CIs

Groups: (Hardly‽) overlapping CIs

Multiplicity?

Groups