Helmut
★★★

Vienna, Austria,
2019-07-21 13:20

Posting: # 20417
Views: 1,959

## Indirect adjusted comparisons in BE [General Sta­tis­tics]

Dear all (and esp. Shuanghe),

I must confess that I don’t get what Luther Gwaza et al.1,2 means by his “pragmatic method”:

This method does not require the assumption of homogeneity of variances ($$SE_d=SE_{A}^{2}+SE_{B}^{2}$$), since it is un­likely verifiable, between studies with small sample sizes that follow Student’s t-test distribution (t0.9,d.f.), whose degrees of freedom are approximated for simplicity as if the variances were homogeneous (n1+n2–2).

Sounds great but how does he pool variances here? If they are weighted by the studies’ sample sizes – which IMHO, is the correct method – we end up in the homoscedastic method, where (in his notation):$$SD_{pooled}^{2}=\frac{(n_1-1)SD_{1}^{2}+(n_2-1)SD_{2}^{2}}{n_1+n_2-2}$$ BTW, is this a typo in the quote? A pooled variance as the sum? I suspect that he simply used the arithmetic mean which would not be correct even for equal samples & variances.
@Shuanghe: How did you do it?

1. Gwaza L, Gordon J, Welink J, Potthast H, Hansson H, Stahl M, García-Arieta A. Statistical approaches to indirectly compare bioequivalence between generics: a comparison of methodologies employing artemether / lume­fantrine 20/120 mg tablets as prequalified by WHO. Eur J Clin Pharmacol. 2012;68(12):1611–8. doi:10.1007/s00228-012-1396-1.
2. Gwaza L. Adjusted Indirect Treatment Comparisons of Bioequivalence Studies. PhD Thesis: Utrecht. 2016. free resource.

Cheers,
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
mittyri
★★

Russia,
2019-07-21 23:46

@ Helmut
Posting: # 20419
Views: 1,760

## Pragmatic assumptions

Dear Helmut,

» I must confess that I don’t get what Luther Gwaza et al.1,2 means by his “pragmatic method”:

agree, intricated explanation
In the discussion part I've found
The method based on a combination of both (SE calculation of the heteroscedastic method and degrees of freedom of the homoscedastic method), which is the simplest numerical calculation, gives results similar to those obtained with the heteroscedastic approach, showing that the contribution of the different method of calculation of the degrees of freedom is negligible.
Heteroscedatic method + simplified DFs

I also found some paper where the authors are using that approach, but I don't think their understanding of 'pragmatic' is correct

Kind regards,
Mittyri
Shuanghe
★★

Spain,
2019-07-22 07:53

@ Helmut
Posting: # 20421
Views: 1,763

## Indirect adjusted comparisons in BE

Dear Helmut,

» I must confess that I don’t get what Luther Gwaza et al.1,2 means by his “pragmatic method”:

I must confess that I didn't understand it either (I even checked the dictionary to see if I misunderstood the word somehow ). I suppose that it's just a name invented by Gwaza as I didn't see any reference citing the source of the name.

» @Shuanghe: How did you do it?

The variance is simply obtained by $$SE_d^2 = SE_1^2 + SE_2^2$$ and the degree of freedom used for confidence interval is $$n_1 + n_2 - 2$$.

I have to admit it that it puzzled me as well so I found all articles that I can found from the same group to see if others might gave better description of the method. So in addition to Gwaza's articles, I have Herranz 2013, Pejčić 2019 etc, but it seems that the excel template is not the only thing Gzawa gave to others---the description of the method are all very similarly cryptic.

OT again:

Thanks a lot for implementing MathJax. Equations are looking much better. one puzzle though: command \LaTeX does do what it supposed to do.

All the best,
Shuanghe
nobody
nothing

2019-07-22 11:32

@ Shuanghe
Posting: # 20423
Views: 1,726

## Indirect adjusted comparisons in BE

"Pragmatic" is normally the term you use if there is no sound theoretical and/or scientific justification for the shortcut you take...

Is there really nothing more in the "Methodology" of the full publication than what mittyri found in the discussion? Wow, who reviewed this publication/thesis? 8-)

Kindest regards, nobody
Shuanghe
★★

Spain,
2019-07-22 13:20

@ nobody
Posting: # 20424
Views: 1,701

## Indirect adjusted comparisons in BE

Dear nobody,

» Is there really nothing more in the "Methodology" of the full publication than what mittyri found in the discussion? Wow, who reviewed this publication/thesis? 8-)

Nope. Not that I can find, e.g., in Pejčić's article*, the description is word for word copy of what Helmut quoted above so i don't want to copy/paste it again.

Pejčić, Zorica; Vučićević, Katarina; García-Arieta, Alfredo and Miljković, Branislava. Adjusted indirect comparisons to assess bioequivalence between generic clopidogrel products in Serbia. Br. J. Clin. Pharmacol., 2019. DOI:10.1111/bcp.13997

All the best,
Shuanghe
Helmut
★★★

Vienna, Austria,
2019-07-22 13:45

@ Shuanghe
Posting: # 20425
Views: 1,704

## Indirect adjusted comparisons in BE

Hi Shuanghe, Mittyri, and nobody,

yep, you are all right. Nobody’s definition of “pragmatic” is correct. IMHO, here it should be called – following Wolfgang Pauli’s footprints – “not even wrong”.
Pooling variances without weighting by the sample size is just false.

Quoting Shuanghe’s reference, Section Data analysis:

In line with recommendations by Gwaza et al., a pragmatic method based on t test was used […]. This method has been shown to give comparable results as the homoscedastic method.

Great. What Gwaza et al.* really wrote/recommended is this:

The [pragmatic] method based on a combination of both (SE calculation of the heteroscedastic method and degrees of freedom of the homocedastic method), which is the simplest numerical calculation, gives results similar to those obtained with the heteroscedastic approach, showing that the contribution of the different method of calculation of the degrees of freedom is negligible.
In conclusion, the homoscedastic method is recommended when performing adjusted indirect comparisons of bioequivalence studies—unless a clear difference in variances is observed in the data, which is unlikely.

Gwaza drives me nuts. What is “SE calculation of the heteroscedastic method”? There is only one correct way to pool variances. No, it’s not $$SE_d^2 = SE_1^2 + SE_2^2$$. The homo- and heteroscedastic methods differ only in the degrees of freedom.
• homoscedastic
$$\nu = n_1+n_2-2$$
• heteroscedastic
$$\nu \approx \frac{\left (\frac{s_{1}^{2}}{n_1}+\frac{s_{1}^{2}}{n_2}\right )^2}{\frac{s_{1}^{4}}{n_1(n_1-1)}+\frac{s_{1}^{4}}{n_2(n_2-1)}}$$
This is not the first paper written by regulators where the conclusions contradict the result.
Comes handy cause most people don’t read beyond the abstract (men on a mission?). When I rank the width of the CIs given in the two tables:
1. The pragmatic method lies between the hetero- and homoscedastic methods.
2. The pragmatic method is much closer to the hetero- than to the homoscedastic method.
(a) and (b) agree what he wrote above. As a regulator why does he recommend the most liberal one (tightest CI)?
If I use the crappy formula I end up with CIs which are not slightly tighter than the ones of the hete­ro­scedastic method – as expected – but slightly wider. Puzzle not solved.
Since I’m dealing with HVD(P)s, where the CV jumps around like crazy between studies I prefer the heteroscedastic approach.

» OT again:

Easy.

» Thanks a lot for implementing MathJax. Equations are looking much better. one puzzle though: command \LaTeX does do what it supposed to do.

There is not \LaTex in MathJax. See there for examples (including a link to the MathJax site).

• Gwaza L, Gordon J, Welink J, Potthast H, Hansson H, Stahl M, García-Arieta A. Statistical approaches to indirectly compare bioequivalence between generics: a comparison of methodologies employing artemether / lume­fantrine 20/120 mg tablets as prequalified by WHO. Eur J Clin Pharmacol. 2012;68(12):1611–8. doi:10.1007/s00228-012-1396-1.

Cheers,
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
nobody
nothing

2019-07-22 16:38

@ Helmut
Posting: # 20427
Views: 1,673

## Indirect adjusted comparisons in BE

Dr. Gwaza is apparently affiliated to the WHO nowadays from what I found by a quick search. Maybe someone should contact him for details on the background of the pragmatic approach employed...

I find that somewhat disturbing to publish a "new" method without any substantial scientific background described and others start using this "published method" for the next few centuries. I see that quite often in recent times in various fields.

Kindest regards, nobody