Bioequivalence and Bioavailability Forum • EMA's POV on higher order designs

Helmut
★★★

Vienna, Austria,
2010-06-08 16:01
(5865 d 01:26 ago)

Posting: # 5470
Views: 5,660

EMA's POV on higher order designs [Regulatives / Guidelines]

Dear all,

we already had a lengthy discussion about the statement in Section 4.1.8 of the new BE-GL

In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA, or a four period study including test and reference in fed and fasted states), the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question.

At last week’s EGA workshop David Brown (MHRA) presented the EMA’s point of view:

[image]
[image] ]

If I apply this method to our example dataset, I get (as expected and already discovered on Feb 5^th):
Full model: PE 103.81% [90% CI: 94.38%-114.19%]
EMA method: PE 103.81% [90% CI: 95.46%-112.89%] ⇐ anticonservative (serious risk to public health)

I suspect that the EMA’s statisticians never performed simulations – or analyzed at least a single dataset.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Dr_Dan
★★

Germany,
2010-06-08 16:36
(5865 d 00:51 ago)

@ Helmut
Posting: # 5471
Views: 4,676

EMA's POV on higher order designs

Post reply

Dear Helmut
Do you know if the US FDA would accept this approach (a three period study including two references, one from EU and another from USA)? If nobody cares about statistics and analysis for each comparison is conducted excluding the data from the treatments that are not relevant (i.e. excluding EU reference), a generic company can safe a huge amount of money (especially when BE is assessed in patients).
Kind regards
Dan

—
Kind regards and have a nice day
Dr_Dan

Helmut
★★★

Vienna, Austria,
2010-06-08 16:50
(5865 d 00:37 ago)

@ Dr_Dan
Posting: # 5472
Views: 4,905

EMA vs. FDA

Post reply

Dear Dan!

❝ Do you know if the US FDA would accept this approach (a three period study including two references, one from EU and another from USA)?

I talked to Barbarba Davit (Acting Director FDA/CDER/OGD) last month in Ljubljana about this issue. Yes, FDA has accepted such studies in the past and will continue doing so. If FDA guidances are concerned, 2way-cross-over studies are recommended, but higher-order designs are not excluded. Anyhow, since the protocol has to be approved by the FDA, they will tell you whether its acceptable or not.

❝ If nobody cares about statistics and analysis for each comparison is conducted excluding the data from the treatments that are not relevant (i.e. excluding EU reference),…

Well, I wouldn’t say that “nobody cares about statistics”. FDA does. Actually Barbara was quite surprised about the “ANOVA and nothing but ANOVA”- and “only fixed effects”-story and couldn’t believe that the guideline is really final…

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Dr_Dan
★★

Germany,
2010-06-08 17:01
(5865 d 00:26 ago)

@ Helmut
Posting: # 5473
Views: 4,740

EMA vs. FDA

Post reply

Hi Helmut

❝ Well, I wouldn’t say that “nobody cares about statistics”. FDA does. Actually Barbara was quite surprised about the “ANOVA and nothing but ANOVA”- and “only fixed effects”-story and couldn’t believe that the guideline is really final…

Yes, you already told me this story in London. In the above mentioned case you should prepare two statistical analysis plans (one for EU and one for FDA), right?

❝ Anyhow, since the protocol has to be approved by the FDA, they will tell you whether its acceptable or not.

If you conduct the BE study in Canada or somewhere else you will not have a FDA approved protocol. I agree if you submit in the US you will get a reply within one month. But who wants to perform a BE study in the US?
Kind regards
Dan

—
Kind regards and have a nice day
Dr_Dan

Helmut
★★★

Vienna, Austria,
2010-06-08 17:28
(5864 d 23:59 ago)

@ Dr_Dan
Posting: # 5475
Views: 4,632

EMA vs. FDA

Post reply

Dear Dan!

❝ In the above mentioned case you should prepare two statistical analysis plans (one for EU and one for FDA), right?

If you want to follow the EMA’s method (which I would not recommend!), yes. One with the flawed method for the EU and one with the full model for the US.

❝ ❝ Anyhow, since the protocol has to be approved by the FDA, they will tell you whether it's acceptable or not.

❝

❝ If you conduct the BE study in Canada or somewhere else you will not have a FDA approved protocol. I agree if you submit in the US you will get a reply within one month. But who wants to perform a BE study in the US?

Right. In such a case I would suggest to follow the phrase which is given in numerous FDA’s guidances: “Contact FDA’s review staff”.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro
★★★

Denmark,
2010-06-08 20:12
(5864 d 21:15 ago)

@ Helmut
Posting: # 5477
Views: 4,644

EMA's POV on higher order designs

Post reply

Dear Helmut,

❝ SUB SEQ PRD FRM

❝ 1 1 1 T

❝ 1 1 2 A

❝ 2 3 1 A

❝ 2 3 2 T

❝ 3 6 2 A

❝ 3 6 3 T

❝ 4 2 1 T etc.

❝ etc.

❝ which can be analysed as usual in PROC GLM with terms for sequence, subject (sequence), period and formulation.

Isn't there something fishy here?
When we do a 2x2x2 BE study then it is common to specify that the analysis is only done on completers, i.e. subjects who have readings for both treatment periods. This is because the GLM discards every subject that has a missing value. In your data listing you have period 1..3 but no subject will have more than two readings, so effectively each subject will have a missing value.

Or have I been too long on the seven seas this time?

Best regards
EM.

Helmut
★★★

Vienna, Austria,
2010-06-08 20:25
(5864 d 21:03 ago)

@ ElMaestro
Posting: # 5478
Views: 4,672

EMA's POV on higher order designs

Post reply

Dear ElMaestro,

❝ Isn't there something fishy here?

❝ In your data listing you have period 1..3 but no subject will have more than two readings, so effectively each subject will have a missing value.

Exactly. Please note that the coding was presented by David Brown (MHRA).

❝ Or have I been too long on the seven seas this time?

No!

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro ★★★ Denmark, 2010-06-08 21:15 (5864 d 20:13 ago) @ Helmut Posting: # 5480 Views: 4,628	EMA's POV on higher order designs Post reply
	Dear Helmut, ❝ Exactly. Please note that the coding was presented by David Brown (MHRA). But I understood you obtained a CI with this coding nevertheless. Could you provide details? Best regards and many thanks, EM. — Pass or fail! ElMaestro

Helmut
★★★

Vienna, Austria,
2010-06-09 19:00
(5863 d 22:27 ago)

@ ElMaestro
Posting: # 5487
Views: 4,581

Example data-set

Post reply

Dear ElMaestro,

❝ But I understood you obtained a CI with this coding nevertheless. Could you provide details?

well, I ran a mixed model (I can’t afford SAS – therefore no GLM). I understand your concerns about incomplete data in GLM. I used the old data-set and deleted all occurances of R2:

subject sequence period treatment response1 response2

   1       1       1       T       120.60    120.60

   1       1       2       R1       81.56     81.56

   2       2       1       R1       96.86     96.86

   2       2       3       T        95.09    105.09

   3       3       2       T        82.46     82.46

   3       3       3       R1      101.30    111.30

   4       4       1       T       114.40    114.40

   4       4       3       R1       89.56     99.56

   5       5       1       R1       99.64     99.64

   5       5       2       T        97.84     97.84

   6       6       2       R1       72.80     72.80

   6       6       3       T        74.43     84.43

   7       1       1       T        76.64     76.64

   7       1       2       R1       83.70     83.70

   8       2       1       R1       86.61     86.61

   8       2       3       T        84.64     94.64

   9       3       2       T       104.80    104.80

   9       3       3       R1       87.69     97.69

  10       4       1       T        96.65     96.65

  10       4       3       R1       96.99    106.99

  11       5       1       R1       96.53     96.53

  11       5       2       T        89.58     89.58

  12       6       2       R1       94.51     94.51

  12       6       3       T        98.80    108.80

See here for instructions to copy-paste values.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2010-06-16 15:37
(5857 d 01:50 ago)

@ Helmut
Posting: # 5528
Views: 4,464

SASophylistic EMA results

Post reply

Dear Helmut, dear ElMaestro,

❝ well, I ran a mixed model (I can’t afford SAS – therefore no GLM).

as a rich man I can GLM :-D

.

So, if somebody only trusts [image]

:
logR1 is log(response1).

Full dataset, response1



The GLM Procedure (all effects fixed!)



Dependent Variable: logR1

                                   Sum of

Source                    DF       Squares   Mean Square  F Value  Pr > F



Model                     15    0.57497834    0.03833189     2.10  0.0615

Error                     20    0.36586207    0.01829310     CV=13.59%

Corrected Total           35    0.94084041



R-Square     Coeff Var      Root MSE    logR1 Mean

0.611133      2.951032      0.135252      4.583211



Source                    DF   Type III SS   Mean Square  F Value  Pr > F



treatment                  2    0.28335528    0.14167764     7.74  0.0032

period                     2    0.02065586    0.01032793     0.56  0.5774

sequence                   5    0.04117234    0.00823447     0.45  0.8082

subject(sequence)          6    0.22979487    0.03829915     2.09  0.0997





                                            Standard

Parameter                   Estimate           Error    t Value    Pr > |t|

T-R1                      0.03740883      0.05521640       0.68      0.5059

T-R2                     -0.16668664      0.05521640      -3.02      0.0068



Parameter                 90% Confidence Limits

T-R1                     -0.05782390   0.13264155

T-R2                     -0.26191937  -0.07145392



back-transformed 90% CI



       point      lower      upper

T-R1  1.03812    0.94382    1.14184

T-R2  0.84646    0.76957    0.93104



EMA method, response1 (neglecting R2)



The GLM Procedure



Dependent Variable: logR1

                                   Sum of

Source                    DF       Squares   Mean Square  F Value  Pr > F



Model                     14    0.24412245    0.01743732     1.39  0.3153

Error                      9    0.11294731    0.01254970     CV=11.24%

Corrected Total           23    0.35706976



R-Square     Coeff Var      Root MSE    logR1 Mean

0.683683      2.477664      0.112025      4.521414



Source                    DF   Type III SS   Mean Square  F Value  Pr > F



treatment                  1    0.00839652    0.00839652     0.67  0.4345

period                     2    0.03068207    0.01534103     1.22  0.3391

sequence                   5    0.04852972    0.00970594     0.77  0.5923

subject(sequence)          6    0.14060220    0.02343370     1.87  0.1919





                                            Standard

Parameter                   Estimate           Error    t Value    Pr > |t|

T-R1                      0.03740883      0.04573420       0.82      0.4345



Parameter                 90% Confidence Limits

T-R1                     -0.04642712   0.12124478



back-transformed 90% CI



       point      lower      upper

T-R1  1.03812    0.95463    1.12890

Dear Helmut, interpret I your sentence above

❝ If you want to follow the EMA’s method (which I would not recommend!)

in the answer to Dan right if I assume you would recommend the full model / dataset for evaluation even if the study is targeted to EMA? Who is afraid of EMA?

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2010-06-16 16:18
(5857 d 01:09 ago)

@ d_labes
Posting: # 5530
Views: 4,440

SASophylistic EMA results

Post reply

Dear D. Labes!

THX for the comparison; differences* to WinNonlin’s results only in the last decimal place…

Maybe some SAS-believing regulator reads your results. Excuse my bold format:

Smaller CV, tighter CI, anticonservative, liberal,
serious risk to the public health.

❝ Dear Helmut, interpret I your sentence above

❝ ❝ If you want to follow the EMA’s method (which I would not recommend!) in the answer to Dan right if I assume you would recommend the full model / dataset for evaluation even if the study is targeted to EMA? Who is afraid of EMA?

It depends whether you have guts or not. I would not use amputee’s statistics – and make the rationale clear in the protocol. It's one of the basic assumptions in a non-replicated cross-over study to have IDD – or a common variance. If the authority rejects to approve the protocol (based on the GL), it’s their responsibility to increase patient’s risk in an unpredictable manner. Dan’s example would call for two SAPs (correct model for FDA and beheaded ‘model’ for EMA), IMHO.

In the meantime I still hope for some kind of clarification (a Q&A or whatsoever) from the almighty oracle EMA.

Edit 2012-07-18: Identical results for response1 compared to SAS in Phoenix/WinNonlin 6.3 with all fixed effects:
Model PE 90% CI CV Full 1.03812 0.94382 – 1.14184 0.13587 EMA 1.03812 0.95463 – 1.12890 0.11238
Analysis of response2 (period effects):
Model PE 90% CI CV Full 1.04055 0.94932 – 1.14054 0.13087 EMA 1.04055 0.95834 – 1.12980 0.11030

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes