Bioequivalence and Bioavailability Forum

Main page Policy/Terms of Use Abbreviations Latest Posts

 Log-in |  Register |  Search

Back to the forum  Query: 2017-11-24 06:45 CET (UTC+1h)
 
Helmut
Hero
Homepage
Vienna, Austria,
2010-06-22 18:53

Posting: # 5551
Views: 15,844
 

 EMA BE-GL: Clarifications / Corrections? [BE/BA News]

Dear all,

according to rumours at the recent Workshop in Budapest (235 participants!) EMA started an initiative involving the PK-Drafting Group and the Biostatistics Drafting Group of EWP to clarify/correct some open issues of the new BE-GL.
Points under discussion are
  • “Only fixed effects ANOVA” in replicate designs. RMLE under consideration.
  • Outliers. MAD of R-R will be an option.
  • The “T-R1-R2” issue.
We can expect not only SAS-code, but example data-sets in order to validate other software.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
Hero

Berlin, Germany,
2010-06-23 10:09

@ Helmut
Posting: # 5552
Views: 14,179
 

 Good news!?

Dear Helmut,

seems this are good news.

Hopefully the results are not according to:


"Der Berg kreißte und gebar eine Maus."
(The mountain labored and brought forth a mouse.)


Regards,

Detlew
Helmut
Hero
Homepage
Vienna, Austria,
2010-06-23 10:11

@ d_labes
Posting: # 5553
Views: 14,210
 

 Good news!?

Dear D. Labes!

» seems this are good news.

Well, let’s see and cross our fingers. ;-)

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Helmut
Hero
Homepage
Vienna, Austria,
2011-02-05 18:48

@ Helmut
Posting: # 6562
Views: 13,696
 

 Update

Dear all,

the document is expected to be published in mid-February 2011.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-16 13:44

@ Helmut
Posting: # 6762
Views: 13,523
 

 Q&A published 14 March 2011

Dear all,

the clarification is part of the recent version of the “Questions & Answers: Positions on specific questions addressed to the Pharmacokinetics Working Party” EMA/618604/2000 Rev. 3, dated 26 January 2011, published 14 March 2011.
The interesting part is Section 11. Clarification on the recommended statistical method for the analysis of a bioequivalence study (pages 21–32).

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
Hero

Denmark,
2011-03-16 14:20

@ Helmut
Posting: # 6763
Views: 13,413
 

 Q&A published 14 March 2011

Hehe,

A simple linear mixed model, which assumes identical within-subject variability (Method B), may be acceptable as long as results obtained with the two methods do not lead to different regulatory decisions. However, in borderline cases and when there are many included subjects who only provide data for a subset of the treatment periods, additional analysis using method A might be required.
At the time of protocol writing you do not know if there will be data gaps. Therefore, you cannot write which evaluation method you will ultimately be using. This means (??) that one has to apply both methods, check if the BE conclusions are the same and then, in case there are differences, discard a method which gives unbiased variance estimates.

An advantage of Method C is that it directly calculates s2wr However, sometimes the algorithm fails to converge.
No, the Al Gore Rhythm will converge if the guy doing the statistics knows what he is doing in terms of controlling the ini-values and other optimizer settings (flat multidimensional likelihood surfaces are hypothetical).
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-17 04:23

@ Helmut
Posting: # 6766
Views: 13,585
 

 Phoenix/WinNonlin 6.1.0.173

Dear all,

I could reproduce EMA’s results in Phoenix. Interesting points for Dataset II:
  • Method C (FDA’s 2001 Appendix E)
    Warning: Newton's algorithm converged with modified Hessian. Output is suspect.
    Model may be over-specified. A simpler model could be tried.

  • A result for CVWT of 8.65%… T is not replicated – we had that already; too lazy to search now.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
Hero

Berlin, Germany,
2011-03-17 11:00
(edited by d_labes on 2011-03-17 14:06)

@ Helmut
Posting: # 6770
Views: 13,396
 

 ANOVA party prevails

Dear All,

that makes me dumbfound! :crying:
Seems the ANOVA fraction of EMA statisticians has triumphed all along the line. Thus our crossing fingers was of no effect.

To summarize my understanding of this so-called "clarification":
  • Method C, the FDA approach, although giving more conservative CIs (Quote: "... Method C gives wider intervals ...", page 23) is dead. It is not named "Compatible with CHMP guideline". Thus we can not go with the same statistical evaluation for the FDA and EMA!
  • Method B, simple mixed model assuming equal variabilities for Test and Reference and no formulation-by-subject interaction but random effect for subjects, acceptable if same results as Method A. For me it follows I have to use Method A.
    Quote: "... in borderline cases and when there are many included subjects who only provide data for a subset of the treatment periods, additional analysis using method A might be required.", page 25.
  • Method A, ANOVA assuming equal variabilities for Test and Reference and no formulation-by-subject interaction and all effects as fixed (including subjects!), is the method of choice to evaluate replicate cross-over studies, which are originally aimed to overcome the impossibility to estimate separate intra-subject variances for Test and Reference in a classical 2x2 cross-over.
  • To overcome the flaw in the point above one has to evaluate the intra-subject variability for the Reference with neglecting a considerable part of the data, namely neglecting all data under Test.
  • Evaluation of intra-subject variability of Test is not necessary at all (is not mentioned with any word), even in the case of a fully replicate design (data set I).
  • Missing data will be handled adequately by the simple ANOVA(?). They termed that "unbalanced" in dataset I.
Do you think I have got their points?

The rationale behind that all I can't and will not discuss seriously :no:.
I thank my God that I'm only a quantum-theoretical chemist educationally and not a statistician. Thus I must not understand :cool:.

BTW: The CVWT of Method C from SAS for dataset II is 3.87%.
Proc MIXED is complaining: The Mixed Procedure
Convergence criteria met but final hessian is not positive definite.

This is a strong sign of an over-specified model. That may be one of the sources of wider CIs compared to the simpler models.

Regards,

Detlew
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-19 02:59

@ d_labes
Posting: # 6779
Views: 13,348
 

 THX!

Bless you, sir, and all your house, unto the seventh generation!
You are a most noble and shining example of all that's right and good and true here.


» Do you think I have got their points?

Yes. Exactly. :crying:


Edit: Data Set I is funny. Some subjects have missing data in one period, but data in a subsequent one (e.g., subject 11’s third period is missing). Obviously not data from the ‘real world’. Did you have a look at box plots and/or QQ-plots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?
Quote from last June’s Q&A:

On a case by case basis, a study could be acceptable if the bioequivalence requirements are met both including the outlier subject (using the scaled average bioequivalence approach and the within-subject CV with this subject) and after exclusion of the outlier (using the within-subject CV without this subject).
An outlier test is not an expectation of the medicines agencies but outliers could be shown by a box plot. This would allow the medicines agencies to compare the data between them.


Excluding subjects 45 and 52 CVWR (EMA’s method) drops from 47.0% to 32.2%. Maybe that’s a hidden trick question to us:
                     Scaled  AR      width
Full data set:     71.23 - 140.40    69.17
Outliers excluded: 78.79 - 126.93    48.14

The 90% CIs of the full data set of 107.11 - 124.89 (Method A) and 107.17 - 124.97 (Method B) are also within the narrower scaled AR – but it’s a pretty close shave!

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
Hero

Berlin, Germany,
2011-03-24 11:55

@ Helmut
Posting: # 6803
Views: 13,036
 

 Outlier

Dear Helmut,

» "Bless you, sir, and all your house, unto the seventh generation!
» You are a most noble and shining example of all that's right and good and true here."
This quote, or is it your own poetry, is really :google: resistant. First time
that I noticed the Net is not omniscient!

» ... Did you have a look at box plots and/or QQ-plots of residuals?
» What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?

Which residuals did you analyse? Which model? Please enlighten me.
What do you think: which type of boxplot should we apply? Best for our sponsors would be a simple boxplot with whiskers up to minimum/maximum not showing any 'outlier' :-D.

Regards,

Detlew
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-24 15:09

@ d_labes
Posting: # 6806
Views: 13,267
 

 Outliers - yes, but how?

Dear D. Labes!

» This quote, or is it your own poetry, …

I borrowed it from a personal discussion page at Wikipedia.

» … is really google resistant. First time that I noticed the Net is not omniscient!

Well, personal discussion pages of WP are blocked from being indexed by Google. I have blocked some pages (i.e., the download section and the latest posts) from Google as well. Requires two lines in the domain’s robots.txt
User-agent: *
Disallow: /foo/

… and adding rel="nofollow" to respective links within other pages (the same is done here; look at the HTML code).

» » … Did you have a look at box plots and/or QQ-plots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?
»
» Which residuals did you analyse? Which model? Please enlighten me.

I used EMA’s crippled model (test removed), Sequence+Subject(Sequence)+Period and threw away one period’s residuals (same values, but different in signs to the respective other one).

» What do you think: which type of boxplot should we apply? Best for our sponsors would be a simple boxplot with whiskers up to minimum/maximum not showing any 'outlier' :-D.

:rotfl:

I have chosen ±3×IQR following the convention (!) that values within 1.5-3×IQR are ‘mild’ outliers and outside 3×IQR are ‘severe’ outliers. I’m exploring full replicate data-sets right now – outliers almost in all of them. Don’t know whether residuals make any sense at all (see also this post: period ratios instead?). I would end up with different numbers of outliers, depending on the method of calculation of the IQR (see R-manual). Period ratios (second / first administration):
Type 3: SAS according to R-doc - 2 outliers (#46: 3.524, #45: 26.08)
Type 5: or is this SAS? - 2 outliers
Type 6: Minitab, SPSS, Phoenix/WinNonlin - 2 outliers
Type 7: default in S, R - 3 outliers (as above + #13: 3.349)

                               CVWR    Scaled  AR      width
Full data set:                 47.0  71.23 - 140.40    69.17
#45, #52 excluded (residuals): 32.2  78.79 - 126.93    48.14
#45, #46 excluded (ratios):    35.3  77.08 - 129.73    52.65


Don’t know what to do. :confused: Suggestions?
Maybe Q&A is an abbreviation for Questions and Ambiguities.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
Hero

Berlin, Germany,
2011-03-28 14:16

@ Helmut
Posting: # 6822
Views: 12,925
 

 Questions and Ambiguities

Dear Helmut,

» Period ratios (second / first administration):
» Type 3: SAS according to R-doc - 2 outliers (#46: 3.524, #45: 26.08)
» Type 5: or is this SAS? - 2 outliers
» Type 6: Minitab, SPSS, Phoenix/WinNonlin - 2 outliers
» ...

SAS has 5 different percentile definitions. The default is:
Let n*p=j+g where j is the integer part, g is the fractional part, n is the number of values, x the ordered values.
Let y denote the percentile. Then (SAS PCTLDEF=5)
  y = 0.5*(xj+xj+1) if g=0
  y = xj           if g>0

This corresponds to R's Type 2 I think.

» Don’t know what to do. :confused: Suggestions?

If the 'outlier' considerations based on the crippled EMA model makes any sense at all, which I'm not convinced at all :no:, I would vote for an analysis of the residuals, not the period ratios also they appeared a natural choice on the first view.
Pro's:
  • Period effects accounted for
  • Analysis within the log domain (deemed necessary for AUC, Cmax)
  • Distribution hopefully nearly normal, nearly symmetric because of the previous point, such that the 3xIQR rule can be applied reasonably
  • Independent of order (first/second or second/first)
Con's:
  • Period effects different from the whole model, because of throwing away all data under Test, residuals therefore may be biased
  • Residuals not independent because of the model fitted (2 values of residual on a subject only different in sign, must throw away 1/2 of them)
But these Con's are unavoidable because of the EMA 'model', whatever here is modelled.
To add more ambiguities:
  • simple residuals?
  • studentized residuals?
  • external studentized residuals (leave-one-out residuals)?
  • other influence statistics (e.g. Cook's distance)?

Regards,

Detlew
d_labes
Hero

Berlin, Germany,
2011-04-04 08:53

@ Helmut
Posting: # 6855
Views: 13,089
 

 Residuals and Outliers in Replicate Design Crossover Studies

Dear Helmut, dear All!

FYI:

Robert Schall, Laszlo Endrenyi, Arne Ring
Residuals and Outliers in Replicate Design Crossover Studies
Journal of Biopharmaceutical Statistics, 20: 4, 835 — 849

Online accessible here as preprint or here.

IMHO this fits excellent into the framework of scaled ABE evaluated via suitable intra-subject contrasts as outlined in the Progesterone guidance :ok:.

Regards,

Detlew
ElMag
Junior

2011-03-24 12:45
(edited by ElMag on 2011-03-24 14:44)

@ Helmut
Posting: # 6805
Views: 13,154
 

 Info requested

» Did you have a look at box plots and/or QQ-plots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?

Hello there!!
Can you tell me if you have created the box-plots based on the ratio of Cmax between the two periods? If not, please indicate which data have you selected and how did you use them in order to create the box-plot.

Thank you!
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-24 16:50

@ ElMag
Posting: # 6808
Views: 13,053
 

 Confused as well...

Dear ElMag!

See above. Don’t know whether this makes sense at all. :confused:

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Priyanka_S
Junior

2011-03-21 14:28

@ Helmut
Posting: # 6789
Views: 13,246
 

 Q&A published 14 March 2011

Dear all,

We tried the SAS code given by EMA for replicated designs by using method A with data set II (three period data). But it is showing the following error. Please clarify this.

WARNING: ADJUST=T implies no adjustment for simultaneous inference.

Best Regards

Priyanka S
d_labes
Hero

Berlin, Germany,
2011-03-21 16:08
(edited by d_labes on 2011-03-21 16:21)

@ Priyanka_S
Posting: # 6790
Views: 14,009
 

 SAS code: Warning

Dear Priyanka,

» ... But it is showing the following error ...
» WARNING: ADJUST=T implies no adjustment for simultaneous inference.

as the log states this is not an ERROR:, but a WARNING:. And this warning is nonsense in the context.
You request with the option ADJUST=T confidence intervals using the t-quantile without eventually necessary multiplicity adjustments.
But multiplicity adjustment is not necessary in comparing only two possible LSMeans. It would be eventually necessary if there are more than two. Proc GLM had could that figured out, but SAS in his great wisdom had decided nevertheless to throw a warning.
Note that this warning disappears if you omit the ADJUST=T option in the LSMeans statement. Doing so Proc GLM reverts to the default method for CI calculation: confidence intervals using the t-quantile without eventually necessary multiplicity adjustments, i.e. the same as with ADJUST=T, but without a warning. What a stroke of genius :cool:.

This annoying warningo-phylistic behaviour is notorious in SAS 9.2. You can observe it in numerous instances. Check if SAS does what you intend and forget the warnings if it does.

BTW: The WARNING: has nothing to do with the data set used. You will get it also with data set I.

Regards,

Detlew
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-21 22:38

@ d_labes
Posting: # 6791
Views: 13,200
 

 Copy & paste

Dear D. Labes!

I'm not gifted with [image], but I'm wondering how EMA got their results for 'Method C' with this statement:
  estimate 'test-ref' formulation -1 1/ CL alpha=0.10; :cool:

@ Priyanka:
If you want to repeat results from any type of software, first use the code exactly as it is given. Only if your results don't match, try to modify your code (sometimes published code contains typos...).
  ADJUST=T is not stated by FDA (2001), EMA's Q&A (2011), and FDA's Progesterone Guidance (2010, 2011).

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
Hero

Berlin, Germany,
2011-03-22 09:24

@ Helmut
Posting: # 6794
Views: 13,242
 

 Subtleties, flaws, questions

Dear Helmut!

» ... but I'm wondering how EMA got their results for 'Method C' with this statement: estimate 'test-ref' formulation -1 1/ CL alpha=0.10;

This is one of the hidden gems in SAS coding :-D. But it is correct if you use the codes 'R' and 'T' for the formulations. SAS orders them lexically 'R' coming first and expects the coefficients for the difference in the estimate statement in that order.

» @ Priyanka:
» ... ADJUST=T is not stated by FDA (2001), EMA's Q&A (2011), and FDA's Progesterone Guidance (2010, 2011).

Here the Great Admin err's. It is stated in EMA's Q&A Method 1:
proc glm data=replicate;
  class formulation subject period sequence;
  model logDATA= sequence subject_(sequence) period formulation;
  *the space above may lead to hard to discover errors!;
  estimate "test-ref" formulation -1+1;
  test h=sequence e=subject(sequence);
  lsmeans formulation / adjust=t pdiff=control("R") CL alpha=0.10;
run;


But as I said above, it is not necessary here. Also the pdiff=control("R") is not necessary. But it does here the job of ordering T first and thus giving the LSMeans difference T-R because again the LSMean of 'R' comes first and the difference 1-2 is calculated.
Moreover the whole lsmeans statement is superfluous if you code the option /CLparm alpha=0.1 in the model statement. Then the estimate statement will give the CI like Proc MIXED does as default.

BTW: I'm not sure if the test of the sequence effect as coded from Great Oracle EMA is appropriate in case of missings. Using the 'Capt'n EM calls me bogus' Random statement I get a mixture of MSerror and MSsubject(sequence) as denominator of the corresponding F-test. The degrees of freedom are also adapted according to Satterthwaite.

BTW2: The code for obtaining the intra-subject variance taken literally will bring us directly to the Type III hell. Output (I have named their DATA as AUC):
--- GLM-ANOVA Analysis of REF. within-subject var. for log(AUC) ---

The GLM Procedure

Dependent Variable: logval   log(AUC)

                                Sum of
Source                DF       Squares    Mean Square   F Value   Pr > F

Model                 78   120.2314511      1.5414289      7.73   <.0001
Error                 71    14.1512621      0.1993136
Corrected Total      149   134.3827132


Source                DF     Type I SS    Mean Square   F Value   Pr > F

sequence               1     0.0880117      0.0880117      0.44   0.5085
subject(sequence)     75   119.3707667      1.5916102      7.99   <.0001
period                 2     0.7726727      0.3863363      1.94   0.1515


Source                DF   Type III SS    Mean Square   F Value   Pr > F

sequence               0     0.0000000       .              .      .
subject(sequence)     75   118.6616824      1.5821558      7.94   <.0001
period                 2     0.7726727      0.3863363      1.94   0.1515


As you see: Subtleties and flaws in coding, questions over questions after that 'Clarification'.
But what could we expect other if considering the scientific foundation of that all?

Regards,

Detlew
Helmut
Hero
Homepage
Vienna, Austria,
2011-03-27 20:35

@ d_labes
Posting: # 6818
Views: 12,979
 

 Subtleties, flaws, questions

Dear D. Labes!

» Source         DF   Type III SS   Mean Square  [...]
»
» sequence        0    0.0000000    .            [...]
» [...]

Zero degrees of freedom is not a lot.
I love SAS‘s ‘.’ - Phoenix/WinNonlin spits out ‘Not estimable’, but I can trim it to ‘.’ or even ‘0’… :-D

Open issues IMHO:
  • Outliers – which method? Model residuals or R/R ratios?
  • Obviously EMA is not interested in CVWT (Method C) at all. Since GLM throws away incomplete data (Method A), why should one go with a full replicate design? More periods, more chances of drop-outs.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
The user
Junior

Russia,
2017-03-14 10:16
(edited by The user on 2017-03-14 12:44)

@ Helmut
Posting: # 17151
Views: 2,642
 

 Not estimable in the model

» I love SAS‘s ‘.’ - Phoenix/WinNonlin spits out ‘Not estimable’, but I can trim it to ‘.’ or even ‘0’… :-D

Dear Helmut,

I decided to continue the topic (moreover Simon kindly directed me here).
Regarding "Not estimable". Could you please share your expirience with following issue? When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapears. Comments: it is BE cross over study in 3 groups and all effects are fixed.

BR
ElMaestro
Hero

Denmark,
2017-03-14 11:02

@ The user
Posting: # 17152
Views: 2,629
 

 Not estimable in the model

Hi The user,

» I decided to continue the topic (moreover Simon kindly directed me here).
» Regarding "Not estimable". Could you please share your expirience with following issue? When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapeares. Comments: it is BE cross over study in 3 groups and all effects are fixed.

Hmmmm... thinking loud here... In a type III model Group will have in your case up to three columns in the model matrix, but every column is equally represented by the subjects in the respective groups. Hence Group has zero df's and no effect for any level of Group can be estimated.

Try a type I model with Group before Subjects, then you'll get something that is estimable, I believe.

I could be wrong, but…


Best regards,
ElMaestro

No, I still don't believe much in the usefulness of IVIVCs for OIPs when it comes to picking candidate formulations for the next trial. This is not the same as saying I don't believe in IVIVCs.
The user
Junior

Russia,
2017-03-14 12:13

@ ElMaestro
Posting: # 17154
Views: 2,604
 

 Not estimable in the model

» Try a type I model with Group before Subjects, then you'll get something that is estimable, I believe.

Thank you for your reply.

Sorry, but I did not catch the idea. Should I try this model: Group+Patient(Sequence)+Sequence+Period+Treatment?

One more comment: the groups are unbalanced. Type I is suiatable for balanced groups as I undertood. Looks like I could not use the Type I model.
ElMaestro
Hero

Denmark,
2017-03-14 12:41

@ The user
Posting: # 17155
Views: 2,606
 

 Not estimable in the model

Hi,

» Sorry, but I did not catch the idea. Should I try this model: Group+Patient(Sequence)+Sequence+Period+Treatment?

Yes try that with type I. I think type III may give the same as before if you are not using SAS.


» One more comment: the groups are unbalanced. Type I is suiatable for balanced groups as I undertood. Looks like I could not use the Type I model.

Your CI will be the same. Type I vs Type III is generally a topic that is of a much more sensitive nature to some people than e.g.religion or venereal diseases. People who grew up with SAS stick to type III, and type III only, because that is all they know and therefore they seem to be resistant to common sense. Besides, SAS invented the term "Least Squares Means" and that sounds so good that no reasonable alternative could ever exist, right?

Type I is not better or worse than type III. LS Means are no better than model effects. Depending on contrasts, model effects are LS Means and vice versa. And so forth...

I could be wrong, but…


Best regards,
ElMaestro

No, I still don't believe much in the usefulness of IVIVCs for OIPs when it comes to picking candidate formulations for the next trial. This is not the same as saying I don't believe in IVIVCs.
Helmut
Hero
Homepage
Vienna, Austria,
2017-03-18 21:59

@ The user
Posting: # 17164
Views: 2,439
 

 Not estimable in the model

Hi BR,

» When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapears. Comments: it is BE cross over study in 3 groups and all effects are fixed.

As Simon noted already at Certara’s Forum, its the fixed effects model – not the software. Even without the group-term (i.e., the EMA’s sequence, subject(sequence), period, formulation) you will get an endless list of “Not estimables” simply because the requested combination does not exist in the data set. Example: All subjects are uni­quely coded (1, 2, …, n) and subject 1 is in sequence RT. You will get an estimate. Fine. But the model tries to estimate subject 1 in sequence TR as well. Not estimable! That’s correct because a datum with such a coding does not exist.

Since you are posting from Russia what do you want to achieve? Satisfy the «Экспертами» (see this post)? Every time I was in Moscow we had endless & fruitless debates about it… All relevant documents (2008 GL, 2013 “Red Book”, 2015 EEU GL) are more or less translations of the EMA’s GL (the EEU GL spiced with some parts of the WHO’s GL). What does the EMA’s GL say about groups (or more important sites)? Nothing! Only:

The precise model to be used for the analysis should be pre-specified in the protocol. The statistical analysis should take into account sources of variation that can be reasonably assumed to have an effect on the response variable.

Is it reasonable to assume such an effect if a study was performed in multiple groups due to logistic reasons (e.g., limited capacity of the clinical site)? I don’t think so. Hence, in the EU generally data are simply pooled and the common model (without a group term) is used.
Different sites are much more problematic. I recently saw a multi-site study where the sites clearly showed different results (averages differed tenfold). It was a cancer drug and some sites were pretty small. If (if!) all sites would have balanced sequences it would have been still no problem but this was not the case. Actually there was a highly significant (p <0.001) site-by-treatment interaction. If one would naïvely pool the sites the treatment effect would be biased.

OK, back to the EEU GL (the last paragraph of section 94):

Если предполагается проведение исследования в нескольких группах из логистических соображений, об этом необходимо явно указать в прото­коле исследования; при этом, если в отчете отсутствуют результаты статис­тического анализа, учитывающие многогрупповой характер исследования, необходимо представить научное обоснование отсутствия таких результатов.


My interpretion:
  1. State already in the protocol that the study will be performed in multiple groups and give a justification that an effect on the treatment comparison can be reasonably ruled out (i.e., same site, same procedures, all subjects randomized before splitting, same batches of T and R, short time interval between groups, blahblah).
  2. If you want to go the hard way: Modify the FDA’s multi-group models (i.e., mixed-effects ⇒ all effects fixed). More about that later.
I would always try #1. Whether the experts will swallow that is another story.
#2 can be nasty! Start with “Model 1” (fixed effects* in Phoenix-notation):

Group + Sequence + Sequence(Group) + Period(Group) + Treatment +
Treatment * Group + Subject(Sequence * Group)

  1. If the term Treatment * Group (the treatment-by-group interaction) is not significant at the 0.1 (!) level remove this term and perform the analysis by “Model 2”. The between-group test is not very sensitive (sloppy: has low power). Therefore, the FDA requires testing at 0.1 (and not at 0.05).
  2. If the test is significant, you are not allowed to pool the data and can only run the conventional model with the data of the largest group. Good luck! The loss of power likely will be extreme. Furthermore, you could expect false positives – and consequently throw away 10% of your studies…
“Model 2”:

Group + Sequence + Sequence(Group) + Period(Group) + Treatment +
Subject(Sequence*Group)


Yes, you will see an awful lot of “Not estimables”. :-D


  • I have seen a data set (three groups: 19, 19, 18) where both fixed effects models did not converge (neither in Phoenix nor in SAS/JMP). No problem with the mixed-effects models (i.e., Subject(Sequence*Group) random)… The statistician of the MHRA accepted it.

[image]Regards,
Helmut Schütz 
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Back to the forum Activity
 Thread view
Bioequivalence and Bioavailability Forum | Admin contact
17,494 Posts in 3,746 Threads, 1,087 registered users;
33 users online (0 registered, 33 guests).

The belief that there is only one truth and
that oneself is in possession of it,
seems to me the deepest root of all
that is evil in the world.    Max Born

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
XHTML/CSS RSS Feed