Bioequivalence and Bioavailability Forum

Helmut
★★★

Vienna, Austria,
2010-06-22 20:53
(5217 d 01:05 ago)

Posting: # 5551
Views: 40,155

EMA BE-GL: Clarifications / Corrections? [BE/BA News]

Dear all,

according to rumours at the recent Workshop in Budapest (235 participants!) EMA started an initiative involving the PK-Drafting Group and the Biostatistics Drafting Group of EWP to clarify/correct some open issues of the new BE-GL.
Points under discussion are

“Only fixed effects ANOVA” in replicate designs. RMLE under consideration.
Outliers. MAD of R-R will be an option.
The “T-R₁-R₂” issue.

We can expect not only SAS-code, but example data-sets in order to validate other software.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes ★★★ Berlin, Germany, 2010-06-23 12:09 (5216 d 09:49 ago) @ Helmut Posting: # 5552 Views: 36,691	Good news!? Post reply
	Dear Helmut, seems this are good news. Hopefully the results are not according to: "Der Berg kreißte und gebar eine Maus." (The mountain labored and brought forth a mouse.) — Regards, Detlew

Helmut ★★★ Vienna, Austria, 2010-06-23 12:11 (5216 d 09:47 ago) @ d_labes Posting: # 5553 Views: 36,714	Good news!? Post reply
	Dear D. Labes! ❝ seems this are good news. Well, let’s see and cross our fingers. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes

Helmut ★★★ Vienna, Austria, 2011-02-05 19:48 (4989 d 01:10 ago) @ Helmut Posting: # 6562 Views: 36,255	Update Post reply
	Dear all, the document is expected to be published in mid-February 2011. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes

Helmut
★★★

Vienna, Austria,
2011-03-16 14:44
(4950 d 06:15 ago)

@ Helmut
Posting: # 6762
Views: 35,965

Q&A published 14 March 2011

Post reply

Dear all,

the clarification is part of the recent version of the “Questions & Answers: Positions on specific questions addressed to the Pharmacokinetics Working Party” EMA/618604/2000 Rev. 3, dated 26 January 2011, published 14 March 2011.
The interesting part is Section 11. Clarification on the recommended statistical method for the analysis of a bioequivalence study (pages 21–32).

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro
★★★

Denmark,
2011-03-16 15:20
(4950 d 05:39 ago)

@ Helmut
Posting: # 6763
Views: 35,815

Q&A published 14 March 2011

Post reply

Hehe,

A simple linear mixed model, which assumes identical within-subject variability (Method B), may be acceptable as long as results obtained with the two methods do not lead to different regulatory decisions. However, in borderline cases and when there are many included subjects who only provide data for a subset of the treatment periods, additional analysis using method A might be required.
At the time of protocol writing you do not know if there will be data gaps. Therefore, you cannot write which evaluation method you will ultimately be using. This means (??) that one has to apply both methods, check if the BE conclusions are the same and then, in case there are differences, discard a method which gives unbiased variance estimates.

An advantage of Method C is that it directly calculates s²_wr However, sometimes the algorithm fails to converge.
No, the Al Gore Rhythm will converge if the guy doing the statistics knows what he is doing in terms of controlling the ini-values and other optimizer settings (flat multidimensional likelihood surfaces are hypothetical).

Helmut
★★★

Vienna, Austria,
2011-03-17 05:23
(4949 d 15:35 ago)

@ Helmut
Posting: # 6766
Views: 36,330

Phoenix/WinNonlin 6.1.0.173

Post reply

Dear all,

I could reproduce EMA’s results in Phoenix. Interesting points for Dataset II:

Method C (FDA’s 2001 Appendix E)
Warning: Newton's algorithm converged with modified Hessian. Output is suspect. Model may be over-specified. A simpler model could be tried.
A result for CV_WT of 8.65%… T is not replicated – we had that already; too lazy to search now.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2011-03-17 12:00
(4949 d 08:58 ago)

(edited by d_labes on 2011-03-17 14:06)
@ Helmut
Posting: # 6770
Views: 36,142

ANOVA party prevails

Post reply

Dear All,

that makes me dumbfound! :crying:

Seems the ANOVA fraction of EMA statisticians has triumphed all along the line. Thus our crossing fingers was of no effect.

To summarize my understanding of this so-called "clarification":

Method C, the FDA approach, although giving more conservative CIs (Quote: "... Method C gives wider intervals ...", page 23) is dead. It is not named "Compatible with CHMP guideline". Thus we can not go with the same statistical evaluation for the FDA and EMA!
Method B, simple mixed model assuming equal variabilities for Test and Reference and no formulation-by-subject interaction but random effect for subjects, acceptable if same results as Method A. For me it follows I have to use Method A.
Quote: "... in borderline cases and when there are many included subjects who only provide data for a subset of the treatment periods, additional analysis using method A might be required.", page 25.
Method A, ANOVA assuming equal variabilities for Test and Reference and no formulation-by-subject interaction and all effects as fixed (including subjects!), is the method of choice to evaluate replicate cross-over studies, which are originally aimed to overcome the impossibility to estimate separate intra-subject variances for Test and Reference in a classical 2x2 cross-over.
To overcome the flaw in the point above one has to evaluate the intra-subject variability for the Reference with neglecting a considerable part of the data, namely neglecting all data under Test.
Evaluation of intra-subject variability of Test is not necessary at all (is not mentioned with any word), even in the case of a fully replicate design (data set I).
Missing data will be handled adequately by the simple ANOVA(?). They termed that "unbalanced" in dataset I.

Do you think I have got their points?

The rationale behind that all I can't and will not discuss seriously :no:

.
I thank my God that I'm only a quantum-theoretical chemist educationally and not a statistician. Thus I must not understand :cool:

.

BTW: The CV_WT of Method C from SAS for dataset II is 3.87%.
Proc MIXED is complaining:

 The Mixed Procedure

Convergence criteria met but final hessian is not positive definite.

This is a strong sign of an over-specified model. That may be one of the sources of wider CIs compared to the simpler models.

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2011-03-19 03:59
(4947 d 16:59 ago)

@ d_labes
Posting: # 6779
Views: 35,955

THX!

Post reply

Bless you, sir, and all your house, unto the seventh generation!
You are a most noble and shining example of all that's right and good and true here.

❝ Do you think I have got their points?

Yes. Exactly. :crying:

Edit: Data Set I is funny. Some subjects have missing data in one period, but data in a subsequent one (e.g., subject 11’s third period is missing). Obviously not data from the ‘real world’. Did you have a look at box plots and/or QQ-plots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?
Quote from last June’s Q&A:

On a case by case basis, a study could be acceptable if the bioequivalence requirements are met both including the outlier subject (using the scaled average bioequivalence approach and the within-subject CV with this subject) and after exclusion of the outlier (using the within-subject CV without this subject).
An outlier test is not an expectation of the medicines agencies but outliers could be shown by a box plot. This would allow the medicines agencies to compare the data between them.

Excluding subjects 45 and 52 CV_WR (EMA’s method) drops from 47.0% to 32.2%. Maybe that’s a hidden trick question to us:


                     Scaled  AR      width

Full data set:     71.23 - 140.40    69.17 

Outliers excluded: 78.79 - 126.93    48.14

The 90% CIs of the full data set of 107.11 - 124.89 (Method A) and 107.17 - 124.97 (Method B) are also within the narrower scaled AR – but it’s a pretty close shave!

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2011-03-24 12:55
(4942 d 08:03 ago)

@ Helmut
Posting: # 6803
Views: 35,440

Outlier

Post reply

Dear Helmut,

❝ "Bless you, sir, and all your house, unto the seventh generation!

❝ You are a most noble and shining example of all that's right and good and true here."

This quote, or is it your own poetry, is really :google:

resistant. First time
that I noticed the Net is not omniscient!

❝ ... Did you have a look at box plots and/or QQ-plots of residuals?

❝ What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?

Which residuals did you analyse? Which model? Please enlighten me.
What do you think: which type of boxplot should we apply? Best for our sponsors would be a simple boxplot with whiskers up to minimum/maximum not showing any 'outlier' :-D

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2011-03-24 16:09
(4942 d 04:49 ago)

@ d_labes
Posting: # 6806
Views: 36,197

Outliers - yes, but how?

Post reply

Dear D. Labes!

❝ This quote, or is it your own poetry, …

I borrowed it from a personal discussion page at Wikipedia.

❝ … is really google resistant. First time that I noticed the Net is not omniscient!

Well, personal discussion pages of WP are blocked from being indexed by . I have blocked some pages (i.e., the download section and the latest posts) from as well. Requires two lines in the domain’s robots.txt…

User-agent: *

Disallow: /foo/

… and adding rel="nofollow" to respective links within other pages (the same is done here; look at the HTML code).

❝ ❝ … Did you have a look at box plots and/or QQ-plots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)?

❝

❝ Which residuals did you analyse? Which model? Please enlighten me.

I used EMA’s crippled model (test removed), Sequence+Subject(Sequence)+Period and threw away one period’s residuals (same values, but different in signs to the respective other one).

❝ What do you think: which type of boxplot should we apply? Best for our sponsors would be a simple boxplot with whiskers up to minimum/maximum not showing any 'outlier' :-D .

I have chosen ±3×IQR following the convention (!) that values within 1.5-3×IQR are ‘mild’ outliers and outside 3×IQR are ‘severe’ outliers. I’m exploring full replicate data-sets right now – outliers almost in all of them. Don’t know whether residuals make any sense at all (see also this post: period ratios instead?). I would end up with different numbers of outliers, depending on the method of calculation of the IQR (see R-manual). Period ratios (second / first administration):
Type 3: SAS according to R-doc - 2 outliers (#46: 3.524, #45: 26.08)
Type 5: or is this SAS? - 2 outliers
Type 6: Minitab, SPSS, Phoenix/WinNonlin - 2 outliers
Type 7: default in S, R - 3 outliers (as above + #13: 3.349)

                               CVWR    Scaled  AR      width

Full data set:                 47.0  71.23 - 140.40    69.17 

#45, #52 excluded (residuals): 32.2  78.79 - 126.93    48.14

#45, #46 excluded (ratios):    35.3  77.08 - 129.73    52.65

Don’t know what to do. :confused:

Suggestions?
Maybe Q&A is an abbreviation for Questions and Ambiguities.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2011-03-28 16:16
(4938 d 05:42 ago)

@ Helmut
Posting: # 6822
Views: 35,503

Questions and Ambiguities

Post reply

Dear Helmut,

❝ Period ratios (second / first administration):

❝ Type 3: SAS according to R-doc - 2 outliers (#46: 3.524, #45: 26.08)

❝ Type 5: or is this SAS? - 2 outliers

❝ Type 6: Minitab, SPSS, Phoenix/WinNonlin - 2 outliers

❝ ...

SAS has 5 different percentile definitions. The default is:
Let n*p=j+g where j is the integer part, g is the fractional part, n is the number of values, x the ordered values.
Let y denote the percentile. Then (SAS PCTLDEF=5)


  y = 0.5*(x_j+x_j+1) if g=0

  y = x_j           if g>0

This corresponds to R's Type 2 I think.

❝ Don’t know what to do. :confused: Suggestions?

If the 'outlier' considerations based on the crippled EMA model makes any sense at all, which I'm not convinced at all :no:

, I would vote for an analysis of the residuals, not the period ratios also they appeared a natural choice on the first view.
Pro's:

Period effects accounted for
Analysis within the log domain (deemed necessary for AUC, Cmax)
Distribution hopefully nearly normal, nearly symmetric because of the previous point, such that the 3xIQR rule can be applied reasonably
Independent of order (first/second or second/first)

Con's:

Period effects different from the whole model, because of throwing away all data under Test, residuals therefore may be biased
Residuals not independent because of the model fitted (2 values of residual on a subject only different in sign, must throw away 1/2 of them)

But these Con's are unavoidable because of the EMA 'model', whatever here is modelled.
To add more ambiguities:

simple residuals?
studentized residuals?
external studentized residuals (leave-one-out residuals)?
other influence statistics (e.g. Cook's distance)?

—
Regards,

Detlew

d_labes
★★★

Berlin, Germany,
2011-04-04 10:53
(4931 d 11:05 ago)

@ Helmut
Posting: # 6855
Views: 35,781

Residuals and Outliers in Replicate Design Crossover Studies

Post reply

Dear Helmut, dear All!

FYI:

Robert Schall, Laszlo Endrenyi, Arne Ring
Residuals and Outliers in Replicate Design Crossover Studies
Journal of Biopharmaceutical Statistics, 20: 4, 835 — 849

Online accessible here as preprint or here.

IMHO this fits excellent into the framework of scaled ABE evaluated via suitable intra-subject contrasts as outlined in the Progesterone guidance :ok:

—
Regards,

Detlew

ElMag ☆ 2011-03-24 13:45 (4942 d 07:13 ago) (edited by ElMag on 2011-03-24 14:44) @ Helmut Posting: # 6805 Views: 35,789	Info requested Post reply
	❝ Did you have a look at box plots and/or QQ-plots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)? Hello there!! Can you tell me if you have created the box-plots based on the ratio of Cmax between the two periods? If not, please indicate which data have you selected and how did you use them in order to create the box-plot. Thank you!

Helmut ★★★ Vienna, Austria, 2011-03-24 17:50 (4942 d 03:09 ago) @ ElMag Posting: # 6808 Views: 35,545	Confused as well... Post reply
	Dear ElMag! See above. Don’t know whether this makes sense at all. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes

Priyanka_S ☆ 2011-03-21 15:28 (4945 d 05:30 ago) @ Helmut Posting: # 6789 Views: 35,745	Q&A published 14 March 2011 Post reply
	Dear all, We tried the SAS code given by EMA for replicated designs by using method A with data set II (three period data). But it is showing the following error. Please clarify this. WARNING: ADJUST=T implies no adjustment for simultaneous inference. — Best Regards Priyanka S

d_labes
★★★

Berlin, Germany,
2011-03-21 17:08
(4945 d 03:50 ago)

@ Priyanka_S
Posting: # 6790
Views: 37,745

SAS code: Warning

Post reply

Dear Priyanka,

❝ ... But it is showing the following error ...

❝ WARNING: ADJUST=T implies no adjustment for simultaneous inference.

as the log states this is not an ERROR:, but a WARNING:. And this warning is nonsense in the context.
You request with the option ADJUST=T confidence intervals using the t-quantile without eventually necessary multiplicity adjustments.
But multiplicity adjustment is not necessary in comparing only two possible LSMeans. It would be eventually necessary if there are more than two. Proc GLM had could that figured out, but SAS in his great wisdom had decided nevertheless to throw a warning.
Note that this warning disappears if you omit the ADJUST=T option in the LSMeans statement. Doing so Proc GLM reverts to the default method for CI calculation: confidence intervals using the t-quantile without eventually necessary multiplicity adjustments, i.e. the same as with ADJUST=T, but without a warning. What a stroke of genius :cool:

.

This annoying warningo-phylistic behaviour is notorious in SAS 9.2. You can observe it in numerous instances. Check if SAS does what you intend and forget the warnings if it does.

BTW: The WARNING: has nothing to do with the data set used. You will get it also with data set I.

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2011-03-21 23:38
(4944 d 21:21 ago)

@ d_labes
Posting: # 6791
Views: 35,703

Copy & paste

Post reply

Dear D. Labes!

I'm not gifted with , but I'm wondering how EMA got their results for 'Method C' with this statement:
estimate 'test-ref' formulation -1 1/ CL alpha=0.10; :cool:

@ Priyanka:
If you want to repeat results from any type of software, first use the code exactly as it is given. Only if your results don't match, try to modify your code (sometimes published code contains typos...).
ADJUST=T is not stated by FDA (2001), EMA's Q&A (2011), and FDA's Progesterone Guidance (2010, 2011).

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2011-03-22 10:24
(4944 d 10:35 ago)

@ Helmut
Posting: # 6794
Views: 36,107

Subtleties, flaws, questions

Post reply

Dear Helmut!

❝ ... but I'm wondering how EMA got their results for 'Method C' with this statement: estimate 'test-ref' formulation -1 1/ CL alpha=0.10;

This is one of the hidden gems in SAS coding :-D

. But it is correct if you use the codes 'R' and 'T' for the formulations. SAS orders them lexically 'R' coming first and expects the coefficients for the difference in the estimate statement in that order.

❝ @ Priyanka:

❝ ... ADJUST=T is not stated by FDA (2001), EMA's Q&A (2011), and FDA's Progesterone Guidance (2010, 2011).

Here the Great Admin err's. It is stated in EMA's Q&A Method 1:


proc glm data=replicate;

  class formulation subject period sequence;

  model logDATA= sequence subject_(sequence) period formulation; 

  *the space above may lead to hard to discover errors!;

  estimate "test-ref" formulation -1+1;

  test h=sequence e=subject(sequence);

  lsmeans formulation / adjust=t pdiff=control("R") CL alpha=0.10;

run;

But as I said above, it is not necessary here. Also the pdiff=control("R") is not necessary. But it does here the job of ordering T first and thus giving the LSMeans difference T-R because again the LSMean of 'R' comes first and the difference 1-2 is calculated.
Moreover the whole lsmeans statement is superfluous if you code the option /CLparm alpha=0.1 in the model statement. Then the estimate statement will give the CI like Proc MIXED does as default.

BTW: I'm not sure if the test of the sequence effect as coded from Great Oracle EMA is appropriate in case of missings. Using the 'Capt'n EM calls me bogus' Random statement I get a mixture of MS_error and MS_{subject(sequence)} as denominator of the corresponding F-test. The degrees of freedom are also adapted according to Satterthwaite.

BTW2: The code for obtaining the intra-subject variance taken literally will bring us directly to the Type III hell. Output (I have named their DATA as AUC):

--- GLM-ANOVA Analysis of REF. within-subject var. for log(AUC) ---



The GLM Procedure



Dependent Variable: logval   log(AUC)



                                Sum of

Source                DF       Squares    Mean Square   F Value   Pr > F



Model                 78   120.2314511      1.5414289      7.73   <.0001

Error                 71    14.1512621      0.1993136

Corrected Total      149   134.3827132





Source                DF     Type I SS    Mean Square   F Value   Pr > F



sequence               1     0.0880117      0.0880117      0.44   0.5085

subject(sequence)     75   119.3707667      1.5916102      7.99   <.0001

period                 2     0.7726727      0.3863363      1.94   0.1515





Source                DF   Type III SS    Mean Square   F Value   Pr > F



sequence               0     0.0000000       .              .      .

subject(sequence)     75   118.6616824      1.5821558      7.94   <.0001

period                 2     0.7726727      0.3863363      1.94   0.1515

As you see: Subtleties and flaws in coding, questions over questions after that 'Clarification'.
But what could we expect other if considering the scientific foundation of that all?

—
Regards,

Detlew

Helmut
★★★

Vienna, Austria,
2011-03-27 22:35
(4938 d 23:23 ago)

@ d_labes
Posting: # 6818
Views: 35,706

Subtleties, flaws, questions

Post reply

Dear D. Labes!

❝ Source DF Type III SS Mean Square [...]

❝

❝ sequence 0 0.0000000 . [...]

❝ [...]

Zero degrees of freedom is not a lot.
I love SAS‘s ‘.’ - Phoenix/WinNonlin spits out ‘Not estimable’, but I can trim it to ‘.’ or even ‘0’… :-D

Open issues IMHO:

Outliers – which method? Model residuals or R/R ratios?
Obviously EMA is not interested in CV_WT (Method C) at all. Since GLM throws away incomplete data (Method A), why should one go with a full replicate design? More periods, more chances of drop-outs.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

The user
☆

Russia,
2017-03-14 11:16
(2760 d 09:43 ago)

(edited by The user on 2017-03-14 12:44)
@ Helmut
Posting: # 17151
Views: 25,274

Not estimable in the model

Post reply

❝ I love SAS‘s ‘.’ - Phoenix/WinNonlin spits out ‘Not estimable’, but I can trim it to ‘.’ or even ‘0’… :-D

Dear Helmut,

I decided to continue the topic (moreover Simon kindly directed me here).
Regarding "Not estimable". Could you please share your expirience with following issue? When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapears. Comments: it is BE cross over study in 3 groups and all effects are fixed.

BR

ElMaestro
★★★

Denmark,
2017-03-14 12:02
(2760 d 08:56 ago)

@ The user
Posting: # 17152
Views: 25,050

Not estimable in the model

Post reply

Hi The user,

❝ I decided to continue the topic (moreover Simon kindly directed me here).

❝ Regarding "Not estimable". Could you please share your expirience with following issue? When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapeares. Comments: it is BE cross over study in 3 groups and all effects are fixed.

Hmmmm... thinking loud here... In a type III model Group will have in your case up to three columns in the model matrix, but every column is equally represented by the subjects in the respective groups. Hence Group has zero df's and no effect for any level of Group can be estimated.

Try a type I model with Group before Subjects, then you'll get something that is estimable, I believe.

—
Pass or fail!
ElMaestro

The user ☆ Russia, 2017-03-14 13:13 (2760 d 07:45 ago) @ ElMaestro Posting: # 17154 Views: 25,075	Not estimable in the model Post reply
	❝ Try a type I model with Group before Subjects, then you'll get something that is estimable, I believe. Thank you for your reply. Sorry, but I did not catch the idea. Should I try this model: Group+Patient(Sequence)+Sequence+Period+Treatment? One more comment: the groups are unbalanced. Type I is suiatable for balanced groups as I undertood. Looks like I could not use the Type I model.

ElMaestro
★★★

Denmark,
2017-03-14 13:41
(2760 d 07:17 ago)

@ The user
Posting: # 17155
Views: 25,048

Not estimable in the model

Post reply

Hi,

❝ Sorry, but I did not catch the idea. Should I try this model: Group+Patient(Sequence)+Sequence+Period+Treatment?

Yes try that with type I. I think type III may give the same as before if you are not using SAS.

❝ One more comment: the groups are unbalanced. Type I is suiatable for balanced groups as I undertood. Looks like I could not use the Type I model.

Your CI will be the same. Type I vs Type III is generally a topic that is of a much more sensitive nature to some people than e.g.religion or venereal diseases. People who grew up with SAS stick to type III, and type III only, because that is all they know and therefore they seem to be resistant to common sense. Besides, SAS invented the term "Least Squares Means" and that sounds so good that no reasonable alternative could ever exist, right?

Type I is not better or worse than type III. LS Means are no better than model effects. Depending on contrasts, model effects are LS Means and vice versa. And so forth...

—
Pass or fail!
ElMaestro

Helmut
★★★

Vienna, Austria,
2017-03-18 22:59
(2755 d 21:59 ago)

@ The user
Posting: # 17164
Views: 25,281

Not estimable in the model

Post reply

Hi BR,

❝ When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapears. Comments: it is BE cross over study in 3 groups and all effects are fixed.

As Simon noted already at Certara’s Forum, it’s the fixed effects model – not the software. Even without the group-term (i.e., the EMA’s sequence, subject(sequence), period, formulation) you will get an endless list of “Not estimables” simply because the requested combination does not exist in the data set. Example: All subjects are uniquely coded (1, 2, …, n) and subject 1 is in sequence RT. You will get an estimate. Fine. But the model tries to estimate subject 1 in sequence TR as well. Not estimable! That’s correct because a datum with such a coding does not exist.

Since you are posting from Russia what do you want to achieve? Satisfy the «Экспертами» (see this post)? Every time I was in Moscow we had endless & fruitless debates about it… All relevant documents (2008 GL, 2013 “Red Book”, 2015 EEU GL) are more or less translations of the EMA’s GL (the EEU GL spiced with some parts of the WHO’s GL). What does the EMA’s GL say about groups (or more important sites)? Nothing! Only:

The precise model to be used for the analysis should be pre-specified in the protocol. The statistical analysis should take into account sources of variation that can be reasonably assumed to have an effect on the response variable.

Is it reasonable to assume such an effect if a study was performed in multiple groups due to logistic reasons (e.g., limited capacity of the clinical site)? I don’t think so. Hence, in the EU generally data are simply pooled and the common model (without a group term) is used.
Different sites are much more problematic. I recently saw a multi-site study where the sites clearly showed different results (averages differed tenfold). It was a cancer drug and some sites were pretty small. If (if!) all sites would have balanced sequences it would have been still no problem but this was not the case. Actually there was a highly significant (p <0.001) site-by-treatment interaction. If one would naïvely pool the sites the treatment effect would be biased.

OK, back to the EEU GL (the last paragraph of section 94):

Если предполагается проведение исследования в нескольких группах из логистических соображений, об этом необходимо явно указать в протоколе исследования; при этом, если в отчете отсутствуют результаты статистического анализа, учитывающие многогрупповой характер исследования, необходимо представить научное обоснование отсутствия таких результатов.

My interpretion:

State already in the protocol that the study will be performed in multiple groups and give a justification that an effect on the treatment comparison can be reasonably ruled out (i.e., same site, same procedures, all subjects randomized before splitting, same batches of T and R, short time interval between groups, blahblah).
If you want to go the hard way: Modify the FDA’s multi-group models (i.e., mixed-effects ⇒ all effects fixed). More about that later.

I would always try #1. Whether the experts will swallow that is another story.
#2 can be nasty! Start with “Model 1” (fixed effects* in Phoenix-notation):

Group + Sequence + Sequence(Group) + Period(Group) + Treatment +
Treatment * Group + Subject(Sequence * Group)

If the term Treatment * Group (the treatment-by-group interaction) is not significant at the 0.1 (!) level remove this term and perform the analysis by “Model 2”. The between-group test is not very sensitive (sloppy: has low power). Therefore, the FDA requires testing at 0.1 (and not at 0.05).
If the test is significant, you are not allowed to pool the data and can only run the conventional model with the data of the largest group. Good luck! The loss of power likely will be extreme. Furthermore, you could expect false positives – and consequently throw away 10% of your studies…

“Model 2”:

Group + Sequence + Sequence(Group) + Period(Group) + Treatment +
Subject(Sequence*Group)

Yes, you will see an awful lot of “Not estimables”. :-D

I have seen a data set (three groups: 19, 19, 18) where both fixed effects models did not converge (neither in Phoenix nor in SAS/JMP). No problem with the mixed-effects models (i.e., Subject(Sequence*Group) random)… The statistician of the MHRA accepted it.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Beholder
★

Russia,
2021-11-10 15:35
(1058 d 05:23 ago)

@ Helmut
Posting: # 22670
Views: 6,060

Food not the same between groups

Post reply

Hi, Helmut!

❝ (i.e., same site, same procedures, all subjects randomized before splitting, same batches of T and R, short time interval between groups, blahblah).

Many times the approach with resigning from testing group effect based on justification in protocol that volunteers undergoes the same procedures, take the same batch of drug etc., was fine and very helpful for many protocols. But. Recently one clinic has used two slightly different types of menu for two groups. Volunteers ate the same food in period 1 and 2. Could this situation lead to necessity to test group effect since not the same procedures were implemented to two groups or this difference has too little weight?

—
Best regards
Beholder

ElMaestro
★★★

Denmark,
2021-11-17 10:50
(1051 d 10:08 ago)

@ Beholder
Posting: # 22672
Views: 5,560

Food not the same between groups

Post reply

Hi beholder,

❝ Many times the approach with resigning from testing group effect based on justification in protocol that volunteers undergoes the same procedures, take the same batch of drug etc., was fine and very helpful for many protocols. But. Recently one clinic has used two slightly different types of menu for two groups. Volunteers ate the same food in period 1 and 2. Could this situation lead to necessity to test group effect since not the same procedures were implemented to two groups or this difference has too little weight?

It could easily lead to a significiant group effect. Whether you need to test for that is up to the assessor. If you don't they'll probably ask for it. If you do, they'll probably tell you it was unnecessary (but behind the curtains they have checked the p-value).

—
Pass or fail!
ElMaestro

EMA BE-GL: Clarifications / Corrections? [BE/BA News]

Good news!?

Good news!?

Update

Q&A published 14 March 2011

Q&A published 14 March 2011

Phoenix/WinNonlin 6.1.0.173

ANOVA party prevails

THX!

Outlier

Outliers - yes, but how?

Questions and Ambiguities

Residuals and Outliers in Replicate Design Crossover Studies

Info requested

Confused as well...

Q&A published 14 March 2011

SAS code: Warning

Copy & paste

Subtleties, flaws, questions

Subtleties, flaws, questions

Not estimable in the model

Not estimable in the model

Not estimable in the model

Not estimable in the model

Not estimable in the model

Food not the same between groups

Food not the same between groups