Bioequivalence and Bioavailability Forum • "Removal" of an outlier: subject vs period

lechia
☆

C of U,
2013-08-07 01:23
(4295 d 12:01 ago)

Posting: # 11228
Views: 13,241

"Removal" of an outlier: subject vs period [Outliers]

Hi all,

Let's say I have a study where two products are tested against a reference (3 period, incomplete block). Test 2 product for one of the subjects is an outlier for AUC and Cmax as per protocol (>3 studentized residuals away from the mean).

The question is, should this subject be removed completely from the study or just the period in question?

From a statistical point of view, only one of the test products/periods was an outlier, however, I'm of the view that given that since we don't know what caused the "unusual" profile in the Test 2 profile, perhaps it's better to remove the subject altogether. For example, the subject might not have consumed the whole tablet in the Test 1 and ref periods and then swallowed all the pills in one go (obviously highly unlikely due to the various checks in place and the fact that the first two periods look "fine"). The subject could have fallen ill, became dehydrated, ate something, consumed a "protocol banned" product, etc. without informing staff in the previous period, but this only partly affected the results then; the effect was really only seen in the next period. You can all probably come up with other possibilities for why Test 1 and/or ref could have also been affected, even if not to the same degree.

Looking forward to reading your input.
Mac

ElMaestro
★★★

Denmark,
2013-08-07 13:09
(4295 d 00:15 ago)

@ lechia
Posting: # 11230
Views: 11,624

"Removal" of an outlier: subject vs period

Post reply

Hi Mac,

❝ The question is, should this subject be removed completely from the study or just the period in question?

Where do you intend to submit?
What does your protocol say about outlier issues?
What do CRFs and raw data tell about the subject in question (or possibly: the subject in question compared to subjects you consider more normal or clustered around a mean of sorts)?

❝ (...) since we don't know what caused the "unusual" profile in the Test 2 profile, perhaps it's better to remove the subject altogether.

Or perhaps it is better not to remove any data at all in order to make sure that the true product performances are reflected in the CIs.

—
Pass or fail!
ElMaestro

Helmut
★★★

Vienna, Austria,
2013-08-07 15:48
(4294 d 21:36 ago)

@ lechia
Posting: # 11232
Views: 11,781

Canada: subject, not period

Post reply

Hi Mac,

welcome to the club! You are one of the <1% of members editing their profile. ;-)

❝ Let's say I have a study where two products are tested against a reference (3 period, incomplete block).

I don’t understand what you mean by incomplete block when you tested three treatments in three periods. Generally in an IBD: treatments > periods.

❝ Test 2 product for one of the subjects is an outlier for AUC and Cmax as per protocol (>3 studentized residuals away from the mean).

❝

❝ The question is, should this subject be removed completely from the study or just the period in question?

Depends on what you have specified in the protocol. By removing one period you’ll end up with an incomplete data set which might be evaluated by a mixed effects model (SAS-speak: PROC MIXED). See also Senn’s quote in this post. Note that PROC GLM will always drop the entire subject even if you kept the other periods in the data set. Quite often CIs by PROC MIXED (incomplete data) are very close the the ones obtained by PROC GLM (imbalanced data). For an example see this post.

❝ From a statistical point of view, only one of the test products/periods was an outlier, however, I'm of the view that given that since we don't know what caused the "unusual" profile in the Test 2 profile, perhaps it's better to remove the subject altogether.

Likely. The Canadian guidance talks only about removal of outlying subjects, not observations.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

d_labes
★★★

Berlin, Germany,
2013-08-07 16:46
(4294 d 20:38 ago)

@ Helmut
Posting: # 11233
Views: 11,826

Proc GLM with missings

Post reply

Dear Helmut!

❝ ... Note that PROC GLM will always drop the entire subject even if you kept the other periods in the data set.

This is an widely spread rumour, also here in the forum. But nevertheless isn't true :no:

.

Here an example: 3x6x3 data of 12 subjects with one subject in one period set to missing.
Proc GLM output:

                                    The GLM procedure

                      Analysis of Variance - Tests of Model Effects

                                             

   Source               DF              SS     Mean Square    F Value    Pr > F



   treatment             2        0.024103        0.012051       1.00    0.3873

   period                2        0.005885        0.002942       0.24    0.7863

   subject(sequence)     6        0.439062        0.073177       6.06    0.0011

   -----------------

   Error: MS(Error)     19        0.229576        0.012083                     

   Error: MSE

 



   Source               DF              SS     Mean Square    F Value    Pr > F



   sequence              5        0.448448        0.089690       1.23    0.3991

   -----------------

   Error            6.0032        0.438708        0.073079                     

   Error: 0.9984*MS(subject(sequence)) + 0.0016*MS(Error)

df(subjects)= df(sequence)+df(subject(sequence))=11 (=N-1). All subjects used!
df(Error) = 2*N-4 = 20 if no missings, but here 1 missing i.e. =19. All data used!

What SAS does with the sequence effect test (random statement used) in such cases is left for discussion. Never understood it really.

—
Regards,

Detlew

jag009 ★★★ NJ, 2013-08-07 17:03 (4294 d 20:21 ago) @ d_labes Posting: # 11234 Views: 11,517	Proc GLM with missings Post reply
	Hi Detlew, ❝ This is an widely spread rumour, also here in the forum. But nevertheless isn't true . So Proc GLM is no different from Proc Mixed in that aspect? John

Helmut
★★★

Vienna, Austria,
2013-08-07 17:21
(4294 d 20:03 ago)

@ d_labes
Posting: # 11236
Views: 11,602

Proc GLM with missings

Post reply

Dear Detlew!

❝ ❝ ... Note that PROC GLM will always drop the entire subject even if you kept the other periods in the data set.

❝

❝ This is an widely spread rumour, also here in the forum. But nevertheless isn't true :no: .

❝

❝ What SAS does with the sequence effect test (random statement used) in such cases is left for discussion. Never understood it really.

THX for enlightening an SAS-illiterate. Now I remember that Jean-Michel showed me that a while ago. As an SAS-initiate he clicked around so fast that I couldn’t follow. He posted it also here. Will keep my mouth shut in the future.
BTW, I wonder where the origin of this rumour might be. :confused:

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ElMaestro
★★★

Denmark,
2013-08-07 17:39
(4294 d 19:45 ago)

@ Helmut
Posting: # 11237
Views: 11,531

Proc GLM with missings

Post reply

Hi Helmut,

❝ BTW, I wonder where the origin of this rumour might be. :confused:

I thought the rumour was true for 2,2,2-BE evaluations where data from one period are missing.

d_labes: As for the bogus statement, I must say that the online SAS manual is not particularly informative. Since it does not give any behavioural particulars for GLM with bogus, I think it just behaves the same regardless of whether there are missing cells or not?!?

I would be interested to know how it figures out ls mean treatment difference. If there is a missing cell then I guess it is not the difference of treatment ls means. And what abut the df's for the treatment differences in such cases.

—
Pass or fail!
ElMaestro

ElMaestro
★★★

Denmark,
2013-08-07 18:22
(4294 d 19:03 ago)

@ ElMaestro
Posting: # 11241
Views: 11,587

Proc GLM with missings

Post reply

Hi again,

further to my previous post:

After having fiddled a little with pen and paper I think GLM or any other lm software might have to discard any subject in a n-period cross over where n-1 periods are missing.
Try and compare with PROC GLM:

A 3 period dataset with two periods missing for one subject.
A 3-period dataset with three periods missing for one subjects (same subject as pt. 2).

Do the two fits coincide?

—
Pass or fail!
ElMaestro

d_labes
★★★

Berlin, Germany,
2013-08-07 18:46
(4294 d 18:38 ago)

@ ElMaestro
Posting: # 11245
Views: 11,620

Proc GLM random

Post reply

Dear ElMaestro,

❝ d_labes: As for the bogus statement, I must say that the online SAS manual is not particularly informative. Since it does not give any behavioural particulars for GLM with bogus, I think it just behaves the same regardless of whether there are missing cells or not?!?

I'm not sure if I get your point here.
The random statement is used by me solely for the purpose of doing the 'right' F-test for the sequence effects. In case of no missings this statement correctly figures out that the denominator for this test is subject(sequence). Ok, Ok let your silly-o-meter in your pocket :-D

.
If there are missings some linear combination of MSE and MSsubject(sequence) is used. Why? As already said: never really understood.

❝ I would be interested to know how it figures out ls mean treatment difference. If there is a missing cell then I guess it is not the difference of treatment ls means. And what abut the df's for the treatment differences in such cases.

The random statement has nothing to do with the LSmeans, their diff and associated df, I think. These are calculated with all effects fixed as in any lm software.

—
Regards,

Detlew

d_labes
★★★

Berlin, Germany,
2013-08-07 18:34
(4294 d 18:50 ago)

@ Helmut
Posting: # 11243
Views: 11,843

Proc GLM with missings for 2x2 crossover

Post reply

Dear Helmut!

❝ BTW, I wonder where the origin of this rumour might be. :confused:

As others meanwhile have already guessed: From the observation that the 90% CIs for 2x2x2 crossover are the same, regardless of one period missing or removing the subject with that missing (see f.i. this post).
The subject with the missing does not contribute to the error variance (mse) and by coincidence the df are also equal. Example 24 subjects:
df(error)=N-2=21 if one subject removed
df(error)=N-2=22 if all subjects included, minus 1 for the missing =21.

—
Regards,

Detlew

lechia
☆

C of U,
2013-08-07 17:46
(4294 d 19:39 ago)

@ Helmut
Posting: # 11238
Views: 11,595

Canada: subject, not period

Post reply

❝ welcome to the club! You are one of the <1% of members editing their profile. ;-)

Took some time to actually find where I could edit my profile, but I did read "the rules". :-)

❝ ❝ Let's say I have a study where two products are tested against a reference (3 period, incomplete block).

❝

❝ I don’t understand what you mean by incomplete block when you tested three treatments in three periods. Generally in an IBD: treatments > periods.

What I mean is that even though we have 3 periods and 3 treatments, the comparisons are Test 1 vs. Ref and Test 2 vs. Ref. In other words, we "ignore" one period when comparing the two.

❝ Likely. The Canadian guidance talks only about removal of outlying subjects, not observations.

True, however, whoever prepared the guidance most likely thought about a 2x2 study, where removal of one period "removes" the entire subject from analysis. It's unusual to have two (or more) products being tested as in my example. I guess you're right, however, in that removal of the entire subject from analysis will follow the guidance to the letter and perhaps it's not our place to try to figure out what the agency was thinking when writing the guidance.

To answer El Maestro's questions, Health Canada requires an outlier test to be done (before actually running the stats so that an outlier is removed or not without being affected by the study failing or not). Also, this doesn't pertain to a specific study, but rather I'm thinking of the best way to deal with this if/when it comes up, so the appropriate text can be added to the protocol a priori.

We're using PROC MIXED anyways.

Thanks to all for your input.

Helmut
★★★

Vienna, Austria,
2013-08-07 17:57
(4294 d 19:27 ago)

@ lechia
Posting: # 11239
Views: 11,613

Canada: subject, not period

Post reply

Hi Mac!

❝ Took some time to actually find where I could edit my profile, but I did read "the rules". :-)

Kudos!

❝ ❝ I don’t understand what you mean by incomplete block when you tested three treatments in three periods. Generally in an IBD: treatments > periods.

❝

❝ What I mean is that even though we have 3 periods and 3 treatments, the comparisons are Test 1 vs. Ref and Test 2 vs. Ref. In other words, we "ignore" one period when comparing the two.

Interesting. Search the forum for ‘crippled model’. ;-)

❝ […] whoever prepared the guidance…

Eric Ormsby?

❝ … most likely thought about a 2x2 study, where removal of one period "removes" the entire subject from analysis.

Think so. Also ElMaestro’s guess.

❝ […] perhaps it's not our place to try to figure out what the agency was thinking when writing the guidance.

Oh no; this is the place!

❝ We're using PROC MIXED anyways.

So don’t bother.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

lechia ☆ C of U, 2013-08-12 17:48 (4289 d 19:37 ago) @ Helmut Posting: # 11263 Views: 11,445	Canada: subject, not period Post reply
	❝ Likely. The Canadian guidance talks only about removal of outlying subjects, not observations. How about semi- and fully-replicate studies? Would you still suggest removal of the subject or just the period in question? In some sense, in scaled BE studies the replicate serves as a self-contained re-dose study (which the FDA accepts and EMEA does not).