lechia
☆    

C of U,
2013-08-07 01:23
(3886 d 16:42 ago)

Posting: # 11228
Views: 11,454
 

 "Removal" of an outlier: subject vs period [Outliers]

Hi all,

Let's say I have a study where two products are tested against a reference (3 period, incomplete block). Test 2 product for one of the subjects is an outlier for AUC and Cmax as per protocol (>3 studentized residuals away from the mean).

The question is, should this subject be removed completely from the study or just the period in question?

From a statistical point of view, only one of the test products/periods was an outlier, however, I'm of the view that given that since we don't know what caused the "unusual" profile in the Test 2 profile, perhaps it's better to remove the subject altogether. For example, the subject might not have consumed the whole tablet in the Test 1 and ref periods and then swallowed all the pills in one go (obviously highly unlikely due to the various checks in place and the fact that the first two periods look "fine"). The subject could have fallen ill, became dehydrated, ate something, consumed a "protocol banned" product, etc. without informing staff in the previous period, but this only partly affected the results then; the effect was really only seen in the next period. You can all probably come up with other possibilities for why Test 1 and/or ref could have also been affected, even if not to the same degree.

Looking forward to reading your input.
Mac
ElMaestro
★★★

Denmark,
2013-08-07 13:09
(3886 d 04:56 ago)

@ lechia
Posting: # 11230
Views: 10,141
 

 "Removal" of an outlier: subject vs period

Hi Mac,

❝ The question is, should this subject be removed completely from the study or just the period in question?


Where do you intend to submit?
What does your protocol say about outlier issues?
What do CRFs and raw data tell about the subject in question (or possibly: the subject in question compared to subjects you consider more normal or clustered around a mean of sorts)?

❝ (...) since we don't know what caused the "unusual" profile in the Test 2 profile, perhaps it's better to remove the subject altogether.


Or perhaps it is better not to remove any data at all in order to make sure that the true product performances are reflected in the CIs.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-08-07 15:48
(3886 d 02:17 ago)

@ lechia
Posting: # 11232
Views: 10,255
 

 Canada: subject, not period

Hi Mac,

welcome to the club! You are one of the <1% of members editing their profile. ;-)

❝ Let's say I have a study where two products are tested against a reference (3 period, incomplete block).


I don’t understand what you mean by incomplete block when you tested three treatments in three periods. Generally in an IBD: treatments > periods.

❝ Test 2 product for one of the subjects is an outlier for AUC and Cmax as per protocol (>3 studentized residuals away from the mean).


❝ The question is, should this subject be removed completely from the study or just the period in question?


Depends on what you have specified in the protocol. By removing one period you’ll end up with an incomplete data set which might be evaluated by a mixed effects model (SAS-speak: PROC MIXED). See also Senn’s quote in this post. Note that PROC GLM will always drop the entire subject even if you kept the other periods in the data set. Quite often CIs by PROC MIXED (incomplete data) are very close the the ones obtained by PROC GLM (imbalanced data). For an example see this post.

❝ From a statistical point of view, only one of the test products/periods was an outlier, however, I'm of the view that given that since we don't know what caused the "unusual" profile in the Test 2 profile, perhaps it's better to remove the subject altogether.


Likely. The Canadian guidance talks only about removal of outlying subjects, not observations.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2013-08-07 16:46
(3886 d 01:19 ago)

@ Helmut
Posting: # 11233
Views: 10,289
 

 Proc GLM with missings

Dear Helmut!

❝ ... Note that PROC GLM will always drop the entire subject even if you kept the other periods in the data set.


This is an widely spread rumour, also here in the forum. But nevertheless isn't true :no:.

Here an example: 3x6x3 data of 12 subjects with one subject in one period set to missing.
Proc GLM output:
                                    The GLM procedure
                      Analysis of Variance - Tests of Model Effects
                                             
   Source               DF              SS     Mean Square    F Value    Pr > F

   treatment             2        0.024103        0.012051       1.00    0.3873
   period                2        0.005885        0.002942       0.24    0.7863
   subject(sequence)     6        0.439062        0.073177       6.06    0.0011
   -----------------
   Error: MS(Error)     19        0.229576        0.012083                     
   Error: MSE
 

   Source               DF              SS     Mean Square    F Value    Pr > F

   sequence              5        0.448448        0.089690       1.23    0.3991
   -----------------
   Error            6.0032        0.438708        0.073079                     
   Error: 0.9984*MS(subject(sequence)) + 0.0016*MS(Error)


df(subjects)= df(sequence)+df(subject(sequence))=11 (=N-1). All subjects used!
df(Error) = 2*N-4 = 20 if no missings, but here 1 missing i.e. =19. All data used!

What SAS does with the sequence effect test (random statement used) in such cases is left for discussion. Never understood it really.

Regards,

Detlew
jag009
★★★

NJ,
2013-08-07 17:03
(3886 d 01:02 ago)

@ d_labes
Posting: # 11234
Views: 10,042
 

 Proc GLM with missings

Hi Detlew,

❝ This is an widely spread rumour, also here in the forum. But nevertheless isn't true :no:.


So Proc GLM is no different from Proc Mixed in that aspect?

John
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-08-07 17:21
(3886 d 00:44 ago)

@ d_labes
Posting: # 11236
Views: 10,105
 

 Proc GLM with missings

Dear Detlew!

❝ ❝ ... Note that PROC GLM will always drop the entire subject even if you kept the other periods in the data set.


❝ This is an widely spread rumour, also here in the forum. But nevertheless isn't true :no:.


❝ What SAS does with the sequence effect test (random statement used) in such cases is left for discussion. Never understood it really.


THX for enlightening an SAS-illiterate. Now I remember that Jean-Michel showed me that a while ago. As an SAS-initiate he clicked around so fast that I couldn’t follow. He posted it also here. Will keep my mouth shut in the future.
BTW, I wonder where the origin of this rumour might be. :confused:

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2013-08-07 17:39
(3886 d 00:26 ago)

@ Helmut
Posting: # 11237
Views: 10,057
 

 Proc GLM with missings

Hi Helmut,

❝ BTW, I wonder where the origin of this rumour might be. :confused:


I thought the rumour was true for 2,2,2-BE evaluations where data from one period are missing.

d_labes: As for the bogus statement, I must say that the online SAS manual is not particularly informative. Since it does not give any behavioural particulars for GLM with bogus, I think it just behaves the same regardless of whether there are missing cells or not?!?

I would be interested to know how it figures out ls mean treatment difference. If there is a missing cell then I guess it is not the difference of treatment ls means. And what abut the df's for the treatment differences in such cases.

Pass or fail!
ElMaestro
ElMaestro
★★★

Denmark,
2013-08-07 18:22
(3885 d 23:44 ago)

@ ElMaestro
Posting: # 11241
Views: 10,105
 

 Proc GLM with missings

Hi again,

further to my previous post:

After having fiddled a little with pen and paper I think GLM or any other lm software might have to discard any subject in a n-period cross over where n-1 periods are missing.
Try and compare with PROC GLM:
  1. A 3 period dataset with two periods missing for one subject.
  2. A 3-period dataset with three periods missing for one subjects (same subject as pt. 2).
Do the two fits coincide?

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2013-08-07 18:46
(3885 d 23:19 ago)

@ ElMaestro
Posting: # 11245
Views: 10,138
 

 Proc GLM random

Dear ElMaestro,

❝ d_labes: As for the bogus statement, I must say that the online SAS manual is not particularly informative. Since it does not give any behavioural particulars for GLM with bogus, I think it just behaves the same regardless of whether there are missing cells or not?!?


I'm not sure if I get your point here.
The random statement is used by me solely for the purpose of doing the 'right' F-test for the sequence effects. In case of no missings this statement correctly figures out that the denominator for this test is subject(sequence). Ok, Ok let your silly-o-meter in your pocket :-D.
If there are missings some linear combination of MSE and MSsubject(sequence) is used. Why? As already said: never really understood.

❝ I would be interested to know how it figures out ls mean treatment difference. If there is a missing cell then I guess it is not the difference of treatment ls means. And what abut the df's for the treatment differences in such cases.


The random statement has nothing to do with the LSmeans, their diff and associated df, I think. These are calculated with all effects fixed as in any lm software.

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2013-08-07 18:34
(3885 d 23:31 ago)

@ Helmut
Posting: # 11243
Views: 10,325
 

 Proc GLM with missings for 2x2 crossover

Dear Helmut!

❝ BTW, I wonder where the origin of this rumour might be. :confused:


As others meanwhile have already guessed: From the observation that the 90% CIs for 2x2x2 crossover are the same, regardless of one period missing or removing the subject with that missing (see f.i. this post).
The subject with the missing does not contribute to the error variance (mse) and by coincidence the df are also equal. Example 24 subjects:
df(error)=N-2=21 if one subject removed
df(error)=N-2=22 if all subjects included, minus 1 for the missing =21.

Regards,

Detlew
lechia
☆    

C of U,
2013-08-07 17:46
(3886 d 00:20 ago)

@ Helmut
Posting: # 11238
Views: 10,130
 

 Canada: subject, not period

❝ welcome to the club! You are one of the <1% of members editing their profile. ;-)


Took some time to actually find where I could edit my profile, but I did read "the rules". :-)

❝ ❝ Let's say I have a study where two products are tested against a reference (3 period, incomplete block).


❝ I don’t understand what you mean by incomplete block when you tested three treatments in three periods. Generally in an IBD: treatments > periods.


What I mean is that even though we have 3 periods and 3 treatments, the comparisons are Test 1 vs. Ref and Test 2 vs. Ref. In other words, we "ignore" one period when comparing the two.

❝ Likely. The Canadian guidance talks only about removal of outlying subjects, not observations.


True, however, whoever prepared the guidance most likely thought about a 2x2 study, where removal of one period "removes" the entire subject from analysis. It's unusual to have two (or more) products being tested as in my example. I guess you're right, however, in that removal of the entire subject from analysis will follow the guidance to the letter and perhaps it's not our place to try to figure out what the agency was thinking when writing the guidance.

To answer El Maestro's questions, Health Canada requires an outlier test to be done (before actually running the stats so that an outlier is removed or not without being affected by the study failing or not). Also, this doesn't pertain to a specific study, but rather I'm thinking of the best way to deal with this if/when it comes up, so the appropriate text can be added to the protocol a priori.

We're using PROC MIXED anyways.

Thanks to all for your input.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-08-07 17:57
(3886 d 00:08 ago)

@ lechia
Posting: # 11239
Views: 10,143
 

 Canada: subject, not period

Hi Mac!

❝ Took some time to actually find where I could edit my profile, but I did read "the rules". :-)


Kudos!

❝ ❝ I don’t understand what you mean by incomplete block when you tested three treatments in three periods. Generally in an IBD: treatments > periods.


❝ What I mean is that even though we have 3 periods and 3 treatments, the comparisons are Test 1 vs. Ref and Test 2 vs. Ref. In other words, we "ignore" one period when comparing the two.


Interesting. Search the forum for ‘crippled model’. ;-)

❝ […] whoever prepared the guidance…


Eric Ormsby?

❝ … most likely thought about a 2x2 study, where removal of one period "removes" the entire subject from analysis.


Think so. Also ElMaestro’s guess.

❝ […] perhaps it's not our place to try to figure out what the agency was thinking when writing the guidance.


Oh no; this is the place!

❝ We're using PROC MIXED anyways.


So don’t bother.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
lechia
☆    

C of U,
2013-08-12 17:48
(3881 d 00:18 ago)

@ Helmut
Posting: # 11263
Views: 9,962
 

 Canada: subject, not period

❝ Likely. The Canadian guidance talks only about removal of outlying subjects, not observations.


How about semi- and fully-replicate studies? Would you still suggest removal of the subject or just the period in question? In some sense, in scaled BE studies the replicate serves as a self-contained re-dose study (which the FDA accepts and EMEA does not).
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,638 registered users;
78 visitors (0 registered, 78 guests [including 10 identified bots]).
Forum time: 17:06 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5