Outlier tests: forget it!....what if.... [Bioanalytics]
Dear all,
A very interesting topic!
Could someone please comment the extreme example explained below:
During the study, we normally have 2 QC replicates at three diff. QC levels for each run.
Our QC acceptance criteria for each run: at least 4/6 of all QCs and 1/2 QCs at each level must be within ±15% of the nominal concentration (%Bias).
In addition, we have a during-study acceptance criteria for all QCs at each level: %CV<15% and %Nominal 100±15%.
Now the »extreme« example; nearly »ideal« method… a study of 50 subjects….50 runs…. 100 QCs at each level
49 runs…. all QCs at each level have %Nominal of 100%
1 run… 5 QCs 100% Nominal and one QC at level 1 300% Nominal (run is accepted)
During the investigation of this run you don't find any analytical reason to exclude »300%QC« (chromatography OK, IS response consistent,….) … you suspect a possible sample switch or problem (contamination?) during extraction…but you can't prove that.
All runs within study are accepted according to the 4/6 and 1/2 rule; however, if we calculate %CV and %Bias, we get the following results:
According to the FDA guidelines (“Summary information on intra- and inter-assay values of QC samples and data on intra- and inter-assay accuracy and precision from calibration curves and QC samples used for accepting the analytical run.“), this nearly “ideal” method would be unsuitable due to only one out of 300 QCs failing really badly.
But is this method really not OK?
In general I agree with “no outliers policy”, but on the other hand, FDA “allows” some sort of the outliers: “Reported method validation data and the determination of accuracy and precision should include all outliers; however, calculations of accuracy and precision excluding values that are statistically determined as outliers can also be reported.”
If the “300%QC” would be rejected as an outlier, the method would become ideal.
Any ideas?
Regards
Marko
p.s. I tried to write it shorter, but I just couldn’t manage it.
A very interesting topic!
Could someone please comment the extreme example explained below:
During the study, we normally have 2 QC replicates at three diff. QC levels for each run.
Our QC acceptance criteria for each run: at least 4/6 of all QCs and 1/2 QCs at each level must be within ±15% of the nominal concentration (%Bias).
In addition, we have a during-study acceptance criteria for all QCs at each level: %CV<15% and %Nominal 100±15%.
Now the »extreme« example; nearly »ideal« method… a study of 50 subjects….50 runs…. 100 QCs at each level
49 runs…. all QCs at each level have %Nominal of 100%
1 run… 5 QCs 100% Nominal and one QC at level 1 300% Nominal (run is accepted)
During the investigation of this run you don't find any analytical reason to exclude »300%QC« (chromatography OK, IS response consistent,….) … you suspect a possible sample switch or problem (contamination?) during extraction…but you can't prove that.
All runs within study are accepted according to the 4/6 and 1/2 rule; however, if we calculate %CV and %Bias, we get the following results:
QC level 1 %CV=20(19.6); %Nominal=102 (not acceptable)
QC level 2 %CV=0; %Nominal=100
QC level 3 %CV=0; %Nominal=100According to the FDA guidelines (“Summary information on intra- and inter-assay values of QC samples and data on intra- and inter-assay accuracy and precision from calibration curves and QC samples used for accepting the analytical run.“), this nearly “ideal” method would be unsuitable due to only one out of 300 QCs failing really badly.
But is this method really not OK?
In general I agree with “no outliers policy”, but on the other hand, FDA “allows” some sort of the outliers: “Reported method validation data and the determination of accuracy and precision should include all outliers; however, calculations of accuracy and precision excluding values that are statistically determined as outliers can also be reported.”
If the “300%QC” would be rejected as an outlier, the method would become ideal.
Any ideas?
Regards
Marko
p.s. I tried to write it shorter, but I just couldn’t manage it.
Complete thread:
- outliers test for QC sample mathews 2007-10-11 07:58
- outliers test - but not for QCs! Helmut 2007-10-11 15:31
- Outlier test - but not for QC's yuvrajkatkar 2010-01-19 08:49
- Outlier test - but not for QC's H_Rotter 2010-01-19 12:55
- Outlier tests: forget it! Helmut 2010-01-19 17:36
- Outlier tests: forget it! Ohlbe 2010-01-19 19:00
- Outlier tests: forget it! Helmut 2010-01-19 19:16
- Outlier tests: forget it! Ohlbe 2010-01-19 21:00
- QCs vs. calibrators Helmut 2010-01-19 21:49
- QCs vs. calibrators Ohlbe 2010-01-20 19:01
- QCs vs. calibrators Helmut 2010-01-20 19:26
- QCs vs. calibrators Ohlbe 2010-01-20 19:52
- QCs vs. calibrators Helmut 2010-01-20 19:26
- QCs vs. calibrators Ohlbe 2010-01-20 19:01
- Two thirds of QCs H_Rotter 2010-01-20 16:21
- Two thirds of QCs Ohlbe 2010-01-20 18:49
- Outlier tests: forget it!....what if....moblak 2010-01-25 10:04
- Outlier tests: forget it!....what if.... Ohlbe 2010-01-25 23:19
- QCs vs. calibrators Helmut 2010-01-19 21:49
- Outlier tests: forget it! Ohlbe 2010-01-19 21:00
- Outlier tests: forget it! Helmut 2010-01-19 19:16
- Outlier tests: forget it! Ohlbe 2010-01-19 19:00
- Outlier test - but not for QC's yuvrajkatkar 2010-01-19 08:49
- outliers test - but not for QCs! Helmut 2007-10-11 15:31
