ElMaestro
★★★

Denmark,
2017-07-14 12:37
(2449 d 03:32 ago)

Posting: # 17546
Views: 5,961
 

 Between-batch precision – a misnomer? [Bioanalytics]

Hi all,

I regularly look at bioanalytical validations where between-batch (between-run etc) precision is addressed.
Typically, the CRO or lab will study m batches (runs) with n injections of a given QC level. You can think of it as m vectors of length n containing concentration estimates.

Between-batch precision is typically evaluated by calculating the SD across all n x m values, then dividing by the grand mean, and that seems to generally pass a regulatory inspection.
But that isn't really between-batch precision, by any reasonable definition of between-batch, in my opinion. It is precision without regard to within and between, because the calculation sd does not take into account where the values came from.

If we were to truly evaluate between-batch precision (i.e. what is the CV of our estimate between batches) then we could study m batches, extract the m means and do precision on the means. The reason it isn't done in practice is apparently that it is too complicated or labour intensive (we'd need more than the usual m=3).

If we wish to accept companies just doing sd from the m x n values and using this to qualify the validation then we shouldn't call it between-batch precision. Or alternatively, if we wish to insist on working on an experiment called between-batch precision then we should calculate it differently.

That's my totallly unqualified opinion on this mighty pleasant Friday morning. I only had 1 L of covfefe so far so pardon me if any part of this post can resemble grumpiness. :-)

Pass or fail!
ElMaestro
nobody
nothing

2017-07-15 18:31
(2447 d 21:38 ago)

@ ElMaestro
Posting: # 17550
Views: 5,150
 

 Between-batch precision – a misnomer?

Although not specifically addressed **cough**, I would add to the discussion some old-school knowledge on validation:

Method validation in the bioanalytical laboratory. Buick AR, Doig MV, Jeal SC, Land GS, McDowall RD. J Pharm Biomed Anal. 1990;8(8-12):629-37.


Shah VP, Midha KK, Dighe S, et al. Analytical method validation: bioavailability, bioequivalence, and pharmacokinetic studies. J Pharm Sci. 1992;81:309–312.


What is determined by the methodology described would be "Vergleichspräzision" in Tschörman. It's an old tradition and describes some kind of precision on more than one day/one calibration. There are gazillions of ways to determine variabilities, including/excluding error propagation.
In the end of the day it's only kind of convention how to do it.

Usual way: Problem -> solution. If no problem, don't look for solution ;-)

Kindest regards, nobody
ElMaestro
★★★

Denmark,
2017-07-15 20:00
(2447 d 20:09 ago)

@ ElMaestro
Posting: # 17551
Views: 5,123
 

 State with a single sentence

OK, seeing that I am not able to make myself understood, I will take a different approach:

Can anyone state with a simple sentence which info you wish to extract from the experiment called "between-batch precision" (or inter-batch precision, inter-run etc)?

"The Between-batch experiment tells if [fill in this part of the sentence with words, not arithmetic terms like means, sd etc]".

Once that has been worded carefully (in more or less layman's language, preferably) let us discuss how calculate it meaningfully :-):-):-)

By the way: My concern applies also to between-batch accuracy, to some extent. Though it is harder to visualize, at least inside my walnut-sized brain.

Pass or fail!
ElMaestro
nobody
nothing

2017-07-15 22:03
(2447 d 18:06 ago)

@ ElMaestro
Posting: # 17552
Views: 5,064
 

 State with a single sentence

I would not put tooooo much effort into stuff like this, 'cause you want change anything in practice, if you are not FDA+EMA and some other "key opinion leaders" in one person ;-)

If you want to think some real weird stuff, try to wrap your head around the concept of "LLOQ". THERE are very strange things going on. :-D Fortunately under normal circumstances you have only few samples in this gray-area of reality.

Kindest regards, nobody
Ohlbe
★★★

France,
2017-07-16 04:23
(2447 d 11:46 ago)

@ ElMaestro
Posting: # 17553
Views: 5,077
 

 Yes, definitively

Dear ElMaestro,

❝ But that isn't really between-batch precision, by any reasonable definition of between-batch, in my opinion. It is precision without regard to within and between, because the calculation sd does not take into account where the values came from.


Absolutely. It is rather an overall, Grand Variability.

❝ If we were to truly evaluate between-batch precision (i.e. what is the CV of our estimate between batches) then we could study m batches, extract the m means and do precision on the means.


I'm not sure the information you would get from such calculation would be really meaningful. IMHO, calculating the mean of means does not reflect the variability of the method as applied to subject samples, which are analysed as singlicates.

If I remember correctly there were discussions on how to evaluate these data at the EBF/EUFEPS conference in Brussels in April 2010 while the EMA guideline was in public consultation. At that time, industry was suggesting to have more runs for precision and accuracy, but with less repetitions in each run. So instead of having 3 runs with 5 repetitions each, have one run with 5 repetitions for within-run P&A, and 4 or 5 other runs with just 2 or 3 repetitions. They said that otherwise it was just a repetition of within-run P&A and it was providing less information on between-run. Then, if I remember correctly it was suggested to have an ANOVA to extract the within- and between- components of the variability. I'm writing this from the back of my memory at a time when I should be sleeping in my bed, so I might be totally wrong.

(BTW, the comment was taken into consideration and that's why there is no figure for the number of replicates for between-run P&A in the EMA guideline. A minimum number of replicates is only given for within-run P&A).

❝ The reason it isn't done in practice is apparently that it is too complicated


Yeah, that's basically why the suggestion on the calculation was rejected. I'm not sure the regulators really knew what they would have done with such information and what the acceptance criteria should be ;-).

❝ If we wish to accept companies just doing sd from the m x n values and using this to qualify the validation then we shouldn't call it between-batch precision.


Strictly speaking, yes, you're right. But that's the terminology everybody is used to using, and I wouldn't be surprised to see it again in the ICH guideline.

❝ Or alternatively, if we wish to insist on working on an experiment called between-batch precision then we should calculate it differently.


I think regulators don't really care about the true contribution of the between-run effect on the overall variability of the method. What they are interested in is the overall Grand Variability itself. The question is kind of, is the method good or bad ?

Regards
Ohlbe
ElMaestro
★★★

Denmark,
2017-07-16 14:47
(2447 d 01:22 ago)

@ ElMaestro
Posting: # 17555
Views: 5,082
 

 Revised nomenclature

Hi all,

after the jury's careful deliberations, it is hereby proposed to rename it somewhat and call it
"The between-batch experiment which royally ignores if there are performance differences between batches."

In front of rolling cameras the chairman of the jury added in a sincere tone "...because we are interested in the performance of the method."

:-)

Pass or fail!
ElMaestro
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,638 registered users;
79 visitors (0 registered, 79 guests [including 9 identified bots]).
Forum time: 15:09 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5