Bioequivalence and Bioavailability Forum

Main page Policy/Terms of Use Abbreviations Latest Posts

 Log-in |  Register |  Search

Back to the forum  Query: 2017-10-17 05:44 CEST (UTC+2h)
 
ElMaestro
Hero

Denmark,
2017-07-14 10:37

Posting: # 17546
Views: 732
 

 Between-batch precision – a misnomer? [Bioanalytics]

Hi all,

I regularly look at bioanalytical validations where between-batch (between-run etc) precision is addressed.
Typically, the CRO or lab will study m batches (runs) with n injections of a given QC level. You can think of it as m vectors of length n containing concentration estimates.

Between-batch precision is typically evaluated by calculating the SD across all n x m values, then dividing by the grand mean, and that seems to generally pass a regulatory inspection.
But that isn't really between-batch precision, by any reasonable definition of between-batch, in my opinion. It is precision without regard to within and between, because the calculation sd does not take into account where the values came from.

If we were to truly evaluate between-batch precision (i.e. what is the CV of our estimate between batches) then we could study m batches, extract the m means and do precision on the means. The reason it isn't done in practice is apparently that it is too complicated or labour intensive (we'd need more than the usual m=3).

If we wish to accept companies just doing sd from the m x n values and using this to qualify the validation then we shouldn't call it between-batch precision. Or alternatively, if we wish to insist on working on an experiment called between-batch precision then we should calculate it differently.

That's my totallly unqualified opinion on this mighty pleasant Friday morning. I only had 1 L of covfefe so far so pardon me if any part of this post can resemble grumpiness. :-)

I could be wrong, but…


Best regards,
ElMaestro

- since June 2017 having an affair with the bootstrap.
nobody
Senior

2017-07-15 16:31

@ ElMaestro
Posting: # 17550
Views: 613
 

 Between-batch precision – a misnomer?

Although not specifically addressed **cough**, I would add to the discussion some old-school knowledge on validation:

Method validation in the bioanalytical laboratory. Buick AR, Doig MV, Jeal SC, Land GS, McDowall RD. J Pharm Biomed Anal. 1990;8(8-12):629-37.


Shah VP, Midha KK, Dighe S, et al. Analytical method validation: bioavailability, bioequivalence, and pharmacokinetic studies. J Pharm Sci. 1992;81:309–312.


What is determined by the methodology described would be "Vergleichspräzision" in Tschörman. It's an old tradition and describes some kind of precision on more than one day/one calibration. There are gazillions of ways to determine variabilities, including/excluding error propagation.
In the end of the day it's only kind of convention how to do it.

Usual way: Problem -> solution. If no problem, don't look for solution ;-)

Kindest regards, nobody
ElMaestro
Hero

Denmark,
2017-07-15 18:00

@ ElMaestro
Posting: # 17551
Views: 596
 

 State with a single sentence

OK, seeing that I am not able to make myself understood, I will take a different approach:

Can anyone state with a simple sentence which info you wish to extract from the experiment called "between-batch precision" (or inter-batch precision, inter-run etc)?

"The Between-batch experiment tells if [fill in this part of the sentence with words, not arithmetic terms like means, sd etc]".

Once that has been worded carefully (in more or less layman's language, preferably) let us discuss how calculate it meaningfully :-):-):-)

By the way: My concern applies also to between-batch accuracy, to some extent. Though it is harder to visualize, at least inside my walnut-sized brain.

I could be wrong, but…


Best regards,
ElMaestro

- since June 2017 having an affair with the bootstrap.
nobody
Senior

2017-07-15 20:03

@ ElMaestro
Posting: # 17552
Views: 588
 

 State with a single sentence

I would not put tooooo much effort into stuff like this, 'cause you want change anything in practice, if you are not FDA+EMA and some other "key opinion leaders" in one person ;-)

If you want to think some real weird stuff, try to wrap your head around the concept of "LLOQ". THERE are very strange things going on. :-D Fortunately under normal circumstances you have only few samples in this gray-area of reality.

Kindest regards, nobody
Ohlbe
Hero

France,
2017-07-16 02:23

@ ElMaestro
Posting: # 17553
Views: 575
 

 Yes, definitively

Dear ElMaestro,

» But that isn't really between-batch precision, by any reasonable definition of between-batch, in my opinion. It is precision without regard to within and between, because the calculation sd does not take into account where the values came from.

Absolutely. It is rather an overall, Grand Variability.

» If we were to truly evaluate between-batch precision (i.e. what is the CV of our estimate between batches) then we could study m batches, extract the m means and do precision on the means.

I'm not sure the information you would get from such calculation would be really meaningful. IMHO, calculating the mean of means does not reflect the variability of the method as applied to subject samples, which are analysed as singlicates.

If I remember correctly there were discussions on how to evaluate these data at the EBF/EUFEPS conference in Brussels in April 2010 while the EMA guideline was in public consultation. At that time, industry was suggesting to have more runs for precision and accuracy, but with less repetitions in each run. So instead of having 3 runs with 5 repetitions each, have one run with 5 repetitions for within-run P&A, and 4 or 5 other runs with just 2 or 3 repetitions. They said that otherwise it was just a repetition of within-run P&A and it was providing less information on between-run. Then, if I remember correctly it was suggested to have an ANOVA to extract the within- and between- components of the variability. I'm writing this from the back of my memory at a time when I should be sleeping in my bed, so I might be totally wrong.

(BTW, the comment was taken into consideration and that's why there is no figure for the number of replicates for between-run P&A in the EMA guideline. A minimum number of replicates is only given for within-run P&A).

» The reason it isn't done in practice is apparently that it is too complicated

Yeah, that's basically why the suggestion on the calculation was rejected. I'm not sure the regulators really knew what they would have done with such information and what the acceptance criteria should be ;-).

» If we wish to accept companies just doing sd from the m x n values and using this to qualify the validation then we shouldn't call it between-batch precision.

Strictly speaking, yes, you're right. But that's the terminology everybody is used to using, and I wouldn't be surprised to see it again in the ICH guideline.

» Or alternatively, if we wish to insist on working on an experiment called between-batch precision then we should calculate it differently.

I think regulators don't really care about the true contribution of the between-run effect on the overall variability of the method. What they are interested in is the overall Grand Variability itself. The question is kind of, is the method good or bad ?

Regards
Ohlbe
ElMaestro
Hero

Denmark,
2017-07-16 12:47

@ ElMaestro
Posting: # 17555
Views: 555
 

 Revised nomenclature

Hi all,

after the jury's careful deliberations, it is hereby proposed to rename it somewhat and call it
"The between-batch experiment which royally ignores if there are performance differences between batches."

In front of rolling cameras the chairman of the jury added in a sincere tone "...because we are interested in the performance of the method."

:-)

I could be wrong, but…


Best regards,
ElMaestro

- since June 2017 having an affair with the bootstrap.
Back to the forum Activity
 Thread view
Bioequivalence and Bioavailability Forum | Admin contact
17,394 Posts in 3,725 Threads, 1,071 registered users;
24 users online (0 registered, 24 guests).

The rise of biometry in this 20th century,
like that of geometry in the 3rd century before Christ,
seems to mark out one of the great ages or critical periods
in the advance of the human understanding.    R.A. Fisher

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
XHTML/CSS RSS Feed