moblak ☆ 2010-11-17 12:46 (5307 d 18:07 ago) Posting: # 6154 Views: 11,977 |
|
Hi everyone, our general approach for integration of peaks in chromatograms is one generic method per batch, no smoothing and "manual" integrations. However (as every analyst except regulators knows ![]() This time I will not focus on "manually" (though scientificaly sound) integrated chromatograms that the regulatory agencies are so afraid of ![]() Despite the fact that some agencies prefer the same integration method to be used for validation and study ![]() If we are lucky there is indeed the same method for validation and study, but in reality, we have to adjust certain parameters such as noise level, RT, peak width etc., but never apply smoothing (again historic regulatory tabu). If we took a rather conservative/regulated approch, which tests should/could be performed to justify that smoothing (how much?) could be applied? I was thinking of reintegrating 3xA/P (data used for between run A/P and establishment of regression model) and then comparing the results (raw vs. smoothed data). Since I am far from a statistician expert, how to evaluate the comparison (t-test....)? IMHO, only the "classical bioanalytical" %nominal and %CV wouldn't be enough. Anyway, would such a procedure be sufficient for regulatory agencies to allow smoothing or no-smoothing within the same validation or study. Does anyone have any real experience on this topic? Regards Marko |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2010-11-17 15:32 (5307 d 15:22 ago) @ moblak Posting: # 6155 Views: 10,863 |
|
Dear Marko! ❝ our general approach for integration of peaks in chromatograms is one generic method per batch, no smoothing and "manual" integrations. ❝ However (as every analyst except regulators knows Well, it’s a cumbersome task to find integration parameters suitable for an entire batch, when you have problems in the lower range. I don’t get the point why such a procedure (trial & error!) should be “better” than manual reintegration. ❝ This time I will […] focus on the "type" of integration method-"raw" data or/and smoothing. ❝ Despite the fact that some agencies prefer the same integration method to be used for validation and study, our approach remains one method per batch. ❝ If we are lucky there is indeed the same method for validation and study, but in reality, we have to adjust certain parameters such as noise level, RT, peak width etc., but never apply smoothing (again historic regulatory tabu). You are aware that you never use the raw signal of the detector? Without bunching the chromatogram (especially in LC-MS/MS) would not only look awful, but would be impossible to be integrated at all. Either there is an ion in the detector or not… See also this spring’s discussion at David’s PK-List. ❝ If we took a rather conservative/regulated approch, which tests should/could be performed to justify that smoothing (how much?) could be applied? ❝ I was thinking of reintegrating 3xA/P (data used for between run A/P and establishment of regression model) and then comparing the results (raw vs. smoothed data). Since I am far from a statistician expert, how to evaluate the comparison (t-test....)? IMHO, only the "classical bioanalytical" %nominal and %CV wouldn't be enough. Stupid question: A/P? One possibility would be an approach similar to the one used in cross-validation, namely orthogonal regression. You can test the slope for # 1 and the intercept for # 0. It’s important that you don’t use simple linear regression, because the main assumption there are error-free regressors (x-values). ❝ Anyway, would such a procedure be sufficient for regulatory agencies to allow smoothing or no-smoothing within the same validation or study. No idea. ❝ Does anyone have any real experience on this topic? Not me. ![]() — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
moblak ☆ 2010-11-17 16:26 (5307 d 14:28 ago) @ Helmut Posting: # 6156 Views: 10,720 |
|
Dear Helmut, ❝ You are aware that you never use the raw signal of the detector? That's why I wrote "raw" data. ❝ See also this spring’s discussion at David’s PK-List. A very nice discussion (thanks), but no closures for me ![]() ❝ Stupid question: A/P? Accuracy/Precision By clicking on the link I just got formulaphobia ![]() ❝ You can test the slope for # 1 and the intercept for # 0. I am not sure what do you mean. Could you please explain in more detail? Btw. Do you use smoothing for data processing of your LC-MS/MS chromatograms. Regards Marko |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2010-11-17 17:31 (5307 d 13:23 ago) @ moblak Posting: # 6158 Views: 10,981 |
|
Dear Marko! ❝ ❝ You are aware that you never use the raw signal of the detector? ❝ That's why I wrote "raw" data. Again – what do you mean by “raw data”? ❝ ❝ Stupid question: A/P? ❝ Accuracy/Precision I see. ![]() ❝ By clicking on the link I just got formulaphobia Nice term, but not sooo tough.* I you have R, it would boil down to something like this (on back-calculated concentrations, not A/P): x <- c(1.00,1.10,0.85, 13.0,14.2,15.9, 210,215,190) gives [1] 0.4675463 compared to linear regression linear <- lm(y~x) gives Call: ❝ ❝ You can test the slope for # 1 and the intercept for # 0. ❝ I am not sure what do you mean. Could you please explain in more detail? In the linear model above you get not only the estimated slope and intercept but also their standard errors. You can test the estimates against 0 and 1 by means of these SEs. The (1-α) confidence interval is given by [a,b]±tn-2,1-α/2×SE[a,b]. Now look whether the CI for a includes 0 and the CI of b includes 1. If you find a significant intercept: constant = additive bias (independent from concentration) Significant slope: proportional bias (dependent on concentration). If both are not significant, the methods perform equally well. I’m a little bit short in time to come up with code for SEs in orthogonal regression. There’s a package for R ‘MethComp’ (download), which is not available form CRAN right now. It contains the method ‘Deming’ – it should be possible to extract the standard-errors or do some jackknife. The simple call gives the same results as my code above: Deming(x,y) ❝ Btw. Do you use smoothing for data processing of your LC-MS/MS chromatograms. Well, I left the lab some good years ago… But many CROs I know do so – as long as the resolution of peaks is not negatively affected.
Edit: After reading some stuff, one thing is clear: Never use a t-test in method comparisons! You will only detect a constant bias, but not a proportional one. Package ‘MethComp’ is really nice. Contains even Bland-Altman-Plots. The SEs of the Deming-regression are estimated by bootstrapping. Example: require(MethComp) will give Estimate S.e.(boot) 50% 2.5% 97.5% 0 is well within the 95% confidence interval = no constant bias. 1 is within the CI, but only borderline (you may repeat the bootstrap or request a high number; if you set boot=TRUE the default of 1000 samples is used). Now let’s get at a plot:plot(x,y, xlim=c(0,max(x,y)), ylim=c(0,max(x,y)), plot(x,y, xlim=c(0,16.2), ylim=c(0,16.2), If you don’t have R installed – or it will take ages until your IT department does it for you – you can post a dataset. I would suggest to include back-calculated calibrators and QCs from your ‘raw’ integration and the same dataset ‘smoothed’… — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
moblak ☆ 2010-11-18 15:39 (5306 d 15:15 ago) @ Helmut Posting: # 6167 Views: 10,613 |
|
Hi Helmut, thanks for the explanation...things are getting clearer (I must admit though it's not 100% clear yet...I'll get there in time....eventually) ❝ If you don’t have R installed – or it will take ages until your IT department does it for you I rather not comment on this topic ![]() ❝ you can post a dataset. I really appreciate your help ![]() ![]() Below please find the dataset: Nominal "raw" "smooth" Thanks again for your help Regards Marko |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2010-11-18 16:33 (5306 d 14:21 ago) @ moblak Posting: # 6168 Views: 10,654 |
|
Dear Marko! ❝ ❝ If […] it will take ages until your IT department does it for you ❝ I rather not comment on this topic Well, as the forum’s admin I know your e-mail address, so I can make an educated guess about your company’s IT department. Accept my deep sympathy. ![]() From a regulatory perspective I would assume that the decision on the method should be done only on the calibrators. I was asking for the entire data, because I’m insatiable… I increased the number of bootstrap samples to 50,000. Calibrators: Estimate S.e.(boot) 50% 2.5% 97.5% No constant (0 within 95% CI) and no proportional (1 within CI) bias; methods perform equally well. ![]() QCs: Estimate S.e.(boot) 50% 2.5% 97.5% No constant (0 within CI) and no proportional (1 within CI) bias; methods perform equally well. ![]() Calibrators + QCs: Estimate S.e.(boot) 50% 2.5% 97.5% No constant (0 within CI) and no proportional (1 within CI) bias; methods perform equally well. ![]() — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2010-11-18 19:35 (5306 d 11:19 ago) @ moblak Posting: # 6170 Views: 10,942 |
|
Hi Marko! Another reply; will be lenghty… Let’s dive in. Bootstrapping is nice, but as a random resampling procedure it will not give you the same results if repeated (I would guess, regulators would hate that). Below five runs (calibrators) of 50,000 samples each: Estimate S.e.(boot) 50% 2.5% 97.5% You see that standard errors – and thus the CI – are similar, but not identical (although the decision would always be the same). An alternative would be the Jackknife, because you will always get the same results. Unfortunally there is no package readily available, but I found some code by Terry Therneau in the R-help archives. # Generalized Deming regression, based on Ripley, Analyst, 1987:377-383. For your calibration data: fit <- deming(x,y, xstd=c(1,0), ystd=c(1,0), jackknife=TRUE) I got: Call: Hhm, strange. Intercept and slope are not the same as with the other procedures?! Ah, some weighting is used… Let’s calculate the 95% confidence interval based on the t-distribution anyhow: df <- length(x)-2 I got: Intercept (95% CI): -0.1573358 0.5303742 Again no constant and/or proportional bias. If I have time (haha), I will check at R-help about the weighting. If I set all weights=1, I get still different results… — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
moblak ☆ 2010-11-19 10:25 (5305 d 20:29 ago) @ Helmut Posting: # 6173 Views: 10,542 |
|
Hi Helmut, Now I see how a scientific torture(r) might look like. But seriously, thanks for taking your time and helping me with the statistical calculations. Though I am not sure, I will be able to apply this routinely by myself... I might just either smooth or not smooth ![]() Regards Marko |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2010-11-19 14:47 (5305 d 16:07 ago) @ moblak Posting: # 6175 Views: 10,900 |
|
Dear Marko! ❝ Now I see how a scientific torture(r) might look like. My dear, science is a cruel mistress. ❝ But seriously, thanks for taking your time and helping me with the statistical calculations. Welcome! ❝ Though I am not sure, I will be able to apply this routinely by myself… Maybe your IT-gurus are more happy with a commercial solution. Two products offer 30-day trial downloads. Analyse-it (an Excel ![]() ❝ I might just either smooth or not smooth In your example, there was no difference. If you smooth, how much? As Ohlbe said, resolution must not be affected. Contrary to bundling, smoothing should not change the shape of peaks that much. Last but not least: It seems that this is just an issue with ANVISA! — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
d_labes ★★★ Berlin, Germany, 2010-11-19 17:00 (5305 d 13:54 ago) @ Helmut Posting: # 6176 Views: 10,535 |
|
Hi Helmut! ❝ Hhm, strange. Intercept and slope are not the same as with the other procedures?! Ah, some weighting is used... You set the weights to 1 with your function call! ❝ If I have time (haha), I will check at R-help about the weighting. If I set all weights=1, I get still different results... Save your time for nicer things, f.i. NLYW ![]() Set the call to minimizing the weighted sum of squares in the function to: fit <- optimize(minfun, c(0.1, 10), x=x, y=y, xv=xstd^2, yv=ystd^2, tol=1.e-8) and look what happens ![]() Have a nice weekend. — Regards, Detlew |
Helmut ★★★ ![]() ![]() Vienna, Austria, 2010-11-19 17:19 (5305 d 13:35 ago) @ d_labes Posting: # 6177 Views: 10,627 |
|
Dear D. Labes! ❝ Save your time for nicer things, f.i. NLYW Oh, dear! I'm not sure whether my girlfriend would like that - especially the Y in NLYW. ![]() ❝ Set the call to minimizing the weighted sum of squares in the function to: ❝ Great! For the archive: Call: Code for the plot with the fit and its CI:
![]() You could also ask whether the smoothed results (y) are within the 95% CI of the fit: CI <- CI[1:length(x), ]
for (j in 1:length(x)) { — Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![]() Helmut Schütz ![]() The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
Ohlbe ★★★ France, 2010-11-18 02:39 (5307 d 04:15 ago) @ moblak Posting: # 6160 Views: 10,621 |
|
Dear Marko, ❝ However (as every analyst except regulators knows Agreed, particularly with low concentrations and noisy baselines (which you will get without smoothing). ❝ we (...) never apply smoothing (again historic regulatory tabu). ![]() I agree with Helmut: it can be challenging or even impossible to integrate chromatograms properly without some level of smoothing. Though I agree that smoothing should remain reasonable (the idea is not to mask chromatographic problems and shoulders which could be due to interferences !), it is needed. One question could be whether to authorise changes in smoothing within a study, if one run is noisier than another. Within a run I would consider that the smoothing should remain the same for all samples (it will affect peak area and therefore the concentration). Between runs I would probably try all other parameters first... Regards Ohlbe — Regards Ohlbe |
moblak ☆ 2010-11-18 11:57 (5306 d 18:57 ago) @ Ohlbe Posting: # 6164 Views: 10,560 |
|
Dear Ohlbe, ❝ Ohlbe ❝ France, ❝ 2010-11-18 01:39 Working late? ![]() ❝ Are you thinking of any specific agency ? ANVISA... Now I have to go through Helmut's response... it will take some time. Regards Marko |
Ohlbe ★★★ France, 2010-11-18 14:58 (5306 d 15:56 ago) @ moblak Posting: # 6165 Views: 10,747 |
|
Dear Marko, ❝ Working late? I was in the US, it was not that late there... ❝ ANVISA... Gosh... they really are special ! There was a session on global harmonisation of bioanalytical guidelines and expectations at the PSWC2010 (FIP/AAPS annual congress) in New Orleans on Monday. Eric Woolf from Merck gave a presentation with industry's position. His presentation was not really on US vs. EMA (current hot topic with the draft EMA guideline and FDA revising its own guidance) but rather on problems with other countries. He did not name any but it was quite obvious from the examples he gave that he was pointing at Brazil... By the way, if I understood correctly the presentation from Ariadna Barra (ANVISA) at the same session, ANVISA will also update their bioanalytical guidance and some aspects will get closer to US/EMA. Regards Ohlbe — Regards Ohlbe |