Bioequivalence and Bioavailability Forum

ramesh_7779 ★ 2010-04-02 18:36 (5499 d 17:22 ago) Posting: # 5001 Views: 8,851	WinNonlin PQ [Software] Post reply
	Dear All, Usually PQ compared by values output by software and hand calculation. Is it acceptable to compare the values calculated by excel and winnonlin software, while doing Performance Qualification. With Regards Raja

Jaime_R
★★

Barcelona,
2010-04-02 19:00
(5499 d 16:58 ago)

@ ramesh_7779
Posting: # 5002
Views: 8,107

WinNonlin PQ

Post reply

Dear Rajay,

you can compare everything to anything.
But do you seriously suggest to qualify WinNonlin by results obtained with Excel? How did you qualify the Excel-worksheet(s) in the first place? Please search the forum about the limitations of Excel in statistics.
I would suggest to first run all examples in WinNonlin's "Getting Started Guide" in order to document installation qualification and subsequently perform a black-box validation. Run evaluations for different designs based on published datasets. IMHO most published results are based on either SAS or S-Plus/R.
I would not rely on the validation kit provided by Pharsight (see here about the 2×2=5 problem).

—
Regards, Jaime

ramesh_7779
★

2010-04-03 20:11
(5498 d 15:48 ago)

@ Jaime_R
Posting: # 5013
Views: 7,882

WinNonlin PQ

Post reply

Dear Jamie,

Tahnks for your reply.

I think you did'nt get my question.

We used to compare the result from winnonlin with results from excel (Validated Excel). I feeded the formula in excel which i have taken fromula from winnonlin help.

In Previous company we used to compare the results from winnonlin software and results obtained by calculator(manual calculation by taking exmple of concentration data).

I want to know whether it is acceptable by regulatory agencies. IQ and OQ no problem. But doing PQ winnonlin is of question mark.

My doubt is how to do PQ in winnonlin.

With regards
Raja

Jaime_R
★★

Barcelona,
2010-04-04 01:09
(5498 d 10:50 ago)

@ ramesh_7779
Posting: # 5014
Views: 7,988

WinNonlin PQ

Post reply

Dear Raja,

please see this post first.

❝ Dear Jamie,

Jaime, not Jamie. ;-)

❝ I think you did'nt get my question.

I see.

❝ We used to compare the result from winnonlin with results from excel (Validated Excel). I feeded the formula in excel which i have taken fromula from winnonlin help.

Fine. Are you aware that there were typing errors both in the online-help and the PDF-manuals? You will find the known ones (sic!) in the "Release Notes" of each version. In the worst case two errors of opposite direction in WinNonlin may give the correct result. Or are you talking about WinNonlin's worksheet functions only? They are as buggy as Excel - and will give you the same (wrong) result if pushing to the limits. See this example and Simon's response.

How do you compare anything except the most simple calculations to Excel? There is no documentation how WinNonlin actually performs the calculation (yeah, it's a black box, like all competitors - except R). To quote from v5.2.1's Section on BE:

For nonreplicated crossover designs, the default model is as follows.
Fixed effects model is:
• DependentVariable = Intercept + Sequence + Formulation + Period
Random effects model is the nested term:
• Subject(Sequence)
There is no repeated specification, so the default error model eps~ N(0, sigma² I) will be used. This is equivalent to the classical analysis method, but using maximum likelihood instead of method of moments to estimate inter-subject variance. Using Subject as a random effect this way, the correct standard errors will be computed for sequence means and tests of sequence effects. Using a fixed effect model, one must construct pseudo-F tests by hand to accomplish the same task.
When this default model is used for a standard 2×2 crossover design, WinNonlin creates two additional tabs in the output called Sequential SS and Partial SS, which contain the degrees of freedom (DF), Sum of Squares (SS), Mean Squares (MS), F-statistic and p-value, for each of the model terms.
These tables are also included in the text output. Note that the F-statistic and p-value for the Sequence term are using the correct error term since Subject(Sequence) is a random effect.
If, in addition to using the default model, the data is also ln-transformed, then the Final Variance Parameters tab will contain two additional parameters:
Intersubject CV = sqrt( exp(Var(Subj(Seq)) - 1 )
Intrasubject CV = sqrt( exp(Residual_Variance) - 1 )
Note that, for this default model, Var(Subj(Seq)) is the intersubject (between subject) variance, and Residual_Variance is the intrasubject (within subject) variance.

Good luck in Excel.

—
Regards, Jaime

ramesh_7779 ★ 2010-04-04 11:19 (5498 d 00:40 ago) @ Jaime_R Posting: # 5015 Views: 7,879	WinNonlin PQ Post reply
	Dear Jaime, Thanks for reply, Suggest me whether to go a head with PQ or not. IQ, OQ is enough for winnonlin software. Tell me what regulatory agencies look for winnonlin and SAS software (regarding IQ, OQ and PQ) Give me endpoint. Regards Raja Edit: Linked to Jaime's post. [Helmut]

Nirali
★

India,
2010-04-05 10:52
(5497 d 01:07 ago)

(edited on 2010-04-05 14:18)
@ ramesh_7779
Posting: # 5018
Views: 7,962

WinNonlin PQ

Post reply

Dear Mr. Raja,

As per my knowledge, Regulatory looks for proper Installation, Operations and Performance of the software to assure that analysis has been carried out using valid procedures/equations/formulas.

I have faced audit from regulatory like- ANVISA, USFDA, FRANCE.
Definitely they may ask "Are you using validated software or validated programs?"

❝ We used to compare the result from winnonlin with results from excel (Validated Excel). I feeded the formula in excel which i have taken fromula from winnonlin help.

Instead of doing above exercise with another software-Microsoft Excel, you may validate your Procedures / programs by comparing your software results with reference data available in the published Books/Articles/guidelines.

For Example:
Health Canada Guidelines: "Guidance for Industry: Conduct and Analysis of Bio-availability and Bio-equivalence Studies - Part B: Oral Modified Release Formulations" contains complete data sets and its statistical analysis for Bio-equivalence. You may validate Bio-equivalence analysis of your software by comparing its output with this guideline output using data set given in it.

Same way, you may take data and its analysis from the book/articles and validate it with your software output :-)

I have documented PQ of WinNonlin and few programs of SAS as per above example and it is accepted by regulatory auditors.

Thanks & Regards,
Nirali

Jaime_R
★★

Barcelona,
2010-04-05 17:33
(5496 d 18:26 ago)

@ Nirali
Posting: # 5020
Views: 7,786

WinNonlin PQ

Post reply

Dear Nirali,

agree with your suggestions; that's what I told Raja in my first post as well.

It's important to compare results to different sources.
Canada's GL contains a faulty calculation of CV_intra (see this post); the error was carried over to 2009's draft (e.g. page 35, line 999).
See also this post for a typing error in Chow & Liu's book in all editions except the very first one. Since intra- and inter-subjects residuals are not calculated by WinNonlin (all versions) anyhow, this shouldn't bother Raja. :lol3:

—
Regards, Jaime

ramesh_7779 ★ 2010-04-05 17:57 (5496 d 18:02 ago) @ Nirali Posting: # 5021 Views: 7,792	WinNonlin PQ Post reply
	Dear Nirali, Thanks for your reply. Can you please help in doing Winnonlin PQ. Just now we are planning to do PQ for winnonlin if you dont mind plz help me. I dont know the procedures to go ahead. Kindly do the needful. With Regards Raja

Helmut
★★★

Vienna, Austria,
2010-04-05 18:04
(5496 d 17:54 ago)

@ ramesh_7779
Posting: # 5022
Views: 7,817

No e-mail in the forum!

Post reply

Dear Raja,

it's unsafe to leave an e-mail address in the net.
I have activated the e-mail link in your user profile - you may be contacted by registered users through the forum's contact-form right now. If you don't want to be contacted in the future any more, please alter your personal profile accordingly (bottom link: Edit User data).

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

SDavis
★★

UK,
2010-04-05 22:02
(5496 d 13:57 ago)

@ ramesh_7779
Posting: # 5023
Views: 7,844

WinNonlin PQ

Post reply

Dear Raja, Jaime,

I would just like to qualify your comment as an 'individual'.

❝ I would not rely on the validation kit provided by Pharsight (see here about the 2×2=5 problem).

Firstly to my knowledge there is not an error where 2x2 = 5, in WNL or VS; I think Helmut's point is that if you use the Validation Suite and IF there were actual calculation errors in the Reference sets then these would not be identified. I think this would be a common problem of Vendor supplied Validation kits, however I would stress that Pharsight takes considerable care to validate the original calculations against other Software for consistency and quality before either product is released.

However many of our users have requested a Validation Suite to assist in speeding up their IQ etc. so that is what we have provided. I would state that the next version which will run against Phoenix has several enhancements including continuously monitoring the installation files to ensure they have not be been changed, even after the actual Validation run has been executed.

I would certainly concur with Jaime that I would write in my own Validation plan my assumptions about using Validation Suite and state those extra checks that I might consider necessary to reassure myself that in a reasonable risk-based approach these are sufficient.

If you feel this is a conversation that would be useful with other Pharsight software users then I recommend discussing it on http://www.pharsight.com/extranet as discussed in my post last month.

Edit: URL linked. [Helmut]

—
Simon
Senior Scientific Trainer, Certara™
[link=https://www.youtube.com/watch?v=xX-yCO5Rzag[/link]
https://www.certarauniversity.com/dashboard
https://support.certara.com/forums/

Helmut
★★★

Vienna, Austria,
2010-04-05 23:39
(5496 d 12:20 ago)

@ SDavis
Posting: # 5024
Views: 7,917

Validation POV...

Post reply

Hi Simon!

❝ I think Helmut's point is that if you use the Validation Suite and IF there were actual calculation errors in the Reference sets then these would not be identified.

Exactly. I don't think that any vendor would write a validation pack in such a way that known problems are not detected. One the other hand if the vendor is confident that the correct result of 2×2 is 5, (s)he would test exactly for that result.
My original post referred to the funny idea to compare the output on the screen pixel-by-pixel to the reference machine's. ;-)

I know this rather 'innovative' approach to software validation was abandoned in the meantime.
BTW, what a coincidence: just returning form a three-hours escapade at Pharsight's extranet. I would bet that the Validation Suite 'believes' PHX's bloody a posteriori power is correct (see also this oldie).

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

ramesh_7779 ★ 2010-07-10 14:20 (5400 d 21:38 ago) @ Nirali Posting: # 5604 Views: 7,545	WinNonlin PQ Post reply
	Dear Ms. Nirali, I performed PQ of winnonlin by using Canada Part-B guidelines,but i am getting lot of variations in the result compare to guidelines. Is there any criteria to compare result. please let me know. With Regards Ramesh

Jaime_R
★★

Barcelona,
2010-07-11 21:40
(5399 d 14:18 ago)

@ ramesh_7779
Posting: # 5605
Views: 7,628

Canada...

Post reply

Dear Ramesh,

❝ I performed PQ of winnonlin by using Canada Part-B guidelines,but i am getting lot of variations in the result compare to guidelines. Is there any criteria to compare result. please let me know.

What do you mean by 'lot of variations'? We have discussed already in the past, that Canada's guideline contains a lot of typing errors. If you want to compare PHX/WNL to these results, you have to be cautious. If you estimate lambda_z, be sure not to use the automatic method based on the maximum R²_adj, but set datapoints individually (Tables 11-E/F: TLIN is the starting point and LQCT the end point). All (!!) starting points of the reference (Table 11-F) are wrong - they should be set to 22 instead of 12! Most subjects' profiles show double peaks; 12 hours is ridiculous. IMHO you cannot PQ WNL based on such an erroneous data set - unless you start clicking around to get a match by trial-and-error.

Next problem: NCA results (Tables 11-E/F) are given with limited precision. If you continue with WinNonlin's results you will get differences in the BE calculation. You have to round WNL's values to Canada's precison in order to match results... You can see this as an opportunity to learn WinNonlin's data transformation functions like ROUND(). BTW, you have to calculate the wrong CV_intra as the square-root of the residual MSE manually, because only the correct value is given in the output. ;-)

@Nirali: How did you...?

❝ documented PQ of WinNonlin and few programs of SAS as per above example and it is accepted by regulatory auditors.

—
Regards, Jaime

WinNonlin PQ [Software]

WinNonlin PQ

WinNonlin PQ

WinNonlin PQ

WinNonlin PQ

WinNonlin PQ

WinNonlin PQ

WinNonlin PQ

No e-mail in the forum!

WinNonlin PQ

Validation POV...

WinNonlin PQ

Canada...