yicaoting
★    

NanKing, China,
2017-08-22 20:01
(2409 d 17:27 ago)

Posting: # 17736
Views: 16,786
 

 Validation of FDA's RSABE on NTIDs [RSABE / ABEL]

Dear Helmut,
Hope you are well.

I am trying to manually calculate the upper 90% CL of sWT/sWR in US FDA's RSABE for NTIDs(Draft Guidance on Warfarin).

I use BEBAC's sample data (page 14 of the PPT from this site) to validate/verify my calculation steps.

Data set link
http://bebac.at/downloads/NTID.xls

It will be greatly appreciated if you can help me on the following questions:
1. I manually calculate the sWR and sWT in Excel sheet, they are 0.124392769, 0.055720909, respectively. Are my results EXACTELY right?

2. In your slides, you have present the upper 95% CL of sWT/sWR 0.68427 ≤2.5.
I have calculated
lower 95% CL of sWT/sWR = 0.293238292
upper 95% CL of sWT/sWR = 0.684266753 (this is nearly the same as your result)

As US FDA's Guidance on Warfarin required, the upper 90% CI should be used, I also calculate the following:
lower 90% CL of sWT/sWR = 0.322597242
upper 90% CL of sWT/sWR = 0.621992961

Are the above results right?

I use the function Finv(0.1,16,16) and Finv(0.9,16,16) for the calculation of 90% CL of sWT/sWR,
and Finv(0.05,16,16) and Finv(0.95,16,16) for the calculation of 95% CL of sWT/sWR. Do I use the right function?

2. Would you please guide (step by step) me how to calculate CVwr CVwt? As shown in your slides, they are CVWR 12.49%, CVWT 5.58%. Although these two values are not used in calculation of "upper 90% CL of sWT/sWR".

Thank you for your great sample dataset which provides us a valuable bridge for us to learn and discuss this sparely used procedure.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-08-22 22:25
(2409 d 15:03 ago)

@ yicaoting
Posting: # 17739
Views: 15,397
 

 Validation of FDA's RSABE on NTIDs

Hi Zhang Yong,

❝ Hope you are well.


THX! Below my results obtained in Phoenix WinNonlin 7.0 in full precision.

❝ I manually calculate the sWR and sWT in Excel sheet, they are 0.124392769, 0.055720909, respectively. Are my results EXACTELY right?


swR 0.124392768691665, swT 0.0557209092013223

❝ In your slides, you have present the upper 95% CL of sWT/sWR 0.68427 ≤2.5.

❝ I have calculated

❝ lower 95% CL of sWT/sWR = 0.293238292

❝ upper 95% CL of sWT/sWR = 0.684266753 (this is nearly the same as your result)


lower CL 0.293238291752988, upper CL 0.684266752752176

❝ As US FDA's Guidance on Warfarin required, the upper 90% CI should be used, I also calculate the following:

❝ lower 90% CL of sWT/sWR = 0.322597242

❝ upper 90% CL of sWT/sWR = 0.621992961

❝ Are the above results right?


The guidance might be confusing. See the last bullet point below the formula
  • here α = 0.1.
Hence, 1–0.05 and 1–0.95 are correct.

❝ I use the function Finv(0.1,16,16) and Finv(0.9,16,16) for the calculation of 90% CL of sWT/sWR,

❝ and Finv(0.05,16,16) and Finv(0.95,16,16) for the calculation of 95% CL of sWT/sWR. Do I use the right function?


I have only Excel 2000. :-D Up to v2003 the inverse distributions were wrong. Maybe you have to use F.inv(alpha,df1,df1) or the old workaround Finv(2*alpha,df1,df1). Duno. The correct F-values (ν12=16) are:
Fα∕2,ν12 2.33348362746764, F1−α∕2,ν12 0.428543825304327

❝ Would you please guide (step by step) me how to calculate CVwr CVwt? As shown in your slides, they are CVWR 12.49%, CVWT 5.58%. Although these two values are not used in calculation of "upper 90% CL of sWT/sWR".


According to the guidance sWR and swT are estimated from complete data only (not an issue with this data set) ignoring its structure (solely 'sequence' in the linear model). Your values are correct. Hence, as usual \(CV = \sqrt{e^{s_{w}^{2}} - 1}\). Therefore, we get CVwR 12.49% and CVwT 5.58%.

Personally I would prefer to run a mixed effects model with restricted maximum likelihood which takes the entire information into account (i.e., the FDA’s code of the 2001 guidance and also in the ABE-part of the progesterone guidance). In this model you could have incomplete data and the variances of R and T are simultaneously estimated. I guess that’s impossible in Excel (as it is in R)…
I got: CVwR 15.86% and CVwT 5.73%. Interesting.

Hope that helps.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
yicaoting
★    

NanKing, China,
2017-08-23 07:30
(2409 d 05:58 ago)

@ Helmut
Posting: # 17742
Views: 15,167
 

 Validation of FDA's RSABE on NTIDs

Hi, Helmut.
Thank you for your quick response.

❝ THX! Below my results obtained in Phoenix WinNonlin 7.0 in full precision.

❝ swR 0.124392768691665, swT 0.0557209092013223


Yes, sWR and sWT can be calculate manually. So my results are indentical to yours.

❝ lower CL 0.293238291752988, upper CL 0.684266752752176

Thay are identical to mine.


❝ The guidance might be confusing. See the last bullet point below the formula

  • here α = 0.1.
Hence, 1–0.05 and 1–0.95 are correct.

I am not sure the guidance is correct or not.
But from the formula in the guidance (page 4, line 5) with Alpha=0.1, I guess the guidance is right, it calulates 90% CI of Swt/Swr, but not 95% CI.

Need your clarification? or FDA is wrong?

❝ I have only Excel 2000. :-D Up to v2003 the inverse distributions were wrong. Maybe you have to use F.inv(alpha,df1,df1) or the old workaround Finv(2*alpha,df1,df1). Duno. The correct F-values (ν12=16) are:

Fα∕2,ν12 2.33348362746764, F1−α∕2,ν12 0.428543825304327


In Excel 2003,
Finv(0.05,16,16) = 2.33348362835
Finv(0.95,16,16) = 0.428543825142279

In Excel 2013
Finv(0.05,16,16) = 2.33348362746764
Finv(0.95,16,16) = 0.428543825304327
F.inv(0.05,16,16) = 0.428543825304327
F.inv(0.95,16,16) = 2.33348362746764

❝ According to the guidance sWR and swT are estimated from complete data only (not an issue with this data set) ignoring its structure (solely 'sequence' in the linear model). Your values are correct. Hence, as usual \(CV = \sqrt{e^{s_{w}^{2}} - 1}\). Therefore, we get CVwR 12.49% and CVwT 5.58%.


Thanks.

❝ Personally I would prefer to run a mixed effects model with restricted maximum likelihood which takes the entire information into account (i.e., the FDA’s code of the 2001 guidance and also in the ABE-part of the progesterone guidance). In this model you could have incomplete data and the variances of R and T are simultaneously estimated. I guess that’s impossible in Excel (as it is in R)…

❝ I got: CVwR 15.86% and CVwT 5.73%. Interesting.


I guess your result of 15.86% and 5.73% are obtain from the sheet of "Final Variance Parameters" from ABE results using PHX WNL.

let
a = Var(Period*Formulation*Subject)_21 (the value is 0.024828963992548 from PHX WNL)
b = Var(Period*Formulation*Subject)_22 (the value is 0.003281299568883 from PHX WNL)
CVwr = 100 * SQRT(EXP(a)-1)
CVwt = 100 * SQRT(EXP(b)-1)

I got the same results sa you.

Here the CVwr and CVwt are different from those calculated from Swr and Swt.
I prefer the result calculated from Swr and Swt.
What's your opinion?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-08-23 17:00
(2408 d 20:28 ago)

@ yicaoting
Posting: # 17744
Views: 15,279
 

 Validation of FDA's RSABE on NTIDs

Hi Zhang Yong,

❝ But from the formula in the guidance (page 4, line 5) with Alpha=0.1, I guess the guidance is right, it calulates 90% CI of Swt/Swr, but not 95% CI.

❝ Need your clarification? or FDA is wrong?


The FDA is never wrong. But so am I. :-D The guidance asks to calculate an confidence interval but only the upper confidence limit is assessed. Simply: The 90% CI has an upper and lower 95% CL.

❝ ❝ Personally I would prefer […] to run a mixed effects model with restricted maximum likelihood…

❝ ❝ I got: CVwR 15.86% and CVwT 5.73%. Interesting.


❝ I guess your result of 15.86% and 5.73% are obtain from the sheet of "Final Variance Parameters" from ABE results using PHX WNL.


Correct.

❝ Here the CVwr and CVwt are different from those calculated from Swr and Swt.

❝ I prefer the result calculated from Swr and Swt.

❝ What's your opinion?


For my preferences see above. If you have to follow the guidance that’s not an option.

BTW, if you have Phoenix WinNonlin why bother fiddling around in Excel? I have some doubts whether an agency will accept that. At least the EMA wouldn’t. Quoting the Q&A document:

3.3. Alternative computer programs
[…] Results obtained by alternative, validated statistical programs are also acceptable except spreadsheets because outputs of spreadsheets are not suitable for secondary assessment.

BTW, Certara is working on a validated template for PHX WNL and it will be published in the near future.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
yicaoting
★    

NanKing, China,
2017-08-30 16:33
(2401 d 20:55 ago)

@ Helmut
Posting: # 17756
Views: 15,173
 

 Validation of FDA's RSABE on NTIDs

Hi Helmut
For validation of statistical procedure or software in analysis of BE data of NTID. I am eagerly hope you publish such reference datasets, like those two papers published in AAPS J in 2015 for two way crossover and parallel design.
If you wish, please tell me If i can provide any help.
I now use WNL PHX and SAS and/or Excel to analysis NTID. Excel is used just for SigmaWR and SigmaWR/SigmaWT and it's upper llevel of %CI, not for ABE or RSABE in such case.

BTW, I have developed a program in VBA in Excel for BE analysis, it has passed all 2*2*2 and 2‖2 reference datasets you provided. This program is not intended for commercialization, but will be published for free use in future. Like PKSolver and DDSolver which have been published.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-08-30 17:04
(2401 d 20:23 ago)

@ yicaoting
Posting: # 17757
Views: 15,023
 

 Validation of FDA's RSABE on NTIDs

Hi Yong,

❝ I am eagerly hope you publish such reference datasets, like those two papers published in AAPS J in 2015 for two way crossover and parallel design.


We are working on a paper for the EMA’s ABEL. ;-) We will also provide an R-package. It is still a work in progress. Its current state on GitHub. In the inst/extdata folder our reference data sets. NTIDs are DS02.csv (TRR|RTR|RRT), DS05.csv (TRRT|RTTR), and DS10.csv (TRR|RTT). Sorry, none with TRTR|RTRT…

❝ I now use WNL PHX and SAS and/or Excel to analysis NTID. Excel is used just for SigmaWR and SigmaWR/SigmaWT and it's upper llevel of %CI, not for ABE or RSABE in such case.


Our results with replicateBE and the latest template for PHX/WNL:
DS05.csv: swT∕swR 1.0184, upper CL of σwTσwR 1.4344.
DS10.csv: swT∕swR 1.2566, upper CL of σwTσwR 2.3300.

❝ BTW, I have developed a program in VBA in Excel for BE analysis, it has passed all 2*2*2 and 2‖2 reference datasets you provided. This program is not intended for commercialization, but will be published for free use in future. Like PKSolver and DDSolver which have been published.


Great!

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
CECIF
☆    

Colombia,
2018-02-19 23:20
(2228 d 13:08 ago)

@ Helmut
Posting: # 18436
Views: 13,763
 

 Clarification needed: Validation of FDA's RSABE on NTIDs

Hello,
I need some…clarifying, if you will.
I am also, trying to replicate the FDA guidance on warfarin. I am using R. However I am stuck in steps 2 and 3.
In step 2 I am able to calculate ((YR-YT)^2)-theta*swR2, but it says I should find the upper 95% CI using Howe´s method. Sincerely I just read Howe's method (from the reference in the FDA guidance) for like 4 hours and still can't understand how to use it (maybe it´s because I am not a certified statistician). Can somebody give me a step by step on how to do this?.
An additional comment on step 2: once I find the upper 95% CI, What I am supposed do to with it? it does not seem to appear on the subsequent steps (or at least not alone). I just saw your presentation on RSABEs, and you use the criteria "Bioequivalent If 95% upper CI <0" where does this criterion comes from?
In step 3 “every study should pass the scaled and unscaled and scaled bioequivalence limits”: “every study" refers to:
1. Period 1 vs period 2 (regular bioequivalence study)
2. Period 3 vs period 4 (replicated study)
3. Pooled analysis (Average periods 1 and 3 vs average period 2 and 4)
4. Repeated crossover analysis considering all four periods
Did I miss any?
And “scaled bioequivalence limits” refers to the upper 95% CI (supposedly found in step 2) with an additional (and respective) lower 95% CI?

In the study 4 mentioned above is it correct to use the same methods for calculation of BE limits as the others, but adding another factor for the variable of “replication” in the ANOVA?

And finally, is it necessary to comply with all four steps (and multiple criteria) of this guidance to demonstrate bioequivalence? What happens for example if one of them fails but the other 3 are correct?

I would really appreciate any light you can shed.
d_labes
★★★

Berlin, Germany,
2018-02-20 15:10
(2227 d 21:18 ago)

@ CECIF
Posting: # 18439
Views: 13,705
 

 FDA's RSABE on NTIDs in R

Dear CECIF,

❝ I need some…clarifying, if you will.


I will try my best :cool:.

❝ I am also, trying to replicate the FDA guidance on warfarin. I am using R. However I am stuck in steps 2 and 3.

❝ In step 2 I am able to calculate ((YR-YT)^2)-theta*swR2, but it says I should find the upper 95% CI using Howe´s method. Sincerely I just read Howe's method (from the reference in the FDA guidance) for like 4 hours and still can't understand how to use it (maybe it´s because I am not a certified statistician). Can somebody give me a step by step on how to do this?.


Since you are working with R have a look at this thread, especially at this post.
See here and especially here for some discussions about how to calculate the scaled ABE criterion (linearized) including it's upper 95% CI according to Hyslop's method, sometimes the name Howe is also used.

❝ An additional comment on step 2: once I find the upper 95% CI, What I am supposed do to with it? it does not seem to appear on the subsequent steps (or at least not alone). I just saw your presentation on RSABEs, and you use the criteria "Bioequivalent If 95% upper CI <0" where does this criterion comes from?


This is simple the BE decision to be taken based on the upper 95% CI of the linearized RSABE criterion. Read papers on HVD to understand where it comes from.

❝ In step 3 “every study should pass the scaled and unscaled and scaled bioequivalence limits”: “every study" refers to:

❝ 1. Period 1 vs period 2 (regular bioequivalence study)

❝ 2. Period 3 vs period 4 (replicated study)

❝ 3. Pooled analysis (Average periods 1 and 3 vs average period 2 and 4)

❝ 4. Repeated crossover analysis considering all four periods

❝ Did I miss any?


No, you don't miss. But your understanding is a miss-understanding.
The sentence means: The study under consideration must pass the following criteria:
  • Scaled ABE test: 95% CI of RSABE criterion <0
  • Conventional ABE test: 90% CIs within acceptance range
  • Upper 90% CI of the ratio of intra-subject standard deviations of Test and Reference <= 2.5

❝ In the study 4 mentioned above is it correct to use the same methods for calculation of BE limits as the others, but adding another factor for the variable of “replication” in the ANOVA?


No, no additional factor.
But in the Warfarine guidance a mixed model analysis (SAS Proc MIXED code) is recommended.
Be aware, this is not implementable in R one to one.
What you can do is: Use the 90% CI you already has calculated during calculation of the 90% CI of the RSABE criterion. I myselve never understood why the FDA uses two different methods, one for calculating the RSABE criterion and another one for calculating the conventional 90% CIs :no:.

❝ And finally, is it necessary to comply with all four steps (and multiple criteria) of this guidance to demonstrate bioequivalence? ...


Yes.

Hope this answer helps.

Regards,

Detlew
CECIF
☆    

Colombia,
2018-03-01 22:08
(2218 d 14:19 ago)

@ d_labes
Posting: # 18491
Views: 13,305
 

 FDA's RSABE on NTIDs in R

Dear Detlew

❝ I will try my best :cool:.


Felt like life-saving answer for me.

So I used your R code from the thread you sent me (Marvelous coding btw), on Helmut's datafile http://bebac.at/downloads/NTID.xls

And these are my results:

> FDA_Narrow_CECIF(filename)
        SwR ABE.Lower ABE.upper ratioDS_IC.lower ratioDS_IC.upper     uppRSABE
1 0.1243928 0.9429483  1.029172        0.2932383        0.6842668 -0.009828289


Now, If I understand correctly this is bioequivalent because:

1. 95% CI of RSABE criterion= -0.009828289 <0
2. Conventional ABE test: 0.9 < 0.9429483, 1.11>1.029172 (or is it also with conventional limits 0.8 and 1.25?).
3. Upper 90% CI of the ratio swT/swR: 0.6842668<=2.5

In addition to this, do I still have to demonstrate conventional bioequivalence for:

❝ 1. Period 1 vs period 2 (regular bioequivalence study)

❝ 2. Period 3 vs period 4 (replicated study)

❝ 3. Pooled analysis (Average periods 1 and 3 vs average period 2 and 4)

?
as they do here? https://www.accessdata.fda.gov/drugsatfda_docs/anda/99/40301_Warfarin%20Sodium_Approv.pdf

In any case, many thanks.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2018-03-02 00:39
(2218 d 11:49 ago)

@ CECIF
Posting: # 18492
Views: 13,353
 

 FDA's RSABE on NTIDs in R

Dear CECIF,

❝ Now, If I understand correctly this is bioequivalent because:


❝ 1. 95% CI of RSABE criterion= -0.009828289 <0


Correct.

❝ 2. Conventional ABE test: 0.9 < 0.9429483, 1.11>1.029172 (or is it also with conventional limits 0.8 and 1.25?).


As Detlew already pointed out: 80–125%. See the guidance:

Use the unscaled average bioequivalence procedure to determine BE for individual PK parameter(s). […] should pass […] also regular unscaled bioequivalence limits of 80.00-125.00%.


❝ 3. Upper 90% CI of the ratio swT/swR: 0.6842668<=2.5


Correct.

❝ In addition to this, do I still have to demonstrate conventional bioequivalence for:

❝ ❝ 1. Period 1 vs period 2 (regular bioequivalence study)

❝ ❝ 2. Period 3 vs period 4 (replicated study)

❝ ❝ 3. Pooled analysis (Average periods 1 and 3 vs average period 2 and 4)

❝ ?

❝ as they do here? https://www.accessdata.fda.gov/drugsatfda_docs/anda/99/40301_Warfarin%20Sodium_Approv.pdf


Again, as Detlew wrote: No.
BTW, I don’t see such a (strange) requirement in the ANDA of 1999 (!) you linked.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,639 registered users;
74 visitors (0 registered, 74 guests [including 7 identified bots]).
Forum time: 12:28 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5