usfda_emea
●    

2007-03-06 15:39
(6232 d 03:05 ago)

Posting: # 559
Views: 32,526
 

 Confidence Interval for Transformed data Unbalanced study [General Sta­tis­tics]

Dear,

I am looking for help on estimation of 90% confidence interval for log transformed data of AUC (0-inf) for a bioequivalence study. This study involved two-way crossover, two-period, two-sequence design with a total of 33 subjects completing the study. Each sequence had Period I TR=16 and RT=17, and in Period II TR=17 and RT=16; thus, total of 33 completed the study.

In order to estimate the 90% confidence interval, can some one help me with the formula and a worked out example for the ease of understanding. OR Guide me to a site where this can be found.

I believe least-squares mean estimate for each formulation needs to be used to form the difference, together with the standard error. However, in lack of exact formula i am seeking the help of experts out there.

Thanks in advance for the help.
USFDA_EMEA


Edit: Category changed. [Helmut]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2007-03-06 17:46
(6232 d 00:58 ago)

@ usfda_emea
Posting: # 560
Views: 29,209
 

 CI for Transformed data Unbalanced study

Dear USFDA_EMEA!

I guess that’s not your real name :-D

❝ I am looking for help on estimation of 90% confidence interval for log transformed data of AUC (0-inf) for a bioequivalence study. This study involved two-way crossover, two-period, two-sequence design with a total of 33 subjects completing the study. Each sequence had Period I TR=16 and RT=17, and in Period II TR=17 and RT=16; thus, total of 33 completed the study.


OK, the number of subjects in each sequence is n1 and n2, respectively, where n = n1 + n2.
In your case:

n1 = 17
n2 = 16
n  = 33


The assignment of sequence 1 to RT follows the literature, but is only a convention (if you want you may call TR sequence 1 as long as you keep this convention through all calculations…)

❝ In order to estimate the 90% confidence interval, can some one help me with the formula and a worked out example for the ease of understanding. OR Guide me to a site where this can be found.


Let’s see. Which formula are you using right now?
If you have something like 1/n in it, for unbalanced data you have to replace it by 1/n1 + 1/n2.

❝ I believe least-squares mean estimate for each formulation needs to be used to form the difference, together with the standard error. However, in lack of exact formula i am seeking the help of experts out there.


Yes, you are on the right track!
Let's call Xt the LSM of the test, and Xr the LSM of the reference (log-scale). sigma-w is the within- (or intra-) subject standard deviation (sqrt(MSE) from your ANOVA), t(1-alpha,n1+n2-2) is the 0.95 quantile of the central t-distribution with n1+n2-2 degrees of freedom. Then the upper/lower 90% confidence limits (log scale) are given by
Xt - Xr ± t(1-alpha,n1+n2-2) × sqrt(MSE) × sqrt[ 1/2 × (1/n1 + (1/n2) ]

Just a reminder: don’t try evaluations in M$-Excel; the t-quantile (TINV) is implemented in a rather queer way: in order to get the correct value for t(1-alpha,n1+n2-2), you would have to use TINV(2 × alpha, n1+n2-2)!
See
http://www.practicalstats.com/xlsstats/excelstats.html
http://www.mis.coventry.ac.uk/~nhunt/pottel.pdf

I would heartly recommend two textbooks:
  1. S-C Chow and J-p Liu
    Design and Analysis of Bioavailability and Bioequivalence Studies
    Marcel Dekker, New York, 2nd Ed. (2000)
  2. D Hauschke, VW Steinijans and I Pigeot
    Bioequivalence Studies in Drug Development
    Wiley, Chichester (2007)

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
drshiv
★    

India,
2007-03-06 19:49
(6231 d 22:55 ago)

@ Helmut
Posting: # 561
Views: 28,839
 

 CI for Transformed data Unbalanced study

Dear USFDA_EMEA,

For unbalanced designs the solution provided by HS is a good. Other method is to use GLM procedure in SAS (if you are using SAS) to calculate MSE before calculation of CI. Then you can use the formula givn by HS to calculate CI, maybe using Excel (off course after validation of the formula).

Dr. Shiv


Edit: Full quote removed (see this post)! [Helmut]
usfda_emea
●    

2007-03-07 07:21
(6231 d 11:23 ago)

@ Helmut
Posting: # 564
Views: 28,621
 

 CI for Transformed data Unbalanced study

Dear HS,

❝ I guess that’s not your real name :-D


:yes:

❝ Let’s call Xt the LSM of the test, and Xr the LSM of the reference (log-scale).


Can you help me with the formula for LSM (I wish to do a manual calculation, for verification purposes) and a brief explaination about why this method is used for unbalanced study?

Thanks a million for the other formula and the references. :-)

Truly,
USFDA_EMEA
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2007-03-07 13:04
(6231 d 05:40 ago)

@ usfda_emea
Posting: # 565
Views: 29,073
 

 CI for Transformed data Unbalanced study

Dear USFDA_EMEA!

❝ ❝ Let's call Xt the LSM of the test, and Xr the LSM of the reference (log-scale).


❝ Can you help me with the formula for LSM (I wish to do a manual calculation, for verification purposes) and a brief explaination about why this method is used for unbalanced study?


The calculation is not different from a balanced study.

I’ll give you an example for the reference:
Calculate the (arithmetic) mean of log-transformed values in sequence 1 (if seq 1 = RT from period 1) as
Xr1 = sum ( Xr1 … n1 ) / n1

Calculate the (arithmetic) mean of log-transformed values in sequence 2 (if seq 2 = TR from period 2) as
Xr2 = sum ( Xr1 … n2 ) / n2

LSM for the reference is the arithmetic mean of Xr1 and Xr2
Xr = ( Xr1 + Xr2 ) / 2 )


The difference Xt – Xr is also called MLE (the Maximum Likelihood Estimator) of the true difference Mu(t) – Mu(r), which is slightly biased according to the degree of ‘unbalancedness’.
A better estimate of the true difference is MVUE (the Minimum Variance Unbiased Estimator), however, its calculation is a little bit tricky (involves the Γ-distribution).
For details see Chow & Liu’s book.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
usfda_emea
●    

2007-03-07 13:34
(6231 d 05:10 ago)

@ Helmut
Posting: # 566
Views: 28,687
 

 CI for Transformed data Unbalanced study

Dear HS,
:ok:
Ohlbe
★★★

France,
2014-07-30 17:04
(3529 d 02:40 ago)

@ Helmut
Posting: # 13319
Views: 22,711
 

 CI for Transformed data Unbalanced study

Dear Helmut,

❝ ❝ Can you help me with the formula for LSM (I wish to do a manual calculation, for verification purposes) and a brief explaination about why this method is used for unbalanced study?


❝ The calculation is not different from a balanced study.


❝ I’ll give you an example for the reference [...]



How do you calculate the LSM in a full replicate design study (RTRT and TRTR, with some subjects missing here and there) ?

Regards
Ohlbe
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-07-31 04:54
(3528 d 14:50 ago)

@ Ohlbe
Posting: # 13322
Views: 22,811
 

 LSM limbo

Dear Ohlbe,

❝ How do you calculate the LSM in a full replicate design study (RTRT and TRTR, with some subjects missing here and there) ?


Manually? Never done that. :-D If you are thinking about RSABE (in their FDA and EMA flavors), that’s a strange cup of tea. To quote Patterson & Jones1:

[…] simulation findings support that if a subject is missing both reference ob­ser­vations or the test observation, no bias is introduced.
If one R observation is missing, then MoM estimates of δ and σ²l are biased by period effects if they are present. It should be noted that period effects are known to occur in such designs.
If SABE is applied, subjects with one missing R observation should be eliminated from a MoM analysis until an alternative statistical procedure is available. This is un­precedented in our expe­ri­ence in a regulated bioequivalence setting. Traditionally, one does not exclude data unless there is a scientifically or clinically valid reason to do so. However, with the current draft guidance from FDA for progesterone bio­equi­valence, this appears to be the immediate approach to be applied for SABE. A better statistical alternative is restricted maximum likelihood estimation (to eliminate bias) with the use of the bootstrap for inference.


It’s a complete mess. Subjects with incomplete data are excluded from the estimation of σ²WR, but kept in the calculation of the upper 95% boundary (FDA) or excluded for the calculation of the 90% CI (EMA, all fixed effects. A mixed-effects model is not acceptable for EMA). To be honest, currently there are so many flaws in these methods that I would not bother to calculate anything by hand. Accept what you get from the FDA’s and EMA’s SAS-code (or alternative software) and set this question aside until somebody finds a method to deal with the more important question: How to prevent inflation1,2 of the patient’s risk (according to my latest simulations: EMA up to 9.6% and FDA up to 22.5%). This is a nightmare.


  1. Patterson SD and B Jones
    Viewpoint: observations on scaled average bioequivalence
    Pharm Stat 11(1), 1–7 (2011)
    DOI: 10.1002/pst.498
  2. Wonnemann M, Frömke C, and A Koch
    Inflation of the Type I Error: Investigations on Regulatory Recommendations for Bioequivalence of Highly Variable Drugs
    Pharm Res 31 (preprint published 18 July 2014)
    DOI: 10.1007/s11095-014-1450-z

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Ohlbe
★★★

France,
2014-07-31 11:47
(3528 d 07:57 ago)

@ Helmut
Posting: # 13323
Views: 22,477
 

 LSM limbo

Dear Helmut,

Manually? Never done that. :-D


❝ It’s a complete mess. Subjects with incomplete data are excluded from the estimation of σ²WR, but kept in the calculation of the upper 95% boundary (FDA) or excluded for the calculation of the 90% CI (EMA, all fixed effects. A mixed-effects model is not acceptable for EMA).


I would not think for even a split second of trying to recalculate manually the 90 % CI or σ²WR... But I was hoping there would be a simple way to at least recalculate LSMs (or to understand how they are calculated).

Thanks anyway !

Regards
Ohlbe
VStus
★    

Poland,
2017-03-01 15:07
(2584 d 03:37 ago)

@ Helmut
Posting: # 17116
Views: 17,721
 

 90% CI limbo

Dear Helmut,
Dear all,

I have limited data about partial replicate trial (2x3x3) with RSABE, which I want to use to assess the feasibility of same study under EMA regulation (assuming that EU has the same Reference product as US).

Reported parameters:
Number of subjects (observations per treatment arm)
Swr estimated using PROC GLM
Point Estimate (%) from RSABE-approach
95% CI upper confidence bound

Is it feasible to try calculation of 90% CIs?

Can I apply the formula below?

90% CI = log(Point_Estimate) +/- Swr * qt(0.05, df))*100,

where:
df=2*n-3 for partial replicate (2x3x3);
df=3*n-4 for full replicate (2x2x4);

Thank you very much in advance!

Regards, VStus
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-03-03 14:34
(2582 d 04:10 ago)

@ VStus
Posting: # 17126
Views: 17,752
 

 RSABE ⇒ ABEL

Hi VStus,

❝ I have limited data about partial replicate trial (2x3x3) with RSABE, which I want to use to assess the feasibility of same study under EMA regulation (assuming that EU has the same Reference product as US).

❝ Reported parameters:

❝ Number of subjects (observations per treatment arm)

❝ Swr estimated using PROC GLM

❝ Point Estimate (%) from RSABE-approach

❝ […]


❝ Is it feasible to try calculation of 90% CIs?


Yes.

❝ Can I apply the formula below?

❝ 90% CI = log(Point_Estimate) +/- Swr * qt(0.05, df))*100,


No (see this post). Let’s compare the EMA’s Q&A dataset II (n = 24; n1 = n2 = n3 = 8), evaluated by the FDA’s RSABE (PE = 1.022644, swR = 0.11397298) with results of the EMA’s methods: PE 102.26%; 90% CI: Methods A/B 97.32–107.46%, Method C 97.05–107.76%.

n   <- c(8, 8, 8)
pe  <- 1.022644
swR <- 0.11397298
CI  <- exp(log(pe) + c(-1, +1)*qt(1-0.05, 2*sum(n)-3)*sqrt(1/6*swR^2*sum(1/n)))
names(CI) <- c("lower", "upper")
round(100*CI, 2)
# lower  upper
# 97.49 107.28

Alternatively:

library(PowerTOST)
round(100*CI.BE(pe=pe, CV=se2CV(swR), n=n, design="2x3x3"), 2)
# lower  upper
# 97.49 107.28


Pretty close but in this case (swR <0.294) the FDA requires a mixed-effects model for ABE, where s²wR = 0.013246498 (PE 102.26%, 90% CI 97.05–107.76%). Hence, we have to take that into account.

library(PowerTOST)
n    <- c(8, 8, 8)
pe   <- 1.022644
s2wR <- 0.013246498
round(100*CI.BE(pe=pe, CV=mse2CV(s2wR), n=n, design="2x3x3", robust=TRUE), 2)
# lower  upper
# 97.32 107.46

Or the hard way:

CI <- exp(log(pe) + c(-1, +1)*qt(1-0.05, sum(n)-3)*sqrt(1/6*s2wR*sum(1/n)))
names(CI) <- c("lower", "upper")
round(100*CI, 2)
# lower  upper
# 97.32 107.46

Bingo!

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
VStus
★    

Poland,
2017-03-03 16:31
(2582 d 02:13 ago)

@ Helmut
Posting: # 17127
Views: 17,476
 

 RSABE ⇒ ABEL

Dear Helmut,

Thank you very much! I've missed recent post of Detlew when using search function... :blind:

PS: I like this update
n <- c(8,8,8)

Regards, VStus
d_labes
★★★

Berlin, Germany,
2017-03-04 15:49
(2581 d 02:55 ago)

@ VStus
Posting: # 17132
Views: 17,562
 

 s2wR != mse, FDA != EMA

Dear Helmut, dear VStus,

I assume that the s2wR you have (or swR) is the intra-subject variance estimated with R data.
If this is the case the elaborate answer by Helmut is only valid if s2wR is equal to the MSE from the evaluation of all data, which is necessary for calculation of the point estimate and 90% CI of T versus R.

But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.

Moreover the FDA evaluation uses intra-subject constrasts to evaluate s2wR an µT-µR including the 90%CI whereas the EMA recommended method is bases on common ANOVA (aka GLM in SAS speak. Both evaluation methods give different results. Usually slight differences, but ...

Thus my answer to the question "Is it feasible to try calculation of 90% CIs?" for the EMA based on the results of a FDA evaluation is a clear No.

What you get by Helmuts calculations is only an approximate solution which may be good enough but may also terrible fail.

Regards,

Detlew
ElMaestro
★★★

Denmark,
2017-03-04 20:13
(2580 d 22:31 ago)

@ d_labes
Posting: # 17133
Views: 17,960
 

 s2wR != mse, FDA != EMA

I like that post, d_labes :-),


❝ But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.


...and accordingly when the optimizer fails, plugging in the GLM residual (expressed as MSE) as a new starter value for s2wR (and even S2wT where applicable) may be a very decent last-ditch attempt.

❝ Moreover the FDA evaluation uses intra-subject constrasts to evaluate s2wR


What does this mean? Don't you usually get it from the REML covarance matrix?

I think EMA's approach can be condensed into intra-Subject contrasts to derive S2wR from completers.

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2017-03-05 13:18
(2580 d 05:26 ago)

@ ElMaestro
Posting: # 17135
Views: 17,772
 

 s2wR from ISC in FDA approach

Dear ElMaestro!

❝ ...

❝ ❝ Moreover the FDA evaluation uses intra-subject constrasts to evaluate s2wR

❝ What does this mean? Don't you usually get it from the REML covarance matrix?


If you have a look at the SAS code in the progesterone guidance you will discover that s2wR is not obtained from the REML covariance matrix but rather from an ISC evaluation. Means you calculate R-R within the subjects and use that difference in an ANOVA with the sequence as the soley effect.
The REML estimates don't play a role, since a mixed model only comes into the play if CVwR is <=30% and conventional ABE has to be used. Here you may get s2wR from the REML covariance matrix but it is not used in ABE.
That's one of the curiosities of the FDA approach, among others.

❝ I think EMA's approach can be condensed into intra-Subject contrasts to derive S2wR from completers.


Here you err. The EMA approach calls for an evaluation via lm(), GLM or comparable using the R(eference) data only with the effects period and subject (sequence may be also included but doesn't change the estimate of s2wR). This gives some different results to an estimation via ISC as described above. Try it.

BTW: Have a look into Implementation_scaledABE_sims.pdf in the doc subdirectory of R package PowerTOST. There you will find a cookbook manner description of the computations involved in the EMA or FDA approaches for scaled ABE.

Regards,

Detlew
ElMaestro
★★★

Denmark,
2017-03-05 14:50
(2580 d 03:54 ago)

@ d_labes
Posting: # 17136
Views: 17,901
 

 s2wR from ISC in FDA approach

Thanks d_labes,


❝ If you have a look at the SAS code in the progesterone guidance you will discover that s2wR is not obtained from the REML covariance matrix but rather from an ISC evaluation. Means you calculate R-R within the subjects and use that difference in an ANOVA with the sequence as the soley effect.

❝ The REML estimates dont play a role, since a mixed model only comes into the play if CVwR is <=30% and conventional ABE has to be used. Here you may get s2wR from the REML covariance matrix but it is not used in ABE.

❝ That's one of the curiosities of the FDA approach, among others.


Yes that is kind of strange.

❝ ❝ I think EMA's approach can be condensed into intra-Subject contrasts to derive S2wR from completers.


❝ Here you err. The EMA approach calls for an evaluation via lm(), GLM or comparable using the R(eference) data only with the effects period and subject (sequence may be also included but doesnt change the estimate of s2wR). This gives some different results to an estimation via ISC as described above. Try it.


I beg to differ. Chow and Liu showed how the problem expressed as a linear model, can be solved with equations all based on contrast of T and R, when we talk 222BE; this gave them the desired sw. Meaning you can get the quanitity you want by equations or by solving the linear model. That is why some regulators (none mentioned, none forgotten) don't even need to see an ANOVA.

This situation being a linear model which has an analytical solution we can generalise it further and make equations where things are condensed into equations of a similar nature for your ref-replicated design. I don't think it would be very difficult, actually, but I am not convinced that I myself could do it without further ado. The key here be that we rely on (start with) a linear model in which all the residual df's can be said to derive from the replication itself.

Pass or fail!
ElMaestro
VStus
★    

Poland,
2017-03-04 22:41
(2580 d 20:03 ago)

@ d_labes
Posting: # 17134
Views: 17,495
 

 s2wR != mse, FDA != EMA

Dear Detlew,

❝ But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.


And s2wR != s2wT in most cases. Maybe that's the reason why 2x3x3 design is still so popular. It's not possible to assess s2wT...

❝ What you get by Helmuts calculations is only an approximate solution which may be good enough but may also terrible fail.


These calculations are intended only to get an than approximate assumption. Like a pilot study.

Regards, VStus
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-03-06 14:40
(2579 d 04:04 ago)

@ VStus
Posting: # 17138
Views: 17,504
 

 s2wR != mse, FDA != EMA

Hi VStus,

❝ ❝ But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.

❝ And s2wR != s2wT in most cases. Maybe that's the reason why 2x3x3 design is still so popular. It's not possible to assess s2wT...


No agency asks for s²wT in the context of RSABE or ABEL (the FDA’s RSABE for NTIDs is another story but requires a 4-period full replicate anyhow). Hence, why should one report it? :-D
I never understood why one would opt for the partial replicate. Does this equation hold?

Partial replicate = lack of statistical knowledge + “we want only three periods” + given in the FDA’s guidance / the EMA’ Q&A-document (as an example)?


IMHO, at least in pilot studies full replicates are preferable (see there). I once struggled with a Romanian CRO. The sponsor decided to perform a 4-period full replicate in order to possibly end up with a smaller sample size of the pivotal study if CVwT < CVwR. The CRO refused to calculate CVwT (“… because not in accordance with the SAS-code given in the Q&A.”)… Oh boy! Only after the sponsor threatened to go with another CRO, they gave in.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
69 visitors (0 registered, 69 guests [including 4 identified bots]).
Forum time: 18:44 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5