Bioequivalence and Bioavailability Forum

usfda_emea
●

2007-03-06 15:39
(6589 d 09:15 ago)

Posting: # 559
Views: 34,706

Confidence Interval for Transformed data Unbalanced study [General Statistics]

Dear,

I am looking for help on estimation of 90% confidence interval for log transformed data of AUC (0-inf) for a bioequivalence study. This study involved two-way crossover, two-period, two-sequence design with a total of 33 subjects completing the study. Each sequence had Period I TR=16 and RT=17, and in Period II TR=17 and RT=16; thus, total of 33 completed the study.

In order to estimate the 90% confidence interval, can some one help me with the formula and a worked out example for the ease of understanding. OR Guide me to a site where this can be found.

I believe least-squares mean estimate for each formulation needs to be used to form the difference, together with the standard error. However, in lack of exact formula i am seeking the help of experts out there.

Thanks in advance for the help.
USFDA_EMEA

Edit: Category changed. [Helmut]

Helmut
★★★

Vienna, Austria,
2007-03-06 17:46
(6589 d 07:07 ago)

@ usfda_emea
Posting: # 560
Views: 31,073

CI for Transformed data Unbalanced study

Post reply

Dear USFDA_EMEA!

I guess that’s not your real name :-D

❝ I am looking for help on estimation of 90% confidence interval for log transformed data of AUC (0-inf) for a bioequivalence study. This study involved two-way crossover, two-period, two-sequence design with a total of 33 subjects completing the study. Each sequence had Period I TR=16 and RT=17, and in Period II TR=17 and RT=16; thus, total of 33 completed the study.

OK, the number of subjects in each sequence is n₁ and n₂, respectively, where n = n₁ + n₂.
In your case:

n₁ = 17

n₂ = 16

n  = 33

The assignment of sequence 1 to RT follows the literature, but is only a convention (if you want you may call TR sequence 1 as long as you keep this convention through all calculations…)

❝ In order to estimate the 90% confidence interval, can some one help me with the formula and a worked out example for the ease of understanding. OR Guide me to a site where this can be found.

Let’s see. Which formula are you using right now?
If you have something like 1/n in it, for unbalanced data you have to replace it by 1/n1 + 1/n2.

❝ I believe least-squares mean estimate for each formulation needs to be used to form the difference, together with the standard error. However, in lack of exact formula i am seeking the help of experts out there.

Yes, you are on the right track!
Let's call Xt the LSM of the test, and Xr the LSM of the reference (log-scale). sigma-w is the within- (or intra-) subject standard deviation (sqrt(MSE) from your ANOVA), t(1-alpha,n1+n2-2) is the 0.95 quantile of the central t-distribution with n1+n2-2 degrees of freedom. Then the upper/lower 90% confidence limits (log scale) are given by


Xt - Xr ± t(1-alpha,n1+n2-2) × sqrt(MSE) × sqrt[ 1/2 × (1/n1 + (1/n2) ]

Just a reminder: don’t try evaluations in M$-Excel; the t-quantile (TINV) is implemented in a rather queer way: in order to get the correct value for t(1-alpha,n1+n2-2), you would have to use TINV(2 × alpha, n1+n2-2)!
See
http://www.practicalstats.com/xlsstats/excelstats.html
http://www.mis.coventry.ac.uk/~nhunt/pottel.pdf

I would heartly recommend two textbooks:

S-C Chow and J-p Liu
Design and Analysis of Bioavailability and Bioequivalence Studies
Marcel Dekker, New York, 2^nd Ed. (2000)
D Hauschke, VW Steinijans and I Pigeot
Bioequivalence Studies in Drug Development
Wiley, Chichester (2007)

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

drshiv ★ India, 2007-03-06 19:49 (6589 d 05:05 ago) @ Helmut Posting: # 561 Views: 30,675	CI for Transformed data Unbalanced study Post reply
	Dear USFDA_EMEA, For unbalanced designs the solution provided by HS is a good. Other method is to use GLM procedure in SAS (if you are using SAS) to calculate MSE before calculation of CI. Then you can use the formula givn by HS to calculate CI, maybe using Excel (off course after validation of the formula). Dr. Shiv Edit: Full quote removed (see this post)! [Helmut]

usfda_emea
●

2007-03-07 07:21
(6588 d 17:33 ago)

@ Helmut
Posting: # 564
Views: 30,457

CI for Transformed data Unbalanced study

Post reply

Dear HS,

❝ I guess that’s not your real name :-D

❝ Let’s call Xt the LSM of the test, and Xr the LSM of the reference (log-scale).

Can you help me with the formula for LSM (I wish to do a manual calculation, for verification purposes) and a brief explaination about why this method is used for unbalanced study?

Thanks a million for the other formula and the references. :-)

Truly,
USFDA_EMEA

Helmut
★★★

Vienna, Austria,
2007-03-07 13:04
(6588 d 11:50 ago)

@ usfda_emea
Posting: # 565
Views: 30,981

CI for Transformed data Unbalanced study

Post reply

Dear USFDA_EMEA!

❝ ❝ Let's call Xt the LSM of the test, and Xr the LSM of the reference (log-scale).

❝

❝ Can you help me with the formula for LSM (I wish to do a manual calculation, for verification purposes) and a brief explaination about why this method is used for unbalanced study?

The calculation is not different from a balanced study.

I’ll give you an example for the reference:
Calculate the (arithmetic) mean of log-transformed values in sequence 1 (if seq 1 = RT from period 1) as
Xr1 = sum ( Xr1 … n1 ) / n1

Calculate the (arithmetic) mean of log-transformed values in sequence 2 (if seq 2 = TR from period 2) as
Xr2 = sum ( Xr1 … n2 ) / n2

LSM for the reference is the arithmetic mean of Xr1 and

Xr2

Xr = ( Xr1 + Xr2 ) / 2 )

The difference Xt – Xr is also called MLE (the Maximum Likelihood Estimator) of the true difference Mu(t) – Mu(r), which is slightly biased according to the degree of ‘unbalancedness’.
A better estimate of the true difference is MVUE (the Minimum Variance Unbiased Estimator), however, its calculation is a little bit tricky (involves the Γ-distribution).
For details see Chow & Liu’s book.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

usfda_emea ● 2007-03-07 13:34 (6588 d 11:20 ago) @ Helmut Posting: # 566 Views: 30,531	CI for Transformed data Unbalanced study Post reply
	Dear HS,

Ohlbe
★★★

France,
2014-07-30 17:04
(3886 d 08:50 ago)

@ Helmut
Posting: # 13319
Views: 24,574

CI for Transformed data Unbalanced study

Post reply

Dear Helmut,

❝ ❝ Can you help me with the formula for LSM (I wish to do a manual calculation, for verification purposes) and a brief explaination about why this method is used for unbalanced study?

❝

❝ The calculation is not different from a balanced study.

❝

❝ I’ll give you an example for the reference [...]

How do you calculate the LSM in a full replicate design study (RTRT and TRTR, with some subjects missing here and there) ?

—
Regards
Ohlbe

Helmut
★★★

Vienna, Austria,
2014-07-31 04:54
(3885 d 21:00 ago)

@ Ohlbe
Posting: # 13322
Views: 24,666

LSM limbo

Post reply

Dear Ohlbe,

❝ How do you calculate the LSM in a full replicate design study (RTRT and TRTR, with some subjects missing here and there) ?

Manually? Never done that. :-D

If you are thinking about RSABE (in their FDA and EMA flavors), that’s a strange cup of tea. To quote Patterson & Jones¹:

[…] simulation findings support that if a subject is missing both reference observations or the test observation, no bias is introduced.
If one R observation is missing, then MoM estimates of δ and σ²_l are biased by period effects if they are present. It should be noted that period effects are known to occur in such designs.
If SABE is applied, subjects with one missing R observation should be eliminated from a MoM analysis until an alternative statistical procedure is available. This is unprecedented in our experience in a regulated bioequivalence setting. Traditionally, one does not exclude data unless there is a scientifically or clinically valid reason to do so. However, with the current draft guidance from FDA for progesterone bioequivalence, this appears to be the immediate approach to be applied for SABE. A better statistical alternative is restricted maximum likelihood estimation (to eliminate bias) with the use of the bootstrap for inference.

It’s a complete mess. Subjects with incomplete data are excluded from the estimation of σ²_WR, but kept in the calculation of the upper 95% boundary (FDA) or excluded for the calculation of the 90% CI (EMA, all fixed effects. A mixed-effects model is not acceptable for EMA). To be honest, currently there are so many flaws in these methods that I would not bother to calculate anything by hand. Accept what you get from the FDA’s and EMA’s SAS-code (or alternative software) and set this question aside until somebody finds a method to deal with the more important question: How to prevent inflation^1,2 of the patient’s risk (according to my latest simulations: EMA up to 9.6% and FDA up to 22.5%). This is a nightmare.

Patterson SD and B Jones
Viewpoint: observations on scaled average bioequivalence
Pharm Stat 11(1), 1–7 (2011)
DOI: 10.1002/pst.498
Wonnemann M, Frömke C, and A Koch
Inflation of the Type I Error: Investigations on Regulatory Recommendations for Bioequivalence of Highly Variable Drugs
Pharm Res 31 (preprint published 18 July 2014)
DOI: 10.1007/s11095-014-1450-z

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Ohlbe
★★★

France,
2014-07-31 11:47
(3885 d 14:06 ago)

@ Helmut
Posting: # 13323
Views: 24,323

LSM limbo

Post reply

Dear Helmut,

❝ Manually? Never done that. :-D

❝ It’s a complete mess. Subjects with incomplete data are excluded from the estimation of σ²_WR, but kept in the calculation of the upper 95% boundary (FDA) or excluded for the calculation of the 90% CI (EMA, all fixed effects. A mixed-effects model is not acceptable for EMA).

I would not think for even a split second of trying to recalculate manually the 90 % CI or σ²_WR... But I was hoping there would be a simple way to at least recalculate LSMs (or to understand how they are calculated).

Thanks anyway !

—
Regards
Ohlbe

VStus
★

Poland,
2017-03-01 15:07
(2941 d 09:47 ago)

@ Helmut
Posting: # 17116
Views: 19,578

90% CI limbo

Post reply

Dear Helmut,
Dear all,

I have limited data about partial replicate trial (2x3x3) with RSABE, which I want to use to assess the feasibility of same study under EMA regulation (assuming that EU has the same Reference product as US).

Reported parameters:
Number of subjects (observations per treatment arm)
Swr estimated using PROC GLM
Point Estimate (%) from RSABE-approach
95% CI upper confidence bound

Is it feasible to try calculation of 90% CIs?

Can I apply the formula below?

90% CI = log(Point_Estimate) +/- Swr * qt(0.05, df))*100,

where:
df=2*n-3 for partial replicate (2x3x3);
df=3*n-4 for full replicate (2x2x4);

Thank you very much in advance!

Regards, VStus

Helmut
★★★

Vienna, Austria,
2017-03-03 14:34
(2939 d 10:20 ago)

@ VStus
Posting: # 17126
Views: 19,587

RSABE ⇒ ABEL

Post reply

Hi VStus,

❝ I have limited data about partial replicate trial (2x3x3) with RSABE, which I want to use to assess the feasibility of same study under EMA regulation (assuming that EU has the same Reference product as US).

❝ Reported parameters:

❝ Number of subjects (observations per treatment arm)

❝ Swr estimated using PROC GLM

❝ Point Estimate (%) from RSABE-approach

❝ […]

❝ Is it feasible to try calculation of 90% CIs?

Yes.

❝ Can I apply the formula below?

❝ 90% CI = log(Point_Estimate) +/- Swr * qt(0.05, df))*100,

No (see this post). Let’s compare the EMA’s Q&A dataset II (n = 24; n₁ = n₂ = n₃ = 8), evaluated by the FDA’s RSABE (PE = 1.022644, s_wR = 0.11397298) with results of the EMA’s methods: PE 102.26%; 90% CI: Methods A/B 97.32–107.46%, Method C 97.05–107.76%.

n <- c(8, 8, 8) pe <- 1.022644 swR <- 0.11397298 CI <- exp(log(pe) + c(-1, +1)*qt(1-0.05, 2*sum(n)-3)*sqrt(1/6*swR^2*sum(1/n))) names(CI) <- c("lower", "upper") round(100*CI, 2) # lower upper # 97.49 107.28

Alternatively:

library(PowerTOST) round(100*CI.BE(pe=pe, CV=se2CV(swR), n=n, design="2x3x3"), 2) # lower upper # 97.49 107.28

Pretty close but in this case (s_wR <0.294) the FDA requires a mixed-effects model for ABE, where s²_wR = 0.013246498 (PE 102.26%, 90% CI 97.05–107.76%). Hence, we have to take that into account.

library(PowerTOST) n <- c(8, 8, 8) pe <- 1.022644 s2wR <- 0.013246498 round(100*CI.BE(pe=pe, CV=mse2CV(s2wR), n=n, design="2x3x3", robust=TRUE), 2) # lower upper # 97.32 107.46

Or the hard way:

CI <- exp(log(pe) + c(-1, +1)*qt(1-0.05, sum(n)-3)*sqrt(1/6*s2wR*sum(1/n))) names(CI) <- c("lower", "upper") round(100*CI, 2) # lower upper # 97.32 107.46

Bingo!

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

VStus ★ Poland, 2017-03-03 16:31 (2939 d 08:22 ago) @ Helmut Posting: # 17127 Views: 19,328	RSABE ⇒ ABEL Post reply
	Dear Helmut, Thank you very much! I've missed recent post of Detlew when using search function... PS: I like this update `n <- c(8,8,8)` Regards, VStus

d_labes
★★★

Berlin, Germany,
2017-03-04 15:49
(2938 d 09:05 ago)

@ VStus
Posting: # 17132
Views: 19,409

s2wR != mse, FDA != EMA

Post reply

Dear Helmut, dear VStus,

I assume that the s2wR you have (or swR) is the intra-subject variance estimated with R data.
If this is the case the elaborate answer by Helmut is only valid if s2wR is equal to the MSE from the evaluation of all data, which is necessary for calculation of the point estimate and 90% CI of T versus R.

But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.

Moreover the FDA evaluation uses intra-subject constrasts to evaluate s2wR an µT-µR including the 90%CI whereas the EMA recommended method is bases on common ANOVA (aka GLM in SAS speak. Both evaluation methods give different results. Usually slight differences, but ...

Thus my answer to the question "Is it feasible to try calculation of 90% CIs?" for the EMA based on the results of a FDA evaluation is a clear No.

What you get by Helmuts calculations is only an approximate solution which may be good enough but may also terrible fail.

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2017-03-04 20:13
(2938 d 04:40 ago)

@ d_labes
Posting: # 17133
Views: 19,802

s2wR != mse, FDA != EMA

Post reply

I like that post, d_labes :-)

❝ But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.

...and accordingly when the optimizer fails, plugging in the GLM residual (expressed as MSE) as a new starter value for s2wR (and even S2wT where applicable) may be a very decent last-ditch attempt.

❝ Moreover the FDA evaluation uses intra-subject constrasts to evaluate s2wR

What does this mean? Don't you usually get it from the REML covarance matrix?

I think EMA's approach can be condensed into intra-Subject contrasts to derive S2wR from completers.

—
Pass or fail!
ElMaestro

d_labes
★★★

Berlin, Germany,
2017-03-05 13:18
(2937 d 11:35 ago)

@ ElMaestro
Posting: # 17135
Views: 19,731

s2wR from ISC in FDA approach

Post reply

Dear ElMaestro!

❝ ...

❝ ❝ Moreover the FDA evaluation uses intra-subject constrasts to evaluate s2wR

❝ What does this mean? Don't you usually get it from the REML covarance matrix?

If you have a look at the SAS code in the progesterone guidance you will discover that s2wR is not obtained from the REML covariance matrix but rather from an ISC evaluation. Means you calculate R-R within the subjects and use that difference in an ANOVA with the sequence as the soley effect.
The REML estimates don't play a role, since a mixed model only comes into the play if CVwR is <=30% and conventional ABE has to be used. Here you may get s2wR from the REML covariance matrix but it is not used in ABE.
That's one of the curiosities of the FDA approach, among others.

❝ I think EMA's approach can be condensed into intra-Subject contrasts to derive S2wR from completers.

Here you err. The EMA approach calls for an evaluation via lm(), GLM or comparable using the R(eference) data only with the effects period and subject (sequence may be also included but doesn't change the estimate of s2wR). This gives some different results to an estimation via ISC as described above. Try it.

BTW: Have a look into Implementation_scaledABE_sims.pdf in the doc subdirectory of R package PowerTOST. There you will find a cookbook manner description of the computations involved in the EMA or FDA approaches for scaled ABE.

—
Regards,

Detlew

ElMaestro
★★★

Denmark,
2017-03-05 14:50
(2937 d 10:03 ago)

@ d_labes
Posting: # 17136
Views: 19,788

s2wR from ISC in FDA approach

Post reply

Thanks d_labes,

❝ If you have a look at the SAS code in the progesterone guidance you will discover that s2wR is not obtained from the REML covariance matrix but rather from an ISC evaluation. Means you calculate R-R within the subjects and use that difference in an ANOVA with the sequence as the soley effect.

❝ The REML estimates dont play a role, since a mixed model only comes into the play if CVwR is <=30% and conventional ABE has to be used. Here you may get s2wR from the REML covariance matrix but it is not used in ABE.

❝ That's one of the curiosities of the FDA approach, among others.

Yes that is kind of strange.

❝ ❝ I think EMA's approach can be condensed into intra-Subject contrasts to derive S2wR from completers.

❝

❝ Here you err. The EMA approach calls for an evaluation via lm(), GLM or comparable using the R(eference) data only with the effects period and subject (sequence may be also included but doesnt change the estimate of s2wR). This gives some different results to an estimation via ISC as described above. Try it.

I beg to differ. Chow and Liu showed how the problem expressed as a linear model, can be solved with equations all based on contrast of T and R, when we talk 222BE; this gave them the desired sw. Meaning you can get the quanitity you want by equations or by solving the linear model. That is why some regulators (none mentioned, none forgotten) don't even need to see an ANOVA.

This situation being a linear model which has an analytical solution we can generalise it further and make equations where things are condensed into equations of a similar nature for your ref-replicated design. I don't think it would be very difficult, actually, but I am not convinced that I myself could do it without further ado. The key here be that we rely on (start with) a linear model in which all the residual df's can be said to derive from the replication itself.

—
Pass or fail!
ElMaestro

VStus
★

Poland,
2017-03-04 22:41
(2938 d 02:13 ago)

@ d_labes
Posting: # 17134
Views: 19,326

s2wR != mse, FDA != EMA

Post reply

Dear Detlew,

❝ But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.

And s2wR != s2wT in most cases. Maybe that's the reason why 2x3x3 design is still so popular. It's not possible to assess s2wT...

❝ What you get by Helmuts calculations is only an approximate solution which may be good enough but may also terrible fail.

These calculations are intended only to get an than approximate assumption. Like a pilot study.

Regards, VStus

Helmut
★★★

Vienna, Austria,
2017-03-06 14:40
(2936 d 10:13 ago)

@ VStus
Posting: # 17138
Views: 19,368

s2wR != mse, FDA != EMA

Post reply

Hi VStus,

❝ ❝ But s2wR = MSE is not necessarily true in all cases, although it is in many cases a reasonable assumption.

❝ And s2wR != s2wT in most cases. Maybe that's the reason why 2x3x3 design is still so popular. It's not possible to assess s2wT...

No agency asks for s²_wT in the context of RSABE or ABEL (the FDA’s RSABE for NTIDs is another story but requires a 4-period full replicate anyhow). Hence, why should one report it? :-D

I never understood why one would opt for the partial replicate. Does this equation hold?

Partial replicate = lack of statistical knowledge + “we want only three periods” + given in the FDA’s guidance / the EMA’ Q&A-document (as an example)?

IMHO, at least in pilot studies full replicates are preferable (see there). I once struggled with a Romanian CRO. The sponsor decided to perform a 4-period full replicate in order to possibly end up with a smaller sample size of the pivotal study if CV_wT < CV_wR. The CRO refused to calculate CV_wT (“… because not in accordance with the SAS-code given in the Q&A.”)… Oh boy! Only after the sponsor threatened to go with another CRO, they gave in.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Confidence Interval for Transformed data Unbalanced study [General Sta­tis­tics]

CI for Transformed data Unbalanced study

CI for Transformed data Unbalanced study

CI for Transformed data Unbalanced study

CI for Transformed data Unbalanced study

CI for Transformed data Unbalanced study

CI for Transformed data Unbalanced study

LSM limbo

LSM limbo

90% CI limbo

RSABE ⇒ ABEL

RSABE ⇒ ABEL

s2wR != mse, FDA != EMA

s2wR != mse, FDA != EMA

s2wR from ISC in FDA approach

s2wR from ISC in FDA approach

s2wR != mse, FDA != EMA

s2wR != mse, FDA != EMA

Confidence Interval for Transformed data Unbalanced study [General Statistics]