BE-proff
●    

2016-08-24 10:48
(2773 d 08:14 ago)

Posting: # 16566
Views: 12,680
 

 Why do we need pooled CV? [Power / Sample Size]

Hi All,

As my statistics background is rather poor (shame on me) I can't understand purpose of pooled CV in BE-studies.

Unless I am mistaken this parameters should be calculated when there are several studies with different CVs and we need "adequate" CV (correct? :confused:).

But what prevents us from using the highest CV for sample size calculation?
Any ambushes there? :-)


Edit: Category changed. [Helmut]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-08-24 14:07
(2773 d 04:55 ago)

@ BE-proff
Posting: # 16567
Views: 11,651
 

 Maximum CV might be misleading

Hi BE-proff,

❝ […] pooled CV in BE-studies […] should be calculated when there are several studies with different CVs and we need "adequate" CV (correct? :confused:).


Correct.

❝ But what prevents us from using the highest CV for sample size calculation?


Nothing.

❝ Any ambushes there? :-)


No obvious ones. The maximum CV might be pure chance and/or the result of a small sample size. The lower the sample size, the higher its uncertainty is. The CV is an estimate, not carved in stone. Hence, it makes sense to take this uncertainty into account and have a look at its upper confidence limit (I generally use an α of 20%).
Let’s consider 2×2 studies with CVs of 25%. Sample size were 12 (pilot study) and 24 (from the literature or a failed pivotal).

library(PowerTOST)
round(100*CVCL(CV=0.25, df=12-2, side="upper", alpha=0.2)[[2]], 2)
round(100*CVCL(CV=0.25, df=24-2, side="upper", alpha=0.2)[[2]], 2)

We get an upper CL of 32.11% for the small study and 29.19% for the large one. Q.E.D.

One step further: We get CVs of 25 and 15%. Sample sizes were 12 and 24 in one case and the other way ’round in the other.

CV      <- c(0.25, 0.15)
n       <- c(12, 24)
source  <- c("study 1", "study 2")
CVdata1 <- data.frame(CV, n, rep("2x2", 2), source)
CVdata2 <- data.frame(rev(CV), n, rep("2x2", 2), source)
names(CVdata2) <- names(CVdata1) <- c("CV", "n", "design", "source")
print(CVdata1, row.names=FALSE); print(CVdata2, row.names=FALSE)

   CV  n design  source
 0.25 12    2x2 study 1
 0.15 24    2x2 study 2
   CV  n design  source
 0.15 12    2x2 study 1
 0.25 24    2x2 study 2


Let’s get the pooled CVs and calculate their upper CLs:

print(CVpooled(CVdata1, alpha=0.2), verbose=TRUE)
print(CVpooled(CVdata2, alpha=0.2), verbose=TRUE)

Pooled CV = 0.1866 with 32 degrees of freedom
Upper 80% confidence limit of CV = 0.211

Pooled CV = 0.2232 with 32 degrees of freedom
Upper 80% confidence limit of CV = 0.2527


In the first case the pooled CV with 18.66% is closer to the 15% (from n=24), since the study is more influential than the 25% (from n=12). In the second case the order is reversed (22.32% is closer to 25% than to 15%).

Now back to your question whether it would make sense to use the highest CV. I would always use the upper CL. As shown above for CV 25% and n=24 this is 29.19%. If we have additional information from other studies (i.e., work with a pooled CV) we have more degrees of freedom which narrows the CI. In the second case the upper CL is 25.17% which is substantially lower than the 29.19% from the single study.

Hope that helps.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
BE-proff
●    

2016-08-26 12:41
(2771 d 06:22 ago)

@ Helmut
Posting: # 16579
Views: 11,304
 

 Maximum CV might be misleading

Hi Helmut,

Many thanks for your detailed explanation!
Pooled CV looks really interesting :)

One more question

Let's say I have results of 3 studies (imagined):
n=24, CV=0.16
n=36, CV=0.28
n=18, CV=0.14

How to write R-code in this case to calculate CV pooled?

Thank you.

Proff.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-08-26 13:34
(2771 d 05:28 ago)

@ BE-proff
Posting: # 16583
Views: 11,308
 

 Pooling – example

Hi BE-proff,

❝ Let's say I have results of 3 studies (imagined):

❝ n=24, CV=0.16

❝ n=36, CV=0.28

❝ n=18, CV=0.14


❝ How to write R-code in this case to calculate CV pooled?


Adapting my code from above:

library(PowerTOST)
CV     <- c(0.16, 0.28, 0.14)
n      <- c(24, 36, 18)
source <- c("study 1", "study 2", "study 3")
CVdata <- data.frame(CV, n, rep("2x2", length(source)), source)
names(CVdata) <- names(CVdata) <- c("CV", "n", "design", "source")
print(CVdata, row.names=FALSE); print(CVpooled(CVdata, alpha=0.2), verbose=T)

You should get:

   CV  n design  source
 0.16 24    2x2 study 1
 0.28 36    2x2 study 2
 0.14 18    2x2 study 3
Pooled CV = 0.221 with 72 degrees of freedom
Upper 80% confidence limit of CV = 0.2391


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
BE-proff
●    

2016-12-28 15:37
(2647 d 02:25 ago)

@ Helmut
Posting: # 16898
Views: 9,645
 

 Pooling – example

Hi Helmut,

Do you provide calculation of CV upper limit in study protocols?
If yes, how do you explain regulators that you use upper limit instead of CV?

Maybe literature references? ;-)
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-12-29 13:27
(2646 d 04:35 ago)

@ BE-proff
Posting: # 16903
Views: 9,586
 

 in my protocols…

BE-proff

❝ Do you provide calculation of CV upper limit in study protocols?

❝ If yes, how do you explain regulators that you use upper limit instead of CV?


I don’t mention it directly. ;-)
I write something like

Based on a conservative estimate of the CV of … obtained from … and assuming a T/R-ratio of …, … subjects are required to obtain at least … power.


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2016-08-24 15:03
(2773 d 03:59 ago)

@ BE-proff
Posting: # 16568
Views: 11,528
 

 Why do we need pooled CV?

Hi BE_Proff,

❝ As my statistics background is rather poor (shame on me) I can't understand purpose of pooled CV in BE-studies.


Yah it is tricky. You need a CV guess to plan a sample size in yur next trial. You may not know anything in advance, so best you can do (?) may be to look up CV's in the literature.

❝ But what prevents us from using the highest CV for sample size calculation?


Using the highest CV as you say is conservative, not necessarily a bad idea, and will work in your favour in terms of power.

For once I actually disagree with Helmut. At least, I think I do if I got his post right.

To me pooled CV is not always an obvious answer. It is one proposal, but I do not know if it is any better than other proposals.

Pooled variances work through sample sizes - somehow pooled CV's weight the different trials according to sample sizes and then work out an overall estimate from them.
I think this approach "academizes" (is that even a word?) the issue. While the calculation looks fancy with greek sigmas and all, I think it is every bit as useful -if not considerably more useful, even- to read info about the various trials you have access to and ask yourself which one or which ones you think are most solid scientifically. It could be based on CRO reputation, assay technique, bioanalytical A+P details, year, equipment, LLOQ, whatever you have access to. Add to that something which is so extremely important in BE but which by definition cannot be defined scientifically: Gut feeling. Never forget gut feeling. It is what makes you stand out positively from the crowd.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-08-24 15:53
(2773 d 03:09 ago)

@ ElMaestro
Posting: # 16569
Views: 11,530
 

 Yessir! Common sense!

Hi ElMaestro,

❝ For once I actually disagree with Helmut. At least, I think I do if I got his post right.


Only one greek letter in my post. ;-) I have written there:

❝ Doing the math is just the first step. Before you pool CV I would suggest to inspect whether the confidence intervals of the CVs overlap […]. If not, try to find out why (different CROs, populations, bioanalytical methods, …). Use common sense to decide which CVs are reliable enough to pool.


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
mittyri
★★  

Russia,
2016-08-24 17:13
(2773 d 01:49 ago)

@ Helmut
Posting: # 16570
Views: 11,341
 

 Common sense weighting before pooling

Hi Helmut and ElMaestro,

I assume it's time to include weight term (common sense weight) :-D

Kind regards,
Mittyri
DavidManteigas
★    

Portugal,
2016-08-25 13:09
(2772 d 05:53 ago)

@ ElMaestro
Posting: # 16575
Views: 11,267
 

 Gut feeling is the answer!

Hi all,

In my opinion, sample size calculation is 90% gut feeling and 10% math :-D

Sample size for BABE studies is always hard since there is no much information in the literature and the variability of the test product is always unknown. Most of the time, the information we have is from drug interaction or food interaction studies, which in principle will provide us with higher CVs then expected for BABE trials. Other times, you find one or two sources about the CV and rarely the results are completely opposite (ie one reporting a CV of 28% and another a CV of 5%). At least, this is my experience unless I'm not doing my research correctly :-D

I usually go with mittyri conservative approach because negative studies due to lack of power are a guarantee of client insatisfaction ;-) And the costs of a few additional subjects compared with the cost of failure are considerably low and it is always better to play safe, imo.

For therapeutic studies I usually provide more "sophisticated" calculations and sensitivity analysis. Most of the time, the calculations are "Money client have/Cost per patient". But the variability and cost per patient are so much higher than in BABE studies that sometimes adding more 10 ou 20 patients may be out of the budget.
Astea
★★  

Russia,
2016-08-26 22:40
(2770 d 20:22 ago)

@ DavidManteigas
Posting: # 16597
Views: 11,211
 

 The reasons not to pool

Dear all!

As I understand, pooling CV is a part of metha-analysis. In its turn meta-analysis conceals many suprises, Simpson's paradox as example. The question is how to make sure of homogenity of pooling data? What can be the reasons to throw out some outlier data, excluding Gut feeling? Can we pool data from studies with different dosage or studies with different formulations? How do the strength of the dosage affects variability?

"Being in minority, even a minority of one, did not make you mad"
mittyri
★★  

Russia,
2016-08-29 00:05
(2768 d 18:57 ago)

@ Astea
Posting: # 16602
Views: 11,110
 

 The reasons not to pool

Dear Astea,

your questions seems to me rhetorical ;-)

❝ As I understand, pooling CV is a part of metha-analysis. In its turn meta-analysis conceals many suprises, Simpson's paradox as example.


We are pooling not the data sets, but some statistical parameters (pooled CVs), so Simpson's paradox is another story

❝ The question is how to make sure of homogenity of pooling data? What can be the reasons to throw out some outlier data, excluding Gut feeling?


You cannot be sure till the moment you have the CSR in your hands. Even then, I wouldn't be sure taking into account the quality of data management in some CRO. So the gut feeling weighting is the best.
By the way have you heard about pilot studies in Russia? I'm not

❝ Can we pool data from studies with different dosage or studies with different formulations? How do the strength of the dosage affects variability?


I think it depends on many things (PK linearity, sensitivity of the method).
I suppose you have a huge data base (may be even the best in Russia ;-) ), so you can analyze some trends by yourself. In my experience the IntraSCV is usually higher for low strengths (but not for all cases)

Kind regards,
Mittyri
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-08-29 13:56
(2768 d 05:06 ago)

@ Astea
Posting: # 16606
Views: 11,302
 

 Know your drug/formulation!

Dear Astea et al.,

first of all I fully agree with Mittyri. Instead of “gut feeling” I prefer common sense. ;-)

    I try not to think with my gut.
If I’m serious about understanding the world,
thinking with anything besides my brain, as tempting as that might be,
is likely to get me into trouble.

Carl Sagan. The Demon-Haunted World: Science as a Candle in the Dark; Ch. 11: The Dragon in My Garage (1995)

Before we apply any statistical method we have to understand the data generating process. In our field the two most forgotten prerequisites are: Knowing the drug (PK/PD properties) and its formulation (biopharmaceutics). Having information about the clinical setting and bioanalytics is useful as well. Some examples (apart form the ones already given by Mittyri) where pooling is not a good idea:
  • Mixing studies in fasting and fed state. In the latter case the CV generally (but not always) is higher than in the former.
  • Whilst pooling CVs of IR-formulations (say tablets and capsules) generally is not problematic, I would be cautious with MR-formulations of different types (say pellets/capsules and monolithic tablets).
  • Highly metabolized drugs at a dosage near the limit of saturation. If the GMR is not very close to 1, the CV might be inflated.
  • Including studies with LC/MS-MS methods at its infancy (protein precipitation without stable-isotope internal standardization). In the early 2000s studies failed because they were planned based on CVs from earlier studies obtained with e.g., fluorescence detection. The matrix effect hit (CV went through the ceiling) and studies were underpowered.
Sample size estimation (as a part of designing the study) should be done in close collaboration of the statistician with the clinical expert(s). Nowadays “push-the-button” specialists are all too common. Obtaining a basic knowledge in PK/PD/biopharmaceutics is a piece of cake for statisticians. Much more difficult for physicians/pharmacists/… (plug in whatever you like) to understand statistics.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,638 registered users;
80 visitors (0 registered, 80 guests [including 5 identified bots]).
Forum time: 18:03 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5