khaoula
★    

Algeria,
2014-06-06 14:04

Posting: # 13037
Views: 14,317
 

 randomization [Study Per­for­mance]

Hi everybody , I have a question about impact of randomization in result of bioequivlence study:

we have design of bioequivalence study cross over design with 16 subject (randomised), but at the firt period the 15 suject didn't come, after we collected the results of 15 subjects and we analyzed Kinetica without protocol for the management of dropouts
we had abberants results: for Cmax CV = 0,03 % for hight variable drug, power: 50%, with subject effect and subject/sequence effect

if does not take into account the exclusion of subject N 15 have impact in the result of the study?
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-06-06 14:47

@ khaoula
Posting: # 13038
Views: 13,239
 

 Imbalanced cross-overs

Hi Khaoula,

» […] we collected the results of 15 subjects and we analyzed Kinetica without protocol for the management of dropouts

[image]We recently have submitted a paper to the AAPS J (which currently is under review). It seems that Kinetica is not able to correctly deal with imbalanced studies. I strongly suggest to use another soft­ware (we got correct results in R, Phoenix/WinNonlin, EquivTest/PK, and SAS).

» we had abberants results: for Cmax CV = 0,03 % for hight variable drug …

I don’t understand what you mean here. The limit for HVDs/HVDPs is 30% CV of the reference obtained in a replicate design. CVintra from a 2×2 serves only as a hint of a highly variable reference (since pooled from CVWR and CVWT).

» power: 50% …

A posteriori (aka post-hoc) power is irrelevant in BE. Stop calculating it.
Either the study passes, or not.

» with subject effect and subject/sequence effect

Subject effects are normal in a cross-over. It tells you that subjects differ – well, they should…

» […] the exclusion of subject N 15 have impact in the result of the study?

If evaluated by Kinetica, yes. :-(

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
khaoula
★    

Algeria,
2014-06-06 16:03

@ Helmut
Posting: # 13039
Views: 13,138
 

 Imbalanced cross-overs

Hi Helmut,

» I don’t understand what you mean here. The limit for HVDs/HVDPs is 30% CV of the reference obtained in a replicate design. CVintra from a 2×2 serves only as a hint of a highly variable reference (since pooled from CVWR and CVWT).

this is a real study that was done in the institute were I do my post graduate disertation, and the drug is omeprazole, that is a HVDP (in bibliography I read a lot of bioequivalence stydy were CV > 30%) I know that the cross over isn't adapted for this, but CV C max of the study Was very low!!!!!!!!
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-06-07 13:52

@ khaoula
Posting: # 13041
Views: 13,283
 

 CV 0.03%?

Hi Khaoula,

» […] the drug is omeprazole, that is a HVDP (in bibliography I read a lot of bioequivalence stydy were CV > 30%)

Agree.

» […] CV C max of the study Was very low!!!!!!!!

The 0.03% you stated above are practically impossible. Even if you had only monozygotic multiples in your study (violating the “independence” of IID) there would still be variability of the bioanalytical method as a component of the residual variance. Exceptionally good methods still have a CV of ~2–3%. The lowest CVintra of Cmax I ever have seen was ~7% (more than two orders of magnitude higher than yours). Please check your calculations:

\(CV\% = 100\sqrt{e^{MSE}-1}\), where MSE is obtained from ANOVA/GLM of ln-transformed data.

Avoid Kinetica.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
jag009
★★★

NJ,
2014-06-09 15:36

@ khaoula
Posting: # 13042
Views: 13,061
 

 Imbalanced cross-overs

Hi,

I seriously doubt that you ended up with a 0.03% CV for Cmax, particularly for Omeprazole. Can you post a printout of the stats from Kinetica for Cmax?

John
khaoula
★    

Algeria,
2014-06-09 22:07

@ jag009
Posting: # 13044
Views: 13,036
 

 Imbalanced cross-overs

Patient Sequence CmaxT CmaxR  LnCmaxT LnCmaxR
   1       RT     745    765   6.613   6.640
   2       RT     302    216   5.710   5.370
   3       TR     775    633   6.653   6.450
   4       TR     628    468   6.443   6.148
   5       TR     566    569   6.504   6.344
   6       TR     546    525   6.303   6.263
   7       RT     176.86 307   5.175   5.720
   8       RT     531    363   6.275   5.894
   9       RT     518    243   6.250   5.493
  10       RT     572    419   6.349   6.038
  11       TR     704    371   6.557   5.916
  12       TR     694    705   6.542   6.558
  13       TR    1400   1180   7.244   7.073
  14       RT     745    765   6.613   6.640
  15
  16       TR     339    232   5.826   5.447
         
    latin square of Cmax
           
SOURCE       D.F      SS         MS        F        p
Period         1  0.0264787  0.0264787   0.56186  0.4669    NS
Subject(Seq)  13  4.8517     0.373208    7.91934  0.0003379 ***
Formulation    1  0.211625   0.211625    4.4906   0.05391   NS
Sequence       1  0.686301   0.686301   14.5631   0.002141  ***
Error         13  0.61264    0.0471261
Total         29  6.38875

Root Mean Square Error = 0.217086 ; CV = 0.0347883

phi = 1.49843
Power of the test = 0.500777
1 - ( Power of the test ) = 0.499223

around the ratio:[test form]/[ref form])=[1.028, 1.3612]



I havnt randomisation of subject 15, and then they put the result in kinetica they excluded him, they havnt protocol for outlier etc
thank you :-)
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-06-10 00:37

@ khaoula
Posting: # 13045
Views: 13,154
 

 Bag full of bugs

Hi Khaoula,

I’m too lazy to check with my old installation of Kinetica. But there are some points to note:

» SOURCE       D.F      SS         MS        F        p
» Period         1  0.0264787  0.0264787   0.56186  0.4669    NS
» Subject(Seq)  13  4.8517     0.373208    7.91934  0.0003379 ***
» Formulation    1  0.211625   0.211625    4.4906   0.05391   NS
» Sequence       1  0.686301   0.686301   14.5631   0.002141  *** [image]
» Error         13  0.61264    0.0471261
» Total         29  6.38875

Old story. Kinetica calculates the sequence effect wrong – since ages. See this thread.
Instead of F=0.686301/0.0471261=14.5631 (which is highly significant with p=0.002141) the correct test is against Subject(Seq): F=0.686301/0.373208=1.8389 (which is not sig­nificant with p=0.1982).

» Root Mean Square Error = 0.217086

The RMSE is a crude estimate of the CV and should be avoided.

»                                   ; CV = 0.0347883 [image]

That’s nonsense! Is this part of Kinetica’s original output?* Why didn’t you bother to manu­ally calculate the CV according to the formula I gave you above?
\(CV\% = 100\sqrt{e^{0.0471261}-1} = 21.97\%\)

» Power of the test = 0.500777 [image]
» 1 - ( Power of the test ) = 0.499223

Wrong. If you insist in post hoc power: 0.09885. With a T/R-ratio close to the upper limit of the acceptance range would you really expect a ~50% chance to pass BE in 15 subjects?

» around the ratio:[test form]/[ref form])=[1.028, 1.3612] [image]

Yes, but around which ratio? I would guess: \(\sqrt{1.028 \times 1.3612} = {\color{Red} {1.829\ldots}}\) The geometric mean of Test is 559.01 and the one of Reference 461.21. As a first guess of the T/R-ratio we get 461.21/559.01=1.2121. Since this is an imbalanced dataset, we have to use the least squares (or adjusted) means instead, which are 552.58 and 456.58. Therefore, T/R=1.2103. What does Kinetica “calculate” here?

[image]Due to the bug in Kinetica this is likely wrong. In Phoenix/WinNonlin I got:

Hypothesis       DF    SS        MS       F_stat  P_value
Period            1  0.027982  0.027982  0.07021  0.79519
Sequence*Patient 13  5.181510  0.398578  7.49434  0.00045
Sequence          1  0.105223  0.105223  1.97847  0.18300
Formulation       1  0.271883  0.271883  5.11214  0.04155
Error            13  0.691389  0.053184

T/R 121.02% [90% CI: 104.22%, 140.53%]


BTW:
Parameter              Estimate
Var(Sequence*Patient)  0.172697
Var(Residual)          0.053184
Intersubject CV        0.434173
Intrasubject CV        0.233717

Check: CV% = 100√(ℯ0.053184 – 1) = 23.37%. [image] Low for omeprazole, but possible.

If you want an independent evaluation by noncommercial software: In R (you find the script in my lectures) I got:
121.02% [104.22–140.53%] CV 23.37% [image]

» I havnt randomisation of subject 15, and then they put the result in kinetica they excluded him,

The randomization of an excluded subject is not required anyhow.


  • Confirmed. I don’t have the slightest idea how Kinetica gets this number.
    Discovered this interesting post at David’s PKPD-list.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Denmark,
2014-06-10 01:00

@ Helmut
Posting: # 13046
Views: 12,962
 

 Randomisation?

Hi Khaoula and Helmut,

» » I havnt randomisation of subject 15, and then they put the result in kinetica they excluded him,
»
» The randomization of an excluded subject is not required anyhow.

I think what was meant was that the allocated sequence for subject 15 is unknown/missing/evaporated/gone/lost? If aiming for balance which is standard you may be able to guess it pretty well provided the sequences of the other 8+7 subjects are correct :pirate:
Anyways, guessing isn't science, and I just wonder how such an event could happen - could you comment, Khaoula? How could info on randomisation for a single subject be missing?

I could be wrong, but...
Best regards,
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-06-10 14:53

@ ElMaestro
Posting: # 13050
Views: 12,887
 

 Randomisation?

Hi ElMaestro & Khaoula,

» I think what was meant was that the allocated sequence for subject 15 is unknown/missing/evaporated/gone/lost? If aiming for balance which is standard you may be able to guess it pretty well provided the sequences of the other 8+7 subjects are correct :pirate:

Since we have seven subjects in sequence RT and eight in sequence TR, we might guess that #15 was planned for sequence RT. ;-)

» Anyways, guessing isn't science,

Yep.

» and I just wonder how such an event could happen […]

Khaoula wrote in her OP:
» » we have design of bioequivalence study cross over design with 16 subject (randomised), but at the firt period the 15 suject didn't come […]

» How could info on randomisation for a single subject be missing?

Good question.

» » without protocol for the management of dropouts

I wouldn’t call #15 a “drop-out” but rather a “no-show”.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
khaoula
★    

Algeria,
2014-06-11 00:17

@ Helmut
Posting: # 13056
Views: 12,883
 

 Randomisation?

Hi ElMaestro & Helmut
I think that it's important to have randomisation of subject 15 (I dont now the basis of a software (kinetica) and how tu use it), I havnt randomisation because they dont give me it, they asked me to try to understand where is the error, so when I search I found that oeprazole is a HVDP and I try to understand what FDA and EMA do, I concluded that our design is't appropriate (must have replicate design), but with Phoenix/WinNonlin CV=23.37%, so the problem of our study is number of subject, with hightest number we can conclude to biequivalence (the test drug is really bioequivalent of réference, this study was done to learn how to do biequivalennce study)


I calculated CV with error = 0.0471261 and I had CV% = 100√(ℯ0.0471261 – 1) = 21.97% but I thought that I was wrong
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-06-11 02:01

@ khaoula
Posting: # 13057
Views: 13,177
 

 Problems resolved?

Hi Khaoula!

» […] I havnt randomisation because they dont give me it, they asked me to try to understand where is the error, […]

May I ask: Who are “they”?

» […] our design is't appropriate (must have replicate design), […]

Not necessarily. Only if you want to widen the acceptance range (EMA) or go with reference-scaling (FDA). But if CVWR <30% scaling is not allowed.

» […] the problem of our study is number of subject, […]

Mainly. If we assume the T/R-ratio and CV “as carved in stone” – which I would not recommend anyway – you would have needed the following sample sizes in order to demonstrate BE with 80% power:

design  method                     sample size
2×2×2   unscaled (80–125%)            630
2x2×3   fully replicated unscaled     472
2×2×4   fully replicated unscaled     316
2x2x3   scaled (EMA)                  474
2×2×4   scaled (EMA)                  316
2x2x3   scaled (FDA)                  474
2×2×4   scaled (FDA)                  318


Note that reference-scaling would not help with such a low CV.

You have a lot of problems arising from the study:
  • The T/R-ratio of 121% is close to the upper limit of the acceptance range. Since the confidence interval (104.22%, 140.53%) does not include 100%, formulations are significantly different (p 0.04155). Is there an end of bugs in Kinetica? Despite the – wrong! – CI does also not include 100%, formulations are considered not significantly different (p 0.05391). Amazing.
  • With such a large deviation from the reference any attempt to show BE in a larger study is not ethical, IMHO.
  • The CV is untypically low for omeprazole’s Cmax. Maybe you were just lucky. But luck is not what you should expect for the next study.

» the test drug is really bioequivalent of réference, […]

Why do you think so? You would need hundreds of subjects to squeeze the CI within the acceptance range.

» this study was done to learn how to do biequivalennce study

Can you briefly summarize what you have learned?

A drug is that substance which, when injected into a rat,
   will produce a scientific report.
    Anonymous


Pharmacokinetics: one of the magic arts of divination
   whereby needles are stuck into dummies in an attempt
   to predict profits.
    Stephen Senn


» I calculated CV with error = 0.0471261 and I had CV% = 100√(ℯ0.0471261 – 1) = 21.97% but I thought that I was wrong

So instead of following a formula given in numerous papers and textbooks you rather trusted in the output of a piece of software?

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
khaoula
★    

Algeria,
2014-07-24 13:36

@ Helmut
Posting: # 13306
Views: 12,485
 

 Problems resolved?

Hi Helmut

I made ​​a big mistake when I copied the results of Cmax of our study and publish them here, I'm really sorry and confused, can you use the right results and recalculate it with your software kinetica v5 and Phoenix/WinNonlin? I don't want to waste your time and I am so grateful for your help, thank you
[image]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-07-27 01:37

@ khaoula
Posting: # 13317
Views: 12,697
 

 corrected data in PHX/WNL

Hi Khaoula!

» […] can you use the right results and recalculate it with your software kinetica v5 and Phoenix/WinNonlin?

Meanwhile I retired the machine where Kinetica is installed on. Results from Phoenix/WinNonlin6.3:

Hypothesis       DF    SS        MS       F_stat  P_value
Period            1  0.206671  0.206671  0.55377  0.47002
Sequence*Patient 13  4.851704  0.373208  7.80516  0.00036
Sequence          1  0.087929  0.087929  1.83892  0.19816
Formulation       1  0.202662  0.202662  4.23841  0.06014
Error            13  0.621602  0.047816

RefLSM  6.1477 (RefGeoLSM  467.64)
TestLSM 6.3125 (TestGeoLSM 551.40)
Diff    0.1647 (SE 0.08002, df 13)

T/R 117.91% [90% CI: 102.33%, 135.86%]

Parameter              Estimate
Var(Sequence*Patient)  0.162696
Var(Residual)          0.047816
Intersubject CV        0.420332
Intrasubject CV        0.221308


Checks:
T/R = 100(551.40/467.64) = 100ℯ0.1647 = 117.91%
90% CI = 100ℯ0.1647±1.7709√0.047816(1/(2×8)+1/(2×7)) = 102.33–135.86%
CV% = 100√0.047816 – 1 = 21.13%

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-06-11 19:51

@ khaoula
Posting: # 13059
Views: 12,957
 

 Which version of Kinetica?

Dear Khaoula,

I re-calculated your Cmax-data in Kinetica v5.0.10. I didn’t import the “all missing” row of subject #15 and asked for log-transformed analysis. Now I’m even more confused. Below my results:

            ------------------------------------------
            LATIN SQUARE DESIGN : ANOVA TABLE for Cmax
            ------------------------------------------
LATIN SQUARE with Log (neperian) option


SOURCE       D.F    SS         MS          F         p
Period         1  0.0092314  0.0092314   0.174967  0.6826    NS
Subject(Seq)  13  5.18151    0.398578    7.55441   0.0004324 ***
Formulation    1  0.277381   0.277381    5.25731   0.03917   ***
Sequence       1  0.788575   0.788575   14.9462    0.001947  ***
Error         13  0.685892   0.0527609
Total         29  6.94259
---------------------------------------------------------------------
                     N   Mean     SD         SEM     GeoMean  Geo SD
Formulation:num = R 15  6.13386  0.498125  0.128615  461.211  1.64563
Formulation:num = T 15  6.32617  0.477449  0.123277  559.01   1.61196
---------------------------------------------------------------------
Root Mean Square Error = 0.229697 ; CV = 0.0368695

phi = 1.62131
Power of the test = 0.564385
1 - ( Power of the test ) = 0.435615
Minimum detectable difference = 0.192312
---------------------------------------------------------------------
BIOEQUIVALENCE TESTS FOR
Level R and level T

Reference Confidence Interval: [0.8, 1.25]
Geomean Ratio (Test/Reference) = 1.21205
90% standard confidence interval
(around the ratio:[test form]/[ref form])=[1.0448, 1.4061]
t(0.05 - 13df) = 1.7709

  Cannot conclude equivalence.


Still wrong in many places, but closer to Phoenix/WinNonlin and R.
                 PE        90% CI       MSE     CV
PHX/WNL and R  121.02  104.22–140.53  0.053184 23.37
Kinetica 5     121.20  104.48–140.61  0.052761 23.28 (manually from MSE)

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
khaoula
★    

Algeria,
2014-06-12 00:41

@ Helmut
Posting: # 13061
Views: 12,791
 

 Which version of Kinetica?

Dear Helmut,

we have (Kinetica Version 4.4.1)

I sent you a private mail, can you read it? thank you :-)


Edit: Look at your inbox. I have a 4.x version of Kinetica installed on one of my old machines. Maybe I will check – can’t promise. [Helmut]
khaoula
★    

Algeria,
2014-07-25 00:49

@ Helmut
Posting: # 13310
Views: 12,506
 

 Which version of Kinetica?

thank you verry much Helmut !!!!
so, what I saw:
the sequence effect with kinetica V5 is wrong (like V4): seq effect = MS seq/ MS error(within), but the right mathematc formula is seq effect MS seq / MS subj(seq)(between) (I understand that thanks to you)
calculation of CV: the software dont take this rule: [image]but this one: [image] (it's false)

but for GMR, I'm really confused, It's an unbalenced cross over so you told me to use the least squares (or adjusted) means instead I read this post

and use this formula:
[image]

I have'nt found the same result (1,21), but 1,18 like kinetica v4... where is the problem? :-(
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2014-07-27 02:23

@ khaoula
Posting: # 13318
Views: 12,547
 

 adjusted means (aka LSMs)

HI Khaoula,

» the software […] take this rule: [image] (it's false)

Yes, it’s wrong. Where did you find this formula? You would get a negative value for the root. Maybe Kinetica uses (also wrongly) CV% = 100√MSE² – 1 which would give 4.72% – at least closer to your original 3.47% – no idea. It’s not my job to un­veil all potential bugs in a software I neither seriously used nor validated myself…

» but for GMR, I'm really confused, It's an unbalenced cross over so you told me to use the least squares (or adjusted) means instead I read this post
»
» I have'nt found the same result (1,21), but 1,18 like kinetica v4... where is the problem?

Again, no idea. How did you calculate a GMR of 1.21? Your corrected log-transformed data gives a LSM for the reference of 6.1477 and for the test 6.3125. These values are derived exactly as I stated in my old post.

subj seq form per logCmax  subj seq form per logCmax
────────────────────────────────────────────────────
  1   RT   R   1   6.6399    3   TR   R   2   6.4505
  2   RT   R   1   5.7104    4   TR   R   2   6.1485
  7   RT   R   1   5.7268    5   TR   R   2   6.3439
  8   RT   R   1   5.8944    6   TR   R   2   6.2634
  9   RT   R   1   5.4931   11   TR   R   2   5.9162
 10   RT   R   1   6.0379   12   TR   R   2   6.5582
 14   RT   R   1   6.6399   13   TR   R   2   7.0733
 15   RT?                   16   TR   R   2   5.4467
────────────────────────────────────────────────────
               x   6.0203                 x   6.2751


subj seq form per logCmax  subj seq form per logCmax
────────────────────────────────────────────────────
  1   RT   T   2   6.6134    3   TR   T   1   6.6529
  2   RT   T   2   5.3753    4   TR   T   1   6.4425
  7   RT   T   2   5.4806    5   TR   T   1   6.3386
  8   RT   T   2   6.2748    6   TR   T   1   6.3026
  9   RT   T   2   6.2500   11   TR   T   1   6.5568
 10   RT   T   2   6.3491   12   TR   T   1   6.5425
 14   RT   T   2   6.6134   13   TR   T   1   7.2442
 15   RT?                   16   TR   T   1   5.8260
────────────────────────────────────────────────────
               x   6.1367                 x   6.4883


Mean of R in sequence RT is 6.0203 and in sequence TR is 6.2751. LSM = (xRT+xTR)/2 = 6.1477. 
Similarily we get the LSM of the test 6.3125.
Backtransformed LSMs are ℯ6.1477 = 467.64 (R) and ℯ6.3125 = 551.40 (T). Therefore, the GMR is 551.40/467.64 = 1.1791, agreeing with what PHX/WNL reports.


PS: Sometimes in study reports the geometric means of treatments and results of the ANOVA are given in the same table (without the LSMs). Then – if the study was not balanced – people are confused, since the ratio of geometric means does not match the point estimate (calculated from the LSMs). Here the backtransformed geometric means are 471.64 (R) and 557.90 (T). Wrongly calculating the ratio 557.90/471.64 = 1.1829 1.1791…

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
khaoula
★    

Algeria,
2014-08-25 21:41

@ Helmut
Posting: # 13430
Views: 11,990
 

 adjusted means (aka LSMs)

Hi Helmut,
thank you very much for your answer, I'm very sorry to reply so late, at first I did not have internet, then I fell very ill.
so I was wrong, I thought that after I sent the new results you answered me and GMR = 1.21 again !!! so now it's OK there is no problem between kinetica and unbalanced cross over !!! :-D

two problems only:
  1. calculation of CV (but it's not my job to find problem of software).
  2. calculation of sequence effect: I read the book "Design and Analysis of Bioavailability and Bioequivalence Studies "(Shein-Chung Chow) Chapter 3 Statistical Inferences for Effects from a Standard 2*2 Crossover Design, I understood how to calculate the differents effects applying student test and Analysis of Variance :D
can I sent to you tne result of AUCs and you recalculate them with Phoenix/WinNonlin6.3? ( to confirm the miscalculation of sequence effect)?
thank you very much :-) I am very happy to have met you, you help me a lot !!!!!
Activity
 Thread view
Bioequivalence and Bioavailability Forum |  Admin contact
19,514 posts in 4,141 threads, 1,336 registered users;
online 9 (0 registered, 9 guests [including 7 identified bots]).
Forum time (Europe/Vienna): 20:54 CEST

I have not failed 700 times. I have not failed once.
I have succeeded in proving
that those 700 ways will not work.    Thomas Alva Edison

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
HTML5