jag009
★★★

NJ,
2013-04-22 18:39
(3992 d 17:15 ago)

Posting: # 10458
Views: 26,091
 

 FDA's HVD SAS Code from Progesterone Guidance [RSABE / ABEL]

Hi everyone,

I am getting some strange results using the FDA Progesterone Scaled Average Bioequivalence SAS Code → The portion to compute the Unscaled Average 90% C.I.

PROC MIXED
data=pk;
CLASSES SEQ SUBJ PER TRT;
MODEL LAUCT = SEQ PER TRT/ DDFM=SATTERTH;
RANDOM TRT/TYPE=FA0(2) SUB=SUBJ G;
REPEATED/GRP=TRT SUB=SUJ;
ESTIMATE 'T vs. R' TRT 1 -1/CL ALPHA=0.1;
ods output Estimates=unsc1;
title1 'unscaled BE 90% CI - guidance version';
title2 'AUCt';
run; data unsc1;
set unsc1;
unscabe_lower=exp(lower);
unscabe_upper=exp(upper);
run;


Should the Test/Reference Ratio from the Scaled Average BE computation be similar (if not the same) as the ratio computed from the above Unscaled Average BE code?

Thanks
John


Edit: Category changed. [Helmut]
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-04-22 19:47
(3992 d 16:07 ago)

@ jag009
Posting: # 10460
Views: 22,381
 

 Proc MIXED vs. Proc GLM

Hi John,

❝ Should the Test/Reference Ratio from the Scaled Average BE computation be similar (if not the same) as the ratio computed from the above Unscaled Average BE code?


More details, pleeze! Define similar. ;-) Note that FDA’s ABE is evaluated by a Proc MIXED and the partial replicate by Proc GLM. You will see differences for incomplete data-sets (GLM drops subjects, MIXED keeps them).

Even for a fully replicated design (but imbalanced and incomplete like EMA’s data set I) SAS’ merge will drop subjects. In Phoenix I get PEs of 1.1546132 (RSABE) and 1.1565765 (ABE).

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2013-04-22 22:33
(3992 d 13:21 ago)

@ Helmut
Posting: # 10464
Views: 22,208
 

 Proc MIXED vs. Proc GLM

Hi Helmut,

Can you try this and let me know? Yes there are missing datapoints. The other 2 parameters worked out fine. I ran data from another study and everything worked as well. Only this dataset is giving me the finger. The difference is the missing values as other datasets had values for all subjects. Maybe the missing data is messing things up?

Data columns are Subject Period Sequence Formulation PK-parameter

My SAS SCABE for this parameter is

Theta    Bound y    (S2wr)    (sWR)   (T/R) Pt Est  95% Upper Crit
0.79669   -0.22198  0.41135  0.64137    89.8994     -0.18423


I ran Unscaled Avg BE (Progesterone guidance SAS code)

T/R Ratio     90% CI
101.021    80.7739 - 126.343


:confused:

Sequence 1=ABB, 2=BAB, 3=BBA

1 3 BBA A 31.432
2 1 ABB A 23.746
4 1 ABB A 41.258
5 3 BBA A 24.887
6 2 BAB A 237.003
7 3 BBA A 35.132
8 1 ABB A 22.399
9 2 BAB A 77.187
10 2 BAB A 180.386
11 3 BBA A 40.623
12 1 ABB A 15.382
14 2 BAB A -
15 3 BBA A 75.768
17 1 ABB A 15.219
18 2 BAB A 136.554
19 1 ABB A 53.966
20 2 BAB A 18.509
21 3 BBA A 24.439
23 2 BAB A 23.710
25 2 BAB A 16.264
26 3 BBA A 41.160
27 1 ABB A 63.364
28 3 BBA A 46.344
29 1 ABB A 66.059
30 2 BAB A 2.539
31 3 BBA A -
32 1 ABB A 34.002
33 2 BAB A 61.070
34 3 BBA A 149.674
35 2 BAB A 26.440
36 1 ABB A 54.400
37 2 BAB A 39.185
38 3 BBA A 201.983
39 1 ABB A 17.831
40 3 BBA A 85.293
1 1 BBA B 40.705
1 2 BBA B 68.864
2 2 ABB B 13.020
2 3 ABB B 36.121
4 2 ABB B 102.646
4 3 ABB B 54.459
5 1 BBA B 9.396
5 2 BBA B 22.999
6 1 BAB B 309.754
6 3 BAB B 163.058
7 1 BBA B 41.710
7 2 BBA B 44.472
8 2 ABB B 2.915
8 3 ABB B -
9 1 BAB B 88.910
9 3 BAB B 109.897
10 1 BAB B 373.070
10 3 BAB B 529.088
11 1 BBA B 74.469
11 2 BBA B 80.379
12 2 ABB B 26.124
12 3 ABB B 18.094
14 1 BAB B 2.280
14 3 BAB B -
15 1 BBA B 99.409
15 2 BBA B 134.023
17 2 ABB B -
17 3 ABB B 3.694
18 1 BAB B 49.218
18 3 BAB B 42.806
19 2 ABB B 61.764
19 3 ABB B 47.537
20 1 BAB B 74.644
20 3 BAB B 36.181
21 1 BBA B 47.737
21 2 BBA B 30.634
23 1 BAB B 11.766
23 3 BAB B 14.121
25 1 BAB B 29.943
25 3 BAB B 20.244
26 1 BBA B 109.626
26 2 BBA B 80.252
27 2 ABB B 33.349
27 3 ABB B 124.874
28 1 BBA B 29.083
28 2 BBA B 23.024
29 2 ABB B 34.671
29 3 ABB B 23.049
30 1 BAB B 2.782
30 3 BAB B 2.249
31 1 BBA B 57.630
31 2 BBA B -
32 2 ABB B 24.816
32 3 ABB B 39.266
33 1 BAB B 68.419
33 3 BAB B 85.845
34 1 BBA B 51.760
34 2 BBA B 54.845
35 1 BAB B 30.751
35 3 BAB B 2.921
36 2 ABB B 70.271
36 3 ABB B 38.989
37 1 BAB B 114.890
37 3 BAB B 54.372
38 1 BBA B 98.310
38 2 BBA B 147.839
39 2 ABB B 534.900
39 3 ABB B 16.465
40 1 BBA B 176.916
40 2 BBA B 105.915


Thanks
John
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-04-23 00:10
(3992 d 11:44 ago)

@ jag009
Posting: # 10466
Views: 21,979
 

 SAS vs. PHX

Hi John,

❝ My SAS SCABE for this parameter is

Theta    Bound y  (S2wr)    (sWR)   (T/R) Pt Est  95% Upper Crit

0.79669 -0.22198  0.41135  0.64137    89.8994     -0.18423


No need to post Theta (a constant in RSABE). ;-)

My PHX6.3 code gives for RSABE …
boundy     S²wr     Swr       PE       95% upper
-0.221998  0.41138  0.64139  89.8990  -0.18424336


❝ I ran Unscaled Avg BE (Progesterone guidance SAS code)

T/R Ratio     90% CI

101.021    80.7739 - 126.343


… and for ABE
  PE          90% CI
104.782  69.7855 – 157.328


But: PHX’ LME kicked my ass with “Warning 11094: Negative final variance component. Consider omitting this VC structure.” Nice. Old story with partial replicates – overspecified model since T is not repeated. We have seen in the past that SAS and PHX give different results in such a case. BTW, did you get “Convergence criteria met but final hessian is not positive definite.” or somefink similar in Sas?

If I go with the ABE-module I got:
ERROR 11070:  Error in Satterthwaite DF. Try using Residual DF option.
… which I tried, only to get:
ERROR 11070:  Error in Satterthwaite DF. Try using Residual DF option.
That’s funny! Same if I exclude subjects #14 and #31 (no results for T). What the heck?

Running PHX’ PBE/IBE module I first obtained …
Warning 11121: Subject 31 had incomplete design and was discarded.
Warning 11121: Subject 14 had incomplete design and was discarded.
Warning 11121: Subject 8 had incomplete design and was discarded.
Warning 11121: Subject 17 had incomplete design and was discarded.


… and a PE which is exactly like in RSABE 89.899043.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Shuanghe
★★  

Spain,
2013-04-23 13:45
(3991 d 22:09 ago)

@ Helmut
Posting: # 10471
Views: 21,906
 

 SAS vs. PHX

Hi Helmut,

❝ My PHX6.3 code gives for RSABE …

boundy     S²wr     Swr       PE       95% upper

-0.221998  0.41138  0.64139  89.8990  -0.18424336


Just out of curiosity I tried my SAS macro and it only gives Swr, PE, 95% upper. They are the same as yours, which is slightly different from John's.

❝ ❝ I ran Unscaled Avg BE (Progesterone guidance SAS code)

❝ ❝ T/R Ratio     90% CI

❝ ❝ 101.021    80.7739 - 126.343


❝ … and for ABE

  PE          90% CI

104.782  69.7855 – 157.328


Now, my average BE gives:
PE: 1.01021, same as John's :yes:
90% CI: 80.7733 - 126.3450, different from both of yours. :no:
Weird.

❝ But: PHX’ LME kicked my ass with “Warning 11094: Negative final variance component. Consider omitting this VC structure.” Nice. Old story with partial replicates – overspecified model since T is not repeated. We have seen in the past that SAS and PHX give different results in such a case. BTW, did you get “Convergence criteria met but final hessian is not positive definite.” or somefink similar in Sas?


SAS did give me warning of " Convergence criteria met but final hessian is not positive definite".

❝ That’s funny! Same if I exclude subjects #14 and #31 (no results for T). What the heck?


after delete subject 14 and 31, same result for SABE but for ABE:
PE: 98.04322%
90% CI: 78.61817 - 122.26784.
Man, I never doubted the macro before. should I worry about it now? :confused:
Shall we compare the result from EMA's data set?

Edit:
Just tried EMA's 3-period data set, using "logdata" directly from the dataset.
SABE gives:
Swr=0.11397
PE=1.02264
95% upper= -0.00397289


ABE gives the same result as in the EMA's Q&A. 102.264 (97.05-107.76)

delete subject 5 and 8 (don't ask why those subjects)
SABE gives:
Swr=0.11803
PE=1.01341
95% upper= -0.00510258


ABE gives:
PE=101.218
90% CI= 95.7687 - 106.978


Can anyone check it?

All the best,
Shuanghe
d_labes
★★★

Berlin, Germany,
2013-04-23 18:50
(3991 d 17:04 ago)

@ Shuanghe
Posting: # 10475
Views: 21,797
 

 SAS vs. SAS

Hi Shuanghe,

❝ Now, my average BE gives:

PE: 1.01021, same as John's :yes:

90% CI: 80.7733 - 126.3450, different from both of yours. :no:

❝ Weird.


don't worry. My results (using Johns code for ABE as given above at start of the thread) under SAS9.2 are:
  point est.  90% confidence interval
  101.0213%     80.7733   126.3450
  s2wR = 0.4210 -> CVwR = 72.35%

Deleting the 4 subjects with missings (same happens in the intra-subject contrast calculations) gives
  point est.  90% confidence interval
  90.5032%     73.5627   111.3449
  s2wR = 0.4001 -> CVwR = 70.14%

very similar to the numbers used for the RSABE criterion :cool:.

But I don't believe in this numbers anyway. I think the optimizer stops here arbitrarily as almost ever for a partial replicate design in which the intra-individual variability for T is not identifiable.
Any modification to the code, f.i. fitting a model with no subject-by-treatment interaction via the CS covariance structure crashes with infinite likelihood and an estimate of s2wT=0!

I always wondered why the FDA insists on the Proc MIXED code, especially for that design.
On the other hand in the context of RSABE linearized criterion the point estimator and its 90% CI are calculated via intra-subject contrast T-R.
Why not use these results for ABE also :confused:.

To increase the confusion here the results of the mighty oracle EMA code (same Proc GLM as for a 2x2 crossover for the PE and CI, s2wR from analysis of data for R (B) only):
  point est.  90% confidence interval
  98.4476%     78.6492   123.2298
  s2wR = 0.39824522 -> CVwR = 69.94%


So much numbers to choose between :-D.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2019-11-12 14:01
(1597 d 20:53 ago)

@ d_labes
Posting: # 20788
Views: 7,952
 

 SAS vs. SAS vs. Phoenix

Hi to all victims of the partial replicate with incomplete data,

following extensive off-list discussions with PharmCat (who is working on a Julia-library; see here and there) I checked again what we got with the FDA’s covariance structure, Satterthwaite’s df (PE, 90% CI).
Not the slightest idea how I arrived at the results given above. Sorry for the confusion caused.
  • John’s first dataset.
    • John (SAS, v?)
      101.021    80.7739 – 126.343
    • Shuanghe (SAS, v?)
      101.021    80.7733 – 126.3450
    • Detlew (SAS, v9.2)
      101.0213   80.7733 – 126.3450
    • Myself (Phoenix v6.3 and v8.1)
      No convergence with the default setup.
      However, when I lowered the convergence criterion to 1E-11:
      101.0213   80.7733 – 126.3450

  • John’s second dataset (ln AUCi).
    • John (SAS, v?)
      No convergence.
    • Myself (Phoenix v6.3 and v8.1)
      Convergence with warning, tweaking the setup doesn’t help.
      95.2967    88.4014 – 102.7298
      Compound symmetry (as suggested by Detlew)
      95.2967    88.4014 – 102.7298
    • Detlew (SAS, v9.2, CS)
      95.30      88.23   – 102.82
In all cases simplifying the covariance structure (in SAS-lingo FA0(1) instead of the FA0(2) in the guidance): Convergence without warning.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2013-04-23 17:56
(3991 d 17:58 ago)

@ Helmut
Posting: # 10473
Views: 21,516
 

 SAS vs. PHX

Hi Helmut and shanghue,

❝ That’s funny! Same if I exclude subjects #14 and #31 (no results for T). What the heck?


Yes but Proc Mixed takes into account the missing data correct?

Great, I opened up a Pandora's Box...
Let me investigate further and let you guys know.

Calling for Detlew the SAS guru (heheh, one of the gurus to be politically correct) :-D. He might have some idea on this...

Thanks
John
jag009
★★★

NJ,
2013-04-23 19:07
(3991 d 16:47 ago)

@ jag009
Posting: # 10476
Views: 22,447
 

 SAS Warning (Note) on Proc Mixed

Thanks guys (and girls),

Here are the notes from my SAS Log on Proc Mixed.
NOTE: 6 observations are not included because of missing values.
NOTE: Convergence criteria met but final hessian is not positive definite.
NOTE: Asymptotic variance matrix of covariance parameter estimates has been found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=SATTERTH.


I think this is problematic... One runs a dataset and SCABE criteria for Swr not met so he/she resorts to the ABE routine. Tada!

Thanks
John
jag009
★★★

NJ,
2013-04-30 23:09
(3984 d 12:45 ago)

@ jag009
Posting: # 10515
Views: 21,910
 

 Nut Job..

Hi everyone,

Continuing the sage... I now have full dataset and I included AUCt as well.

Once again using the ABE procedure with Proc Mixed from FDA progesterone guidance. I was fine with ln AUCt but did get a message in the SAS log:

NOTE: Convergence criteria met but final hessian is not positive definite.
NOTE: Asymptotic variance matrix of covariance parameter estimates has been found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=SATTERTH.


For AUCt
Ratio: 94.7377; 90% CI: 87.4898-102.586

However, Proc Mixed failed to compute the above for ln AUCi and gave me a warning message in SAS log:

WARNING: Did not converge. :confused: :confused:
WARNING: Output 'Estimates' was not created....


The last output from Proc Mixed on ln AUCi was:
Covariance Parameter Values At Last Iteration
Cov Parm   Subject   Group          Estimate
FA(1,1)    subject                  0.3722
FA(2,1)    subject                  0.3975
FA(2,2)    subject                  0.2362
Residual   subject formulation Ref  0.06248
Residual   subject formulation Test 0.01460


I used the same ABE SAS Code posted in the first message in this thread (and the codes are the same for AUCt and AUCi).

The Dataset:
Subject, Sequence, Period, Treatment, ln AUCt, ln AUCi.
Sequence: ABB = 1, BAB = 2, BBA = 3

1 BBA 3 A 7.392949199 7.966255738
2 ABB 1 A 7.486969192 7.642465619
3 BAB 2 A 7.795040591 7.900675886
4 ABB 1 A 7.704527971 7.810454463
5 BBA 3 A 7.74341453 7.813019037
6 BAB 2 A 8.862899236 8.887092743
7 BBA 3 A 7.474575339 7.520470843
8 ABB 1 A 6.649783454 6.786800469
9 BAB 2 A 7.900081068 7.917934089
10 BAB 2 A 8.498478152 8.537308319
11 BBA 3 A 7.687297563 7.764432727
12 ABB 1 A 7.854077711 8.056187995
14 BAB 2 A 6.834931866 7.311990298
15 BBA 3 A 8.004066923 8.015037211
17 ABB 1 A 7.954169654 8.081352274
18 BAB 2 A 8.813960369 8.878784152
19 ABB 1 A 7.936215453 8.005634299
20 BAB 2 A 7.585678156 7.624085152
21 BBA 3 A 7.882511503 7.908029742
23 BAB 2 A 7.252320394 7.329537863
25 BAB 2 A 7.460390761 7.612326456
26 BBA 3 A 7.519507463 7.544075461
27 ABB 1 A 7.842873653 7.906903707
28 BBA 3 A 7.974813947 8.117347067
29 ABB 1 A 7.952082082 8.030115018
30 BAB 2 A 6.969401496 7.626733448
31 BBA 3 A 7.375110404 7.425354311
32 ABB 1 A 7.854046269 7.909554125
33 BAB 2 A 8.180100682 8.240037917
34 BBA 3 A 8.498685373 8.50785008
35 BAB 2 A 7.689511601 7.753216597
36 ABB 1 A 7.697065912 7.731297188
37 BAB 2 A 7.743985836 7.768691851
38 BBA 3 A 8.860378559 8.90071176
39 ABB 1 A 7.649143902 7.760258994
40 BBA 3 A 8.178085044 8.269703924
42 ABB 1 A 7.699680257 8.477479957
44 ABB 1 A 8.469052606 8.546889654
45 BBA 3 A 8.734765894 8.773995092
46 ABB 1 A 7.438072894 7.502691878
47 BBA 3 A 7.70220194 7.870368371
48 BAB 2 A 8.326805753 8.371119472
49 BAB 2 A 8.23649812 8.251025744
51 ABB 1 A 8.89042446 8.926528534
52 BBA 3 A 8.319488803 8.645876426
53 ABB 1 A 8.254585577 8.26745589
54 BAB 2 A 8.774547697 8.830803941
55 BBA 3 A 7.695208954 7.75602795
56 BAB 2 A 8.088395381 8.173364805
57 ABB 1 A 7.721135381 7.754655632
59 BAB 2 A 7.624704432 7.649123891
1 BBA 1 B 7.918112873 7.956429845
1 BBA 2 B 8.165388938 8.236181092
2 ABB 2 B 7.648429922 7.840311803
2 ABB 3 B 7.767760063 7.99990909
3 BAB 1 B 7.84509492 7.903718188
3 BAB 3 B 7.004410917 7.201138817
4 ABB 2 B 8.273957131 8.299682496
4 ABB 3 B 7.893447812 7.922782677
5 BBA 1 B 7.873074242 7.993014477
5 BBA 2 B 7.928661129 7.973013452
6 BAB 1 B 8.953187117 9.070644877
6 BAB 3 B 8.620163052 8.635798822
7 BBA 1 B 7.733732169 7.925830821
7 BBA 2 B 7.883843938 7.915115941
8 ABB 2 B 7.040298183 7.138905094
8 ABB 3 B 6.455060189 6.919911097
9 BAB 1 B 8.246138252 8.302636666
9 BAB 3 B 8.369926761 8.407009731
10 BAB 1 B 8.854970576 8.867276941
10 BAB 3 B 8.985634977 9.003601402
11 BBA 1 B 8.178239438 8.255348312
11 BBA 2 B 8.286612805 8.339918637
12 ABB 2 B 8.240486403 8.305583617
12 ABB 3 B 7.800706552 7.826649818
14 BAB 1 B 7.553067914 7.897693791
14 BAB 3 B 7.105642559 7.345723072
15 BBA 1 B 8.238080695 8.313684606
15 BBA 2 B 8.446723945 8.453986902
17 ABB 2 B 6.918289598 8.123267843
17 ABB 3 B 7.238486783 7.484658489
18 BAB 1 B 8.00908587 8.197899082
18 BAB 3 B 7.76105637 7.935726911
19 ABB 2 B 8.102567049 8.141680789
19 ABB 3 B 7.693701307 7.710909115
20 BAB 1 B 8.263957891 8.29405139
20 BAB 3 B 8.174826537 8.190936874
21 BBA 1 B 8.120727864 8.167847324
21 BBA 2 B 7.93456468 8.013054162
23 BAB 1 B 7.358441651 7.473058861
23 BAB 3 B 7.48803103 7.535879021
25 BAB 1 B 7.602494691 7.697937968
25 BAB 3 B 7.387379977 7.469852557
26 BBA 1 B 8.329193303 8.344287565
26 BBA 2 B 8.037593588 8.060771153
27 ABB 2 B 7.905330655 7.935414562
27 ABB 3 B 8.082943041 8.29080137
28 BBA 1 B 7.738586593 7.774590185
28 BBA 2 B 7.859365663 7.889316764
29 ABB 2 B 7.888838125 8.04080281
29 ABB 3 B 7.779364076 7.821842385
30 BAB 1 B 6.982335711 7.543216142
30 BAB 3 B 6.697153994 6.829441243
31 BBA 1 B 7.747856304 7.809290887
31 BBA 2 B 7.560504854 7.797942664
32 ABB 2 B 7.759783505 7.817275946
32 ABB 3 B 7.356405667 7.515982167
33 BAB 1 B 8.36886463 8.379766221
33 BAB 3 B 8.216466676 8.276196466
34 BBA 1 B 8.258608715 8.287758895
34 BBA 2 B 7.998750632 8.069192434
35 BAB 1 B 7.574446055 7.644724111
35 BAB 3 B 7.149663272 7.258526933
36 ABB 2 B 8.2490222 8.258958847
36 ABB 3 B 7.327066091 7.5359057
37 BAB 1 B 8.139238284 8.195958196
37 BAB 3 B 7.634118159 7.675717682
38 BBA 1 B 8.444112046 8.468275809
38 BBA 2 B 8.725257703 8.835573174
39 ABB 2 B 7.265019149 7.428235744
39 ABB 3 B 7.08341721 7.144682596
40 BBA 1 B 8.089394434 8.723463832
40 BBA 2 B 8.21551415 8.256136432
42 ABB 2 B 7.876135868 8.032802077
42 ABB 3 B 7.828881103 7.976931205
44 ABB 2 B 8.410692915 8.428963108
44 ABB 3 B 8.589966402 8.605777165
45 BBA 1 B 8.525890857 8.539325065
45 BBA 2 B 8.458267051 8.472018404
46 ABB 2 B 7.296383438 7.439829543
46 ABB 3 B 7.502625684 8.280353272
47 BBA 1 B 8.195452589 8.244132241
47 BBA 2 B 8.349083368 8.362756337
48 BAB 1 B 8.176094027 8.20568553
48 BAB 3 B 7.815589152 7.870187719
49 BAB 1 B 8.186195741 8.24090015
49 BAB 3 B 8.046571451 8.140032993
51 ABB 2 B 9.073040287 9.097873779
51 ABB 3 B 9.039707436 9.101568489
52 BBA 1 B 7.856079031 7.917417221
52 BBA 2 B 8.921983405 8.936348427
53 ABB 2 B 8.04764098 8.076713878
53 ABB 3 B 8.439900183 8.497614983
54 BAB 1 B 8.082637012 8.110054083
54 BAB 3 B 8.632410093 8.658085799
55 BBA 1 B 8.226321237 8.316787172
55 BBA 2 B 8.105905766 8.171993299
56 BAB 1 B 8.215087328 8.369842175
56 BAB 3 B 7.985378156 8.083406692
57 ABB 2 B 7.86516185 7.899924985
57 ABB 3 B 7.706186822 7.809824158
59 BAB 1 B 7.77147021 7.842954906
59 BAB 3 B 7.423932557 7.452101942


Any idea?

Thanks
John
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-05-01 18:21
(3983 d 17:33 ago)

@ jag009
Posting: # 10517
Views: 21,823
 

 Second opinion (PHX 6.3)

Hi John,

❝ For AUCt

❝ Ratio: 94.7377; 90% CI: 87.4898-102.586


PHX tells me:
Ratio: 94.7377; 90% CI: 87.4898–102.586 [image]
Warning 11091: Newton's algorithm converged with modified Hessian. Output is suspect.
Model may be over-specified. A simpler model could be tried.


PHX after 6 iterations:

Final variance parameter estimates:
           lambda(1,1)_11     0.44268399
           lambda(1,2)_11     0.42362946
           lambda(2,2)_11     0.16237525
Var(Period*Formulation*Subject)_21    0.053010265
Var(Period*Formulation*Subject)_22    0.061648310


❝ However, Proc Mixed failed to compute the above for ln AUCi


Contrary to SAS PHX ‘succeeded’ for AUCi as well …

Ratio: 95.2967; 90% CI: 88.4014–102.730

… but throws the same warning as above.

❝ The last output from Proc Mixed on ln AUCi was:

Covariance Parameter Values At Last Iteration

Cov Parm   Subject   Group          Estimate

FA(1,1)    subject                  0.3722

FA(2,1)    subject                  0.3975

FA(2,2)    subject                  0.2362

Residual   subject formulation Ref  0.06248

Residual   subject formulation Test 0.01460


PHX after 6 iterations:

Final variance parameter estimates:
           lambda(1,1)_11     0.37216022
           lambda(1,2)_11     0.39747315
           lambda(2,2)_11     0.15492894
Var(Period*Formulation*Subject)_21    0.0624828
Var(Period*Formulation*Subject)_22    0.0463781

Except for the subject-by-formulation interaction and s²wT quite similar. We have seen with other data sets that SAS and PHX disagree here. Can you post your variances for AUCt as well? I bet we will see differences.

❝ Any idea?


Nope. Let’s wait for the SAS-guru Detlew. :-D
IMHO, since a partial replicate according to FDA’s model is always (!) over-specified there is no guarantee that the LME-engine will converge. Don’t blame SAS and PHX; they warn us… Stupid design. If you want to have only three periods maybe it is better to run a fully replicated design (TRT|RTR) in the future.


P.S.: You are not alone. Last week a colleague posted at Pharsight’s Extranet an example where a replicate design failed to converge for Cmax (but not for AUCt and AUC). Pharsight suggested to change the variance structure to Heterogeneous Compound Symmetry (instead of FDA’s Banded No-Diagonal Factor Analytic [f=2]). In my experience this rarely helps…
BTW, does anybody know the rationale behind FDA’s partial replicate? Higher precision of the estimate of CVwR (see this post and followings)?

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2013-05-01 19:16
(3983 d 16:38 ago)

@ Helmut
Posting: # 10518
Views: 21,524
 

 Second opinion (PHX 6.3)

Thank you Helmut!

Detlew, need help! :-)

Here is the covariance output from SAS on ln AUCt

Covariance Parameter Estimates
Cov Parm  Subject       Group         Estimate
FA(1,1)   subject                     0.4427
FA(2,1)   subject                     0.4236
FA(2,2)   subject                     0.2481
Residual  subject   formulation Ref   0.05301
Residual  subject   formulation Test  0.02648


Question, what does the residual "formulation Test" represent? Is it the residual attributed to both test and ref, while residual "formulation ref" is attributed to the ref (since it was given 2x)? which one would one use to compute the 90% geometric CI then?

❝ IMHO, since a partial replicate according to FDA’s model is always (!) overspecified there is no guarantee that the LME-engines will converge. Don’t blame SAS and PHX; they warn us… Stupid design. If you want to have only three periods maybe it is better to run a fully replicated design (TRT|RTR) in the future.


Any hint on the stat approach?

Thanks
John
ElMaestro
★★★

Denmark,
2013-05-01 20:48
(3983 d 15:06 ago)

@ jag009
Posting: # 10519
Views: 21,391
 

 Second opinion (PHX 6.3)

Hi John,

❝ Question, what does the residual "formulation Test" represent? Is it the residual attributed to both test and ref, while residual "formulation ref" is attributed to the ref (since it was given 2x)? which one would one use to compute the 90% geometric CI then?


Can you ask SAS to spit out the covariance matrix? I mean the one corresponding to ZGZT+R? Then I think you can definitely interpret the variabilities in the context of the model specification.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-05-01 21:16
(3983 d 14:38 ago)

@ jag009
Posting: # 10520
Views: 21,906
 

 In praise of a full replicate

Hi John!

❝ Here is the covariance output from SAS on ln AUCt


Covariance Parameter Estimates

Cov Parm  Subject       Group         Estimate

FA(1,1)   subject                     0.4427

FA(2,1)   subject                     0.4236

FA(2,2)   subject                     0.2481

Residual  subject   formulation Ref   0.05301

Residual  subject   formulation Test  0.02648


OK, similar again (except for the usual suspects).

❝ Question, what does the residual "formulation Test" represent?


Nothing. Only a desperate attempt of the algo to estimate something based on too little information. Think about a parallel design. Intra-subject variability is there, but we cannot extract it (only the total/pooled). We need a cross-over. Same here. s²wT exists, but we have only one observation of T / subject.

❝ Is it the residual attributed to both test and ref, …


Cough. :smoke:

❝ … while residual "formulation ref" is attributed to the ref (since it was given 2x)?


Correct.

❝ which one would one use to compute the 90% geometric CI then?


None of them directly. PHX spits out a standard error of the difference T–R of 0.0474839 (too lazy to dig out the reference on how it is composed from the other variances).

❝ ❝ IMHO, since a partial replicate according to FDA’s model is always (!) overspecified there is no guarantee that the LME-engines will converge. :blahblah:


❝ Any hint on the stat approach?


No idea.

I took the opportunity and massaged you data. I kept all subjects in sequence RTR, and recoded a little bit. Of course I had to change some Rs to Ts… I got:
Subject Sequence Period Treatment lAUCt lAUCi
1 RTR 1 R 7.918112873 7.956429845
1 RTR 2 T 7.392949199 7.966255738
1 RTR 3 R 8.165388938 8.236181092
2 RTR 1 R 7.648429922 7.840311803
2 RTR 2 T 7.486969192 7.642465619
2 RTR 3 R 7.767760063 7.999909090
3 RTR 1 R 7.845094920 7.903718188
3 RTR 2 T 7.795040591 7.900675886
3 RTR 3 R 7.004410917 7.201138817
4 RTR 1 R 8.273957131 8.299682496
4 RTR 2 T 7.704527971 7.810454463
4 RTR 3 R 7.893447812 7.922782677
5 RTR 1 R 7.873074242 7.993014477
5 RTR 2 T 7.743414530 7.813019037
5 RTR 3 R 7.928661129 7.973013452
6 RTR 1 R 8.953187117 9.070644877
6 RTR 2 T 8.862899236 8.887092743
6 RTR 3 R 8.620163052 8.635798822
7 RTR 1 R 7.733732169 7.925830821
7 RTR 2 T 7.474575339 7.520470843
7 RTR 3 R 7.883843938 7.915115941
8 RTR 1 R 7.040298183 7.138905094
8 RTR 2 T 6.649783454 6.786800469
8 RTR 3 R 6.455060189 6.919911097
9 RTR 1 R 8.246138252 8.302636666
9 RTR 2 T 7.900081068 7.917934089
9 RTR 3 R 8.369926761 8.407009731
10 RTR 1 R 8.854970576 8.867276941
10 RTR 2 T 8.498478152 8.537308319
10 RTR 3 R 8.985634977 9.003601402
11 TRT 1 T 8.178239438 8.255348312
11 TRT 2 R 8.286612805 8.339918637
11 TRT 3 T 7.687297563 7.764432727
12 RTR 1 R 8.240486403 8.305583617
12 RTR 2 T 7.854077711 8.056187995
12 RTR 3 R 7.800706552 7.826649818
14 RTR 1 R 7.553067914 7.897693791
14 RTR 2 T 6.834931866 7.311990298
14 RTR 3 R 7.105642559 7.345723072
15 TRT 1 T 8.238080695 8.313684606
15 TRT 2 R 8.446723945 8.453986902
15 TRT 3 T 8.004066923 8.015037211
17 TRT 1 T 7.954169654 8.081352274
17 TRT 2 R 6.918289598 8.123267843
17 TRT 3 T 7.238486783 7.484658489
18 RTR 1 R 8.009085870 8.197899082
18 RTR 2 T 8.813960369 8.878784152
18 RTR 3 R 7.761056370 7.935726911
19 TRT 1 T 7.936215453 8.005634299
19 TRT 2 R 8.102567049 8.141680789
19 TRT 3 T 7.693701307 7.710909115
20 RTR 1 R 8.263957891 8.294051390
20 RTR 2 T 7.585678156 7.624085152
20 RTR 3 R 8.174826537 8.190936874
21 TRT 1 T 8.120727864 8.167847324
21 TRT 2 R 7.934564680 8.013054162
21 TRT 3 T 7.882511503 7.908029742
23 RTR 1 R 7.358441651 7.473058861
23 RTR 2 T 7.252320394 7.329537863
23 RTR 3 R 7.488031030 7.535879021
25 RTR 1 R 7.602494691 7.697937968
25 RTR 2 T 7.460390761 7.612326456
25 RTR 3 R 7.387379977 7.469852557
26 TRT 1 T 8.329193303 8.344287565
26 TRT 2 R 8.037593588 8.060771153
26 TRT 3 T 7.519507463 7.544075461
27 TRT 1 T 7.842873653 7.906903707
27 TRT 2 R 7.905330655 7.935414562
27 TRT 3 T 8.082943041 8.290801370
28 TRT 1 T 7.738586593 7.774590185
28 TRT 2 R 7.859365663 7.889316764
28 TRT 3 T 7.974813947 8.117347067
29 TRT 1 T 7.952082082 8.030115018
29 TRT 2 R 7.888838125 8.040802810
29 TRT 3 T 7.779364076 7.821842385
30 RTR 1 R 6.982335711 7.543216142
30 RTR 2 T 6.969401496 7.626733448
30 RTR 3 R 6.697153994 6.829441243
31 TRT 1 T 7.747856304 7.809290887
31 TRT 2 R 7.560504854 7.797942664
31 TRT 3 T 7.375110404 7.425354311
32 TRT 1 T 7.854046269 7.909554125
32 TRT 2 R 7.759783505 7.817275946
32 TRT 3 T 7.356405667 7.515982167
33 RTR 1 R 8.368864630 8.379766221
33 RTR 2 T 8.180100682 8.240037917
33 RTR 3 R 8.216466676 8.276196466
34 TRT 1 T 8.258608715 8.287758895
34 TRT 2 R 7.998750632 8.069192434
34 TRT 3 T 8.498685373 8.507850080
35 RTR 1 R 7.574446055 7.644724111
35 RTR 2 T 7.689511601 7.753216597
35 RTR 3 R 7.149663272 7.258526933
36 TRT 1 T 7.697065912 7.731297188
36 TRT 2 R 8.249022200 8.258958847
36 TRT 3 T 7.327066091 7.535905700
37 RTR 1 R 8.139238284 8.195958196
37 RTR 2 T 7.743985836 7.768691851
37 RTR 3 R 7.634118159 7.675717682
38 TRT 1 T 8.444112046 8.468275809
38 TRT 2 R 8.725257703 8.835573174
38 TRT 3 T 8.860378559 8.90071176
39 TRT 1 T 7.649143902 7.760258994
39 TRT 2 R 7.265019149 7.428235744
39 TRT 3 T 7.083417210 7.144682596
40 TRT 1 T 8.089394434 8.723463832
40 TRT 2 R 8.215514150 8.256136432
40 TRT 3 T 8.178085044 8.269703924
42 TRT 1 T 7.699680257 8.477479957
42 TRT 2 R 7.876135868 8.032802077
42 TRT 3 T 7.828881103 7.976931205
44 TRT 1 T 8.469052606 8.546889654
44 TRT 2 R 8.410692915 8.428963108
44 TRT 3 T 8.589966402 8.605777165
45 TRT 1 T 8.525890857 8.539325065
45 TRT 2 R 8.458267051 8.472018404
45 TRT 3 T 8.734765894 8.773995092
46 TRT 1 T 7.438072894 7.502691878
46 TRT 2 R 7.296383438 7.439829543
46 TRT 3 T 7.502625684 8.280353272
47 TRT 1 T 8.195452589 8.244132241
47 TRT 2 R 8.349083368 8.362756337
47 TRT 3 T 7.702201940 7.870368371
48 RTR 1 R 8.176094027 8.205685530
48 RTR 2 T 8.326805753 8.371119472
48 RTR 3 R 7.815589152 7.870187719
49 RTR 1 R 8.186195741 8.240900150
49 RTR 2 T 8.236498120 8.251025744
49 RTR 3 R 8.046571451 8.140032993
51 TRT 1 T 8.890424460 8.926528534
51 TRT 2 R 9.073040287 9.097873779
51 TRT 3 T 9.039707436 9.101568489
52 TRT 1 T 7.856079031 7.917417221
52 TRT 2 R 8.921983405 8.936348427
52 TRT 3 T 8.319488803 8.645876426
53 TRT 1 T 8.254585577 8.267455890
53 TRT 2 R 8.047640980 8.076713878
53 TRT 3 T 8.439900183 8.497614983
54 RTR 1 R 8.082637012 8.110054083
54 RTR 2 T 8.774547697 8.830803941
54 RTR 3 R 8.632410093 8.658085799
55 TRT 1 T 8.226321237 8.316787172
55 TRT 2 R 8.105905766 8.171993299
55 TRT 3 T 7.695208954 7.756027950
56 RTR 1 R 8.215087328 8.369842175
56 RTR 2 T 8.088395381 8.173364805
56 RTR 3 R 7.985378156 8.083406692
57 TRT 1 T 7.721135381 7.754655632
57 TRT 2 R 7.865161850 7.899924985
57 TRT 3 T 7.706186822 7.809824158
59 RTR 1 R 7.771470210 7.842954906
59 RTR 2 T 7.624704432 7.649123891
59 RTR 3 R 7.423932557 7.452101942

Now I had a fully replicated design (TRT|RTR). No warnings, no problems with convergence. Unless some genius comes up with a model for the partial replicate which shows no problems with convergence I would avoid it in the future. Alternative: Send your data to [email protected]. :-D

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2013-05-02 14:00
(3982 d 21:54 ago)

@ jag009
Posting: # 10522
Views: 21,575
 

 Third opinion

Hi John,

❝ Detlew, need help! :-)


Here I am! But don't know if I can help anyway. The whole story "Use Proc MIXED code for Partial replicate design" is mysterious to me.

❝ Here is the covariance output from SAS on ln AUCt


Covariance Parameter Estimates

Cov Parm  Subject       Group         Estimate

FA(1,1)   subject                     0.4427

FA(2,1)   subject                     0.4236

FA(2,2)   subject                     0.2481

Residual  subject   formulation Ref   0.05301

Residual  subject   formulation Test  0.02648


❝ Question, what does the residual "formulation Test" represent? Is it the residual attributed to both test and ref


No.
As Helmut already pointed out: an ambiguous attempt of the REML algo to obtain the within-subject variance of the Test formulation. But IMHO the model is over-specified (s2D + s2wT not separable, see below) and therefore there is no guarantee that the value obtained is reasonable.

❝ while residual "formulation ref" is attributed to the ref (since it was given 2x)?


Correct. Unambiguously identifiable.

❝ which one would one use to compute the 90% geometric CI then?


Not clear to me what a 90% geometric CI is :confused:.

The difference µT-µR has as standard error associated with it for the partial replicate design

sd = sqrt((s2D + s2wT + s2wR/2)*sum(1/ni)/seq^2)
where s2D is the variance of the subject-by-formulation interaction, ni are the number of subjects in the sequence groups, seq is the number of sequences.

s2D can be obtained from the G-matrix according to
s2D = g11+g22-2*g12
(see for more details this post).

Since the model seems over-specified try to use a simple model, f.i. neglect s2D which in turn results in a CS variance-covariance structure for the random part. Sometimes this helps.
See also this thread for another even simpler model specification.

BTW: @Helmut, asking the FDA seems a very good idea!

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-05-02 16:41
(3982 d 19:13 ago)

@ d_labes
Posting: # 10524
Views: 21,696
 

 Compound Symmetry

Hi Detlew & John,

❝ […] try to use a simple model, f.i. neglect s2D which in turn results in a CS variance-covariance structure for the random part. Sometimes this helps.


Nice – no warnings in PHX any more (only two iterations with the default settings).

log AUCt

PE: 94.7377 (90% CI: 87.4898 – 102.586) [image]
Final variance parameter estimates:
csDiag_11                           0.00843516
csBlock_11                          0.187534
Var(Period*Formulation*Subject)_21  0.0530102
Var(Period*Formulation*Subject)_22  0.0715068

log AUCi

PE: 95.2967 (90% CI: 88.4014 – 102.730) [image]
Final variance parameter estimates:
csDiag_11                          -0.00942047
csBlock_11                          0.147924
Var(Period*Formulation*Subject)_21  0.0624828
Var(Period*Formulation*Subject)_22  0.0898627


What do you SASians get?

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2013-05-02 18:26
(3982 d 17:27 ago)

@ Helmut
Posting: # 10529
Views: 22,010
 

 Compound Symmetry - SASian (1)

Dear Helmut,

❝ What do you SASians get?


Aye, aye Sir, here I am :-D.

log AUCt
     90% CI
     PE=94.74%  [ 87.49% ... 102.59%] [image]

    Covariance Parameter Estimates

  Cov Parm     Subject    Group    Estimate

  Variance     subject              0.008436
  CS           subject              0.1875
  Residual     subject    tmt A     0.07150
  Residual     subject    tmt B     0.05300

NOTE: Convergence criteria met.


log AUCinf
     90% CI
     PE=95.30%  [ 88.32% ... 102.82%]  [image]

    Covariance Parameter Estimates

  Cov Parm     Subject    Group    Estimate

  Variance     subject                    0 (!)
  CS           subject              0.1441
  Residual     subject    tmt A     0.07465
  Residual     subject    tmt B     0.06088

NOTE: Convergence criteria met.
NOTE: Estimated G matrix is not positive definite.
NOTE: Asymptotic variance matrix of covariance parameter estimates has been found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=SATTERTH.


Please note the 'Variance' parameter in case of AUCinf.
CS Covariance matrix parameterized in Proc MIXED as:
( CS+var  CS
  CS      CS+var)


See this post to notice that we really need 'Variance'=0 in our model (Type CS is only an approximation to that end, hoping the 'Variance' parameter is fitted with a value near zero).
So we eventually have the optimizer to tell that for AUCt. Couldn't figure out in a hurry at moment how to do that.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-05-03 18:07
(3981 d 17:46 ago)

@ d_labes
Posting: # 10535
Views: 21,620
 

 Variance=0

Dear Detlew!

❝ Please note the 'Variance' parameter in case of AUCinf.


… and the negative one obtained in PHX. Seems that SAS’ RLME-engine forces negative values to zero.

❝ See this post to notice that we really need 'Variance'=0 in our model (Type CS is only an approximation to that end, hoping the 'Variance' parameter is fitted with a value near zero).


Ooh – that one. :-D

❝ So we eventually have the optimizer to tell that for AUCt.


I’m not very optimistic whether this is possible in PHX at all; I will ask Pharsight. In PHX for linear mixed effects models the initial estimates are derived by the method of moments:
log AUCt

Starting estimates of variance parameters:
csDiag_11                           0.00881650
csBlock_11                          0.188267
Var(Period*Formulation*Subject)_21  0.0519883
Var(Period*Formulation*Subject)_22  0.0703479

log AUCi

Starting estimates of variance parameters:
csDiag_11                          -0.00961080
csBlock_11                          0.148549
Var(Period*Formulation*Subject)_21  0.0620715
Var(Period*Formulation*Subject)_22  0.0894799


Alternatively one can state initial variances. If I set csDiag_11 (PHX’ terminology) to zero (whilst keeping the others), the optimizer iterates happily around (four iterations instead of two) – only to end up with the same final estimates…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,639 registered users;
82 visitors (0 registered, 82 guests [including 8 identified bots]).
Forum time: 10:54 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5