jag009
★★★

NJ,
2013-04-22 16:39
(2985 d 07:51 ago)

Posting: # 10458
Views: 21,782

## FDA's HVD SAS Code from Progesterone Guidance [RSABE / ABEL]

Hi everyone,

I am getting some strange results using the FDA Progesterone Scaled Average Bioequivalence SAS Code → The portion to compute the Unscaled Average 90% C.I.

PROC MIXED data=pk; CLASSES SEQ SUBJ PER TRT; MODEL LAUCT = SEQ PER TRT/ DDFM=SATTERTH; RANDOM TRT/TYPE=FA0(2) SUB=SUBJ G; REPEATED/GRP=TRT SUB=SUJ; ESTIMATE 'T vs. R' TRT 1 -1/CL ALPHA=0.1; ods output Estimates=unsc1; title1 'unscaled BE 90% CI - guidance version'; title2 'AUCt'; run; data unsc1; set unsc1; unscabe_lower=exp(lower); unscabe_upper=exp(upper); run;

Should the Test/Reference Ratio from the Scaled Average BE computation be similar (if not the same) as the ratio computed from the above Unscaled Average BE code?

Thanks
John

Edit: Category changed. [Helmut]
Helmut
★★★

Vienna, Austria,
2013-04-22 17:47
(2985 d 06:43 ago)

@ jag009
Posting: # 10460
Views: 18,912

## Proc MIXED vs. Proc GLM

Hi John,

» Should the Test/Reference Ratio from the Scaled Average BE computation be similar (if not the same) as the ratio computed from the above Unscaled Average BE code?

More details, pleeze! Define similar. Note that FDA’s ABE is evaluated by a Proc MIXED and the partial replicate by Proc GLM. You will see differences for incomplete data-sets (GLM drops subjects, MIXED keeps them).

Even for a fully replicated design (but imbalanced and incomplete like EMA’s data set I) SAS’ merge will drop subjects. In Phoenix I get PEs of 1.1546132 (RSABE) and 1.1565765 (ABE).

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2013-04-22 20:33
(2985 d 03:57 ago)

@ Helmut
Posting: # 10464
Views: 18,720

## Proc MIXED vs. Proc GLM

Hi Helmut,

Can you try this and let me know? Yes there are missing datapoints. The other 2 parameters worked out fine. I ran data from another study and everything worked as well. Only this dataset is giving me the finger. The difference is the missing values as other datasets had values for all subjects. Maybe the missing data is messing things up?

Data columns are Subject Period Sequence Formulation PK-parameter

My SAS SCABE for this parameter is

Theta    Bound y    (S2wr)    (sWR)   (T/R) Pt Est  95% Upper Crit 0.79669   -0.22198  0.41135  0.64137    89.8994     -0.18423

I ran Unscaled Avg BE (Progesterone guidance SAS code)

T/R Ratio     90% CI 101.021    80.7739 - 126.343

Sequence 1=ABB, 2=BAB, 3=BBA

1 3 BBA A 31.432 2 1 ABB A 23.746 4 1 ABB A 41.258 5 3 BBA A 24.887 6 2 BAB A 237.003 7 3 BBA A 35.132 8 1 ABB A 22.399 9 2 BAB A 77.187 10 2 BAB A 180.386 11 3 BBA A 40.623 12 1 ABB A 15.382 14 2 BAB A - 15 3 BBA A 75.768 17 1 ABB A 15.219 18 2 BAB A 136.554 19 1 ABB A 53.966 20 2 BAB A 18.509 21 3 BBA A 24.439 23 2 BAB A 23.710 25 2 BAB A 16.264 26 3 BBA A 41.160 27 1 ABB A 63.364 28 3 BBA A 46.344 29 1 ABB A 66.059 30 2 BAB A 2.539 31 3 BBA A - 32 1 ABB A 34.002 33 2 BAB A 61.070 34 3 BBA A 149.674 35 2 BAB A 26.440 36 1 ABB A 54.400 37 2 BAB A 39.185 38 3 BBA A 201.983 39 1 ABB A 17.831 40 3 BBA A 85.293 1 1 BBA B 40.705 1 2 BBA B 68.864 2 2 ABB B 13.020 2 3 ABB B 36.121 4 2 ABB B 102.646 4 3 ABB B 54.459 5 1 BBA B 9.396 5 2 BBA B 22.999 6 1 BAB B 309.754 6 3 BAB B 163.058 7 1 BBA B 41.710 7 2 BBA B 44.472 8 2 ABB B 2.915 8 3 ABB B - 9 1 BAB B 88.910 9 3 BAB B 109.897 10 1 BAB B 373.070 10 3 BAB B 529.088 11 1 BBA B 74.469 11 2 BBA B 80.379 12 2 ABB B 26.124 12 3 ABB B 18.094 14 1 BAB B 2.280 14 3 BAB B - 15 1 BBA B 99.409 15 2 BBA B 134.023 17 2 ABB B - 17 3 ABB B 3.694 18 1 BAB B 49.218 18 3 BAB B 42.806 19 2 ABB B 61.764 19 3 ABB B 47.537 20 1 BAB B 74.644 20 3 BAB B 36.181 21 1 BBA B 47.737 21 2 BBA B 30.634 23 1 BAB B 11.766 23 3 BAB B 14.121 25 1 BAB B 29.943 25 3 BAB B 20.244 26 1 BBA B 109.626 26 2 BBA B 80.252 27 2 ABB B 33.349 27 3 ABB B 124.874 28 1 BBA B 29.083 28 2 BBA B 23.024 29 2 ABB B 34.671 29 3 ABB B 23.049 30 1 BAB B 2.782 30 3 BAB B 2.249 31 1 BBA B 57.630 31 2 BBA B - 32 2 ABB B 24.816 32 3 ABB B 39.266 33 1 BAB B 68.419 33 3 BAB B 85.845 34 1 BBA B 51.760 34 2 BBA B 54.845 35 1 BAB B 30.751 35 3 BAB B 2.921 36 2 ABB B 70.271 36 3 ABB B 38.989 37 1 BAB B 114.890 37 3 BAB B 54.372 38 1 BBA B 98.310 38 2 BBA B 147.839 39 2 ABB B 534.900 39 3 ABB B 16.465 40 1 BBA B 176.916 40 2 BBA B 105.915

Thanks
John
Helmut
★★★

Vienna, Austria,
2013-04-22 22:10
(2985 d 02:20 ago)

@ jag009
Posting: # 10466
Views: 18,563

## SAS vs. PHX

Hi John,

» My SAS SCABE for this parameter is
» Theta    Bound y  (S2wr)    (sWR)   (T/R) Pt Est  95% Upper Crit
» 0.79669 -0.22198  0.41135  0.64137    89.8994     -0.18423

No need to post Theta (a constant in RSABE).

My PHX6.3 code gives for RSABE …
boundy     S²wr     Swr       PE       95% upper -0.221998  0.41138  0.64139  89.8990  -0.18424336

» I ran Unscaled Avg BE (Progesterone guidance SAS code)
» T/R Ratio     90% CI
» 101.021    80.7739 - 126.343

… and for ABE
  PE          90% CI 104.782  69.7855 – 157.328

But: PHX’ LME kicked my ass with “Warning 11094: Negative final variance component. Consider omitting this VC structure.” Nice. Old story with partial replicates – overspecified model since T is not repeated. We have seen in the past that SAS and PHX give different results in such a case. BTW, did you get “Convergence criteria met but final hessian is not positive definite.” or somefink similar in Sas?

If I go with the ABE-module I got:
ERROR 11070:  Error in Satterthwaite DF. Try using Residual DF option.
… which I tried, only to get:
ERROR 11070:  Error in Satterthwaite DF. Try using Residual DF option.
That’s funny! Same if I exclude subjects #14 and #31 (no results for T). What the heck?

Running PHX’ PBE/IBE module I first obtained …
Warning 11121: Subject 31 had incomplete design and was discarded. Warning 11121: Subject 14 had incomplete design and was discarded. Warning 11121: Subject 8 had incomplete design and was discarded. Warning 11121: Subject 17 had incomplete design and was discarded.

… and a PE which is exactly like in RSABE 89.899043.

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Shuanghe
★★

Spain,
2013-04-23 11:45
(2984 d 12:44 ago)

(edited by Shuanghe on 2013-04-23 12:27)
@ Helmut
Posting: # 10471
Views: 18,549

## SAS vs. PHX

Hi Helmut,

» My PHX6.3 code gives for RSABE …
» boundy     S²wr     Swr       PE       95% upper
» -0.221998  0.41138  0.64139  89.8990  -0.18424336

Just out of curiosity I tried my SAS macro and it only gives Swr, PE, 95% upper. They are the same as yours, which is slightly different from John's.

» » I ran Unscaled Avg BE (Progesterone guidance SAS code)
» » T/R Ratio     90% CI
» » 101.021    80.7739 - 126.343
»
» … and for ABE
»   PE          90% CI» 104.782  69.7855 – 157.328

Now, my average BE gives:
PE: 1.01021, same as John's
90% CI: 80.7733 - 126.3450, different from both of yours.
Weird.

» But: PHX’ LME kicked my ass with “Warning 11094: Negative final variance component. Consider omitting this VC structure.” Nice. Old story with partial replicates – overspecified model since T is not repeated. We have seen in the past that SAS and PHX give different results in such a case. BTW, did you get “Convergence criteria met but final hessian is not positive definite.” or somefink similar in Sas?

SAS did give me warning of " Convergence criteria met but final hessian is not positive definite".

» That’s funny! Same if I exclude subjects #14 and #31 (no results for T). What the heck?

after delete subject 14 and 31, same result for SABE but for ABE:
PE: 98.04322%
90% CI: 78.61817 - 122.26784.
Man, I never doubted the macro before. should I worry about it now?
Shall we compare the result from EMA's data set?

Edit:
Just tried EMA's 3-period data set, using "logdata" directly from the dataset.
SABE gives:
Swr=0.11397 PE=1.02264 95% upper= -0.00397289

ABE gives the same result as in the EMA's Q&A. 102.264 (97.05-107.76)

delete subject 5 and 8 (don't ask why those subjects)
SABE gives: Swr=0.11803 PE=1.01341 95% upper= -0.00510258

ABE gives:
PE=101.218 90% CI= 95.7687 - 106.978
 
Can anyone check it?

All the best,
Shuanghe
d_labes
★★★

Berlin, Germany,
2013-04-23 16:50
(2984 d 07:39 ago)

@ Shuanghe
Posting: # 10475
Views: 18,428

## SAS vs. SAS

Hi Shuanghe,

» Now, my average BE gives:
» PE: 1.01021, same as John's
» 90% CI: 80.7733 - 126.3450, different from both of yours.
» Weird.

don't worry. My results (using Johns code for ABE as given above at start of the thread) under SAS9.2 are:
  point est.  90% confidence interval   101.0213%     80.7733   126.3450   s2wR = 0.4210 -> CVwR = 72.35%
Deleting the 4 subjects with missings (same happens in the intra-subject contrast calculations) gives
  point est.  90% confidence interval   90.5032%     73.5627   111.3449   s2wR = 0.4001 -> CVwR = 70.14%
very similar to the numbers used for the RSABE criterion .

But I don't believe in this numbers anyway. I think the optimizer stops here arbitrarily as almost ever for a partial replicate design in which the intra-individual variability for T is not identifiable.
Any modification to the code, f.i. fitting a model with no subject-by-treatment interaction via the CS covariance structure crashes with infinite likelihood and an estimate of s2wT=0!

I always wondered why the FDA insists on the Proc MIXED code, especially for that design.
On the other hand in the context of RSABE linearized criterion the point estimator and its 90% CI are calculated via intra-subject contrast T-R.
Why not use these results for ABE also .

To increase the confusion here the results of the mighty oracle EMA code (same Proc GLM as for a 2x2 crossover for the PE and CI, s2wR from analysis of data for R (B) only):
  point est.  90% confidence interval   98.4476%     78.6492   123.2298   s2wR = 0.39824522 -> CVwR = 69.94%

So much numbers to choose between .

Regards,

Detlew
Helmut
★★★

Vienna, Austria,
2019-11-12 13:01
(590 d 10:29 ago)

@ d_labes
Posting: # 20788
Views: 4,774

## SAS vs. SAS vs. Phoenix

Hi to all victims of the partial replicate with incomplete data,

following extensive off-list discussions with PharmCat (who is working on a Julia-package; see here and there) I checked again what we got with the FDA’s covariance structure, Satterthwaite’s df (PE, 90% CI).
Not the slightest idea how I arrived at the results given above. Sorry for the confusion caused.
• John’s first dataset.
• John (SAS, v?)
101.021    80.7739 – 126.343
• Shuanghe (SAS, v?)
101.021    80.7733 – 126.3450
• Detlew (SAS, v9.2)
101.0213   80.7733 – 126.3450
• Myself (Phoenix v6.3 and v8.1)
No convergence with the default setup.
However, when I lowered the convergence criterion to 1E-11:
101.0213   80.7733 – 126.3450

• John’s second dataset (ln AUCi).
• John (SAS, v?)
No convergence.
• Myself (Phoenix v6.3 and v8.1)
Convergence with warning, tweaking the setup doesn’t help.
95.2967    88.4014 – 102.7298
Compound symmetry (as suggested by Detlew)
95.2967    88.4014 – 102.7298
• Detlew (SAS, v9.2, CS)
95.30      88.23   – 102.82
In all cases simplifying the covariance structure (in SAS-lingo FA0(1) instead of the FA0(2) in the guidance): Convergence without warning.

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2013-04-23 15:56
(2984 d 08:34 ago)

@ Helmut
Posting: # 10473
Views: 18,208

## SAS vs. PHX

Hi Helmut and shanghue,

» That’s funny! Same if I exclude subjects #14 and #31 (no results for T). What the heck?

Yes but Proc Mixed takes into account the missing data correct?

Great, I opened up a Pandora's Box...
Let me investigate further and let you guys know.

Calling for Detlew the SAS guru (heheh, one of the gurus to be politically correct) . He might have some idea on this...

Thanks
John
jag009
★★★

NJ,
2013-04-23 17:07
(2984 d 07:23 ago)

@ jag009
Posting: # 10476
Views: 18,852

## SAS Warning (Note) on Proc Mixed

Thanks guys (and girls),

Here are the notes from my SAS Log on Proc Mixed.
NOTE: 6 observations are not included because of missing values. NOTE: Convergence criteria met but final hessian is not positive definite. NOTE: Asymptotic variance matrix of covariance parameter estimates has been found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=SATTERTH.

I think this is problematic... One runs a dataset and SCABE criteria for Swr not met so he/she resorts to the ABE routine. Tada!

Thanks
John
jag009
★★★

NJ,
2013-04-30 21:09
(2977 d 03:21 ago)

@ jag009
Posting: # 10515
Views: 18,444

## Nut Job..

Hi everyone,

Continuing the sage... I now have full dataset and I included AUCt as well.

Once again using the ABE procedure with Proc Mixed from FDA progesterone guidance. I was fine with ln AUCt but did get a message in the SAS log:

NOTE: Convergence criteria met but final hessian is not positive definite.
NOTE: Asymptotic variance matrix of covariance parameter estimates has been found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=SATTERTH.

For AUCt
Ratio: 94.7377; 90% CI: 87.4898-102.586

However, Proc Mixed failed to compute the above for ln AUCi and gave me a warning message in SAS log:

WARNING: Did not converge.
WARNING: Output 'Estimates' was not created....

The last output from Proc Mixed on ln AUCi was:
Covariance Parameter Values At Last Iteration Cov Parm   Subject   Group          Estimate FA(1,1)    subject                  0.3722 FA(2,1)    subject                  0.3975 FA(2,2)    subject                  0.2362 Residual   subject formulation Ref  0.06248 Residual   subject formulation Test 0.01460

I used the same ABE SAS Code posted in the first message in this thread (and the codes are the same for AUCt and AUCi).

The Dataset:
Subject, Sequence, Period, Treatment, ln AUCt, ln AUCi.
Sequence: ABB = 1, BAB = 2, BBA = 3

1 BBA 3 A 7.392949199 7.966255738 2 ABB 1 A 7.486969192 7.642465619 3 BAB 2 A 7.795040591 7.900675886 4 ABB 1 A 7.704527971 7.810454463 5 BBA 3 A 7.74341453 7.813019037 6 BAB 2 A 8.862899236 8.887092743 7 BBA 3 A 7.474575339 7.520470843 8 ABB 1 A 6.649783454 6.786800469 9 BAB 2 A 7.900081068 7.917934089 10 BAB 2 A 8.498478152 8.537308319 11 BBA 3 A 7.687297563 7.764432727 12 ABB 1 A 7.854077711 8.056187995 14 BAB 2 A 6.834931866 7.311990298 15 BBA 3 A 8.004066923 8.015037211 17 ABB 1 A 7.954169654 8.081352274 18 BAB 2 A 8.813960369 8.878784152 19 ABB 1 A 7.936215453 8.005634299 20 BAB 2 A 7.585678156 7.624085152 21 BBA 3 A 7.882511503 7.908029742 23 BAB 2 A 7.252320394 7.329537863 25 BAB 2 A 7.460390761 7.612326456 26 BBA 3 A 7.519507463 7.544075461 27 ABB 1 A 7.842873653 7.906903707 28 BBA 3 A 7.974813947 8.117347067 29 ABB 1 A 7.952082082 8.030115018 30 BAB 2 A 6.969401496 7.626733448 31 BBA 3 A 7.375110404 7.425354311 32 ABB 1 A 7.854046269 7.909554125 33 BAB 2 A 8.180100682 8.240037917 34 BBA 3 A 8.498685373 8.50785008 35 BAB 2 A 7.689511601 7.753216597 36 ABB 1 A 7.697065912 7.731297188 37 BAB 2 A 7.743985836 7.768691851 38 BBA 3 A 8.860378559 8.90071176 39 ABB 1 A 7.649143902 7.760258994 40 BBA 3 A 8.178085044 8.269703924 42 ABB 1 A 7.699680257 8.477479957 44 ABB 1 A 8.469052606 8.546889654 45 BBA 3 A 8.734765894 8.773995092 46 ABB 1 A 7.438072894 7.502691878 47 BBA 3 A 7.70220194 7.870368371 48 BAB 2 A 8.326805753 8.371119472 49 BAB 2 A 8.23649812 8.251025744 51 ABB 1 A 8.89042446 8.926528534 52 BBA 3 A 8.319488803 8.645876426 53 ABB 1 A 8.254585577 8.26745589 54 BAB 2 A 8.774547697 8.830803941 55 BBA 3 A 7.695208954 7.75602795 56 BAB 2 A 8.088395381 8.173364805 57 ABB 1 A 7.721135381 7.754655632 59 BAB 2 A 7.624704432 7.649123891 1 BBA 1 B 7.918112873 7.956429845 1 BBA 2 B 8.165388938 8.236181092 2 ABB 2 B 7.648429922 7.840311803 2 ABB 3 B 7.767760063 7.99990909 3 BAB 1 B 7.84509492 7.903718188 3 BAB 3 B 7.004410917 7.201138817 4 ABB 2 B 8.273957131 8.299682496 4 ABB 3 B 7.893447812 7.922782677 5 BBA 1 B 7.873074242 7.993014477 5 BBA 2 B 7.928661129 7.973013452 6 BAB 1 B 8.953187117 9.070644877 6 BAB 3 B 8.620163052 8.635798822 7 BBA 1 B 7.733732169 7.925830821 7 BBA 2 B 7.883843938 7.915115941 8 ABB 2 B 7.040298183 7.138905094 8 ABB 3 B 6.455060189 6.919911097 9 BAB 1 B 8.246138252 8.302636666 9 BAB 3 B 8.369926761 8.407009731 10 BAB 1 B 8.854970576 8.867276941 10 BAB 3 B 8.985634977 9.003601402 11 BBA 1 B 8.178239438 8.255348312 11 BBA 2 B 8.286612805 8.339918637 12 ABB 2 B 8.240486403 8.305583617 12 ABB 3 B 7.800706552 7.826649818 14 BAB 1 B 7.553067914 7.897693791 14 BAB 3 B 7.105642559 7.345723072 15 BBA 1 B 8.238080695 8.313684606 15 BBA 2 B 8.446723945 8.453986902 17 ABB 2 B 6.918289598 8.123267843 17 ABB 3 B 7.238486783 7.484658489 18 BAB 1 B 8.00908587 8.197899082 18 BAB 3 B 7.76105637 7.935726911 19 ABB 2 B 8.102567049 8.141680789 19 ABB 3 B 7.693701307 7.710909115 20 BAB 1 B 8.263957891 8.29405139 20 BAB 3 B 8.174826537 8.190936874 21 BBA 1 B 8.120727864 8.167847324 21 BBA 2 B 7.93456468 8.013054162 23 BAB 1 B 7.358441651 7.473058861 23 BAB 3 B 7.48803103 7.535879021 25 BAB 1 B 7.602494691 7.697937968 25 BAB 3 B 7.387379977 7.469852557 26 BBA 1 B 8.329193303 8.344287565 26 BBA 2 B 8.037593588 8.060771153 27 ABB 2 B 7.905330655 7.935414562 27 ABB 3 B 8.082943041 8.29080137 28 BBA 1 B 7.738586593 7.774590185 28 BBA 2 B 7.859365663 7.889316764 29 ABB 2 B 7.888838125 8.04080281 29 ABB 3 B 7.779364076 7.821842385 30 BAB 1 B 6.982335711 7.543216142 30 BAB 3 B 6.697153994 6.829441243 31 BBA 1 B 7.747856304 7.809290887 31 BBA 2 B 7.560504854 7.797942664 32 ABB 2 B 7.759783505 7.817275946 32 ABB 3 B 7.356405667 7.515982167 33 BAB 1 B 8.36886463 8.379766221 33 BAB 3 B 8.216466676 8.276196466 34 BBA 1 B 8.258608715 8.287758895 34 BBA 2 B 7.998750632 8.069192434 35 BAB 1 B 7.574446055 7.644724111 35 BAB 3 B 7.149663272 7.258526933 36 ABB 2 B 8.2490222 8.258958847 36 ABB 3 B 7.327066091 7.5359057 37 BAB 1 B 8.139238284 8.195958196 37 BAB 3 B 7.634118159 7.675717682 38 BBA 1 B 8.444112046 8.468275809 38 BBA 2 B 8.725257703 8.835573174 39 ABB 2 B 7.265019149 7.428235744 39 ABB 3 B 7.08341721 7.144682596 40 BBA 1 B 8.089394434 8.723463832 40 BBA 2 B 8.21551415 8.256136432 42 ABB 2 B 7.876135868 8.032802077 42 ABB 3 B 7.828881103 7.976931205 44 ABB 2 B 8.410692915 8.428963108 44 ABB 3 B 8.589966402 8.605777165 45 BBA 1 B 8.525890857 8.539325065 45 BBA 2 B 8.458267051 8.472018404 46 ABB 2 B 7.296383438 7.439829543 46 ABB 3 B 7.502625684 8.280353272 47 BBA 1 B 8.195452589 8.244132241 47 BBA 2 B 8.349083368 8.362756337 48 BAB 1 B 8.176094027 8.20568553 48 BAB 3 B 7.815589152 7.870187719 49 BAB 1 B 8.186195741 8.24090015 49 BAB 3 B 8.046571451 8.140032993 51 ABB 2 B 9.073040287 9.097873779 51 ABB 3 B 9.039707436 9.101568489 52 BBA 1 B 7.856079031 7.917417221 52 BBA 2 B 8.921983405 8.936348427 53 ABB 2 B 8.04764098 8.076713878 53 ABB 3 B 8.439900183 8.497614983 54 BAB 1 B 8.082637012 8.110054083 54 BAB 3 B 8.632410093 8.658085799 55 BBA 1 B 8.226321237 8.316787172 55 BBA 2 B 8.105905766 8.171993299 56 BAB 1 B 8.215087328 8.369842175 56 BAB 3 B 7.985378156 8.083406692 57 ABB 2 B 7.86516185 7.899924985 57 ABB 3 B 7.706186822 7.809824158 59 BAB 1 B 7.77147021 7.842954906 59 BAB 3 B 7.423932557 7.452101942

Any idea?

Thanks
John
Helmut
★★★

Vienna, Austria,
2013-05-01 16:21
(2976 d 08:09 ago)

@ jag009
Posting: # 10517
Views: 18,328

## Second opinion (PHX 6.3)

Hi John,

» For AUCt
» Ratio: 94.7377; 90% CI: 87.4898-102.586

PHX tells me:
Ratio: 94.7377; 90% CI: 87.4898–102.586
Warning 11091: Newton's algorithm converged with modified Hessian. Output is suspect. Model may be over-specified. A simpler model could be tried.

PHX after 6 iterations:

Final variance parameter estimates:            lambda(1,1)_11     0.44268399            lambda(1,2)_11     0.42362946            lambda(2,2)_11     0.16237525 Var(Period*Formulation*Subject)_21    0.053010265 Var(Period*Formulation*Subject)_22    0.061648310

» However, Proc Mixed failed to compute the above for ln AUCi

Contrary to SAS PHX ‘succeeded’ for AUCi as well …

Ratio: 95.2967; 90% CI: 88.4014–102.730

… but throws the same warning as above.

» The last output from Proc Mixed on ln AUCi was:
» Covariance Parameter Values At Last Iteration
» Cov Parm   Subject   Group          Estimate
» FA(1,1)    subject                  0.3722
» FA(2,1)    subject                  0.3975
» FA(2,2)    subject                  0.2362
» Residual   subject formulation Ref  0.06248
» Residual   subject formulation Test 0.01460

PHX after 6 iterations:

Final variance parameter estimates:            lambda(1,1)_11     0.37216022            lambda(1,2)_11     0.39747315            lambda(2,2)_11     0.15492894 Var(Period*Formulation*Subject)_21    0.0624828 Var(Period*Formulation*Subject)_22    0.0463781

Except for the subject-by-formulation interaction and s²wT quite similar. We have seen with other data sets that SAS and PHX disagree here. Can you post your variances for AUCt as well? I bet we will see differences.

» Any idea?

Nope. Let’s wait for the SAS-guru Detlew.
IMHO, since a partial replicate according to FDA’s model is always (!) over-specified there is no guarantee that the LME-engine will converge. Don’t blame SAS and PHX; they warn us… Stupid design. If you want to have only three periods maybe it is better to run a fully replicated design (TRT|RTR) in the future.

P.S.: You are not alone. Last week a colleague posted at Pharsight’s Extranet an example where a replicate design failed to converge for Cmax (but not for AUCt and AUC). Pharsight suggested to change the variance structure to Heterogeneous Compound Symmetry (instead of FDA’s Banded No-Diagonal Factor Analytic [f=2]). In my experience this rarely helps…
BTW, does anybody know the rationale behind FDA’s partial replicate? Higher precision of the estimate of CVwR (see this post and followings)?

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2013-05-01 17:16
(2976 d 07:14 ago)

@ Helmut
Posting: # 10518
Views: 18,141

## Second opinion (PHX 6.3)

Thank you Helmut!

Detlew, need help!

Here is the covariance output from SAS on ln AUCt

Covariance Parameter Estimates Cov Parm  Subject       Group         Estimate FA(1,1)   subject                     0.4427 FA(2,1)   subject                     0.4236 FA(2,2)   subject                     0.2481 Residual  subject   formulation Ref   0.05301 Residual  subject   formulation Test  0.02648

Question, what does the residual "formulation Test" represent? Is it the residual attributed to both test and ref, while residual "formulation ref" is attributed to the ref (since it was given 2x)? which one would one use to compute the 90% geometric CI then?

» IMHO, since a partial replicate according to FDA’s model is always (!) overspecified there is no guarantee that the LME-engines will converge. Don’t blame SAS and PHX; they warn us… Stupid design. If you want to have only three periods maybe it is better to run a fully replicated design (TRT|RTR) in the future.

Any hint on the stat approach?

Thanks
John
ElMaestro
★★★

Denmark,
2013-05-01 18:48
(2976 d 05:42 ago)

@ jag009
Posting: # 10519
Views: 18,064

## Second opinion (PHX 6.3)

Hi John,

» Question, what does the residual "formulation Test" represent? Is it the residual attributed to both test and ref, while residual "formulation ref" is attributed to the ref (since it was given 2x)? which one would one use to compute the 90% geometric CI then?

Can you ask SAS to spit out the covariance matrix? I mean the one corresponding to ZGZT+R? Then I think you can definitely interpret the variabilities in the context of the model specification.

Pass or fail!
ElMaestro
Helmut
★★★

Vienna, Austria,
2013-05-01 19:16
(2976 d 05:14 ago)

@ jag009
Posting: # 10520
Views: 18,443

## In praise of a full replicate

Hi John!

» Here is the covariance output from SAS on ln AUCt
»
» Covariance Parameter Estimates
» Cov Parm  Subject       Group         Estimate
» FA(1,1)   subject                     0.4427
» FA(2,1)   subject                     0.4236
» FA(2,2)   subject                     0.2481
» Residual  subject   formulation Ref   0.05301
» Residual  subject   formulation Test  0.02648

OK, similar again (except for the usual suspects).

» Question, what does the residual "formulation Test" represent?

Nothing. Only a desperate attempt of the algo to estimate something based on too little information. Think about a parallel design. Intra-subject variability is there, but we cannot extract it (only the total/pooled). We need a cross-over. Same here. s²wT exists, but we have only one observation of T / subject.

» Is it the residual attributed to both test and ref, …

Cough.

» … while residual "formulation ref" is attributed to the ref (since it was given 2x)?

Correct.

» which one would one use to compute the 90% geometric CI then?

None of them directly. PHX spits out a standard error of the difference T–R of 0.0474839 (too lazy to dig out the reference on how it is composed from the other variances).

» » IMHO, since a partial replicate according to FDA’s model is always (!) overspecified there is no guarantee that the LME-engines will converge.
»
» Any hint on the stat approach?

No idea.

I took the opportunity and massaged you data. I kept all subjects in sequence RTR, and recoded a little bit. Of course I had to change some Rs to Ts… I got:
Subject Sequence Period Treatment lAUCt lAUCi 1 RTR 1 R 7.918112873 7.956429845 1 RTR 2 T 7.392949199 7.966255738 1 RTR 3 R 8.165388938 8.236181092 2 RTR 1 R 7.648429922 7.840311803 2 RTR 2 T 7.486969192 7.642465619 2 RTR 3 R 7.767760063 7.999909090 3 RTR 1 R 7.845094920 7.903718188 3 RTR 2 T 7.795040591 7.900675886 3 RTR 3 R 7.004410917 7.201138817 4 RTR 1 R 8.273957131 8.299682496 4 RTR 2 T 7.704527971 7.810454463 4 RTR 3 R 7.893447812 7.922782677 5 RTR 1 R 7.873074242 7.993014477 5 RTR 2 T 7.743414530 7.813019037 5 RTR 3 R 7.928661129 7.973013452 6 RTR 1 R 8.953187117 9.070644877 6 RTR 2 T 8.862899236 8.887092743 6 RTR 3 R 8.620163052 8.635798822 7 RTR 1 R 7.733732169 7.925830821 7 RTR 2 T 7.474575339 7.520470843 7 RTR 3 R 7.883843938 7.915115941 8 RTR 1 R 7.040298183 7.138905094 8 RTR 2 T 6.649783454 6.786800469 8 RTR 3 R 6.455060189 6.919911097 9 RTR 1 R 8.246138252 8.302636666 9 RTR 2 T 7.900081068 7.917934089 9 RTR 3 R 8.369926761 8.407009731 10 RTR 1 R 8.854970576 8.867276941 10 RTR 2 T 8.498478152 8.537308319 10 RTR 3 R 8.985634977 9.003601402 11 TRT 1 T 8.178239438 8.255348312 11 TRT 2 R 8.286612805 8.339918637 11 TRT 3 T 7.687297563 7.764432727 12 RTR 1 R 8.240486403 8.305583617 12 RTR 2 T 7.854077711 8.056187995 12 RTR 3 R 7.800706552 7.826649818 14 RTR 1 R 7.553067914 7.897693791 14 RTR 2 T 6.834931866 7.311990298 14 RTR 3 R 7.105642559 7.345723072 15 TRT 1 T 8.238080695 8.313684606 15 TRT 2 R 8.446723945 8.453986902 15 TRT 3 T 8.004066923 8.015037211 17 TRT 1 T 7.954169654 8.081352274 17 TRT 2 R 6.918289598 8.123267843 17 TRT 3 T 7.238486783 7.484658489 18 RTR 1 R 8.009085870 8.197899082 18 RTR 2 T 8.813960369 8.878784152 18 RTR 3 R 7.761056370 7.935726911 19 TRT 1 T 7.936215453 8.005634299 19 TRT 2 R 8.102567049 8.141680789 19 TRT 3 T 7.693701307 7.710909115 20 RTR 1 R 8.263957891 8.294051390 20 RTR 2 T 7.585678156 7.624085152 20 RTR 3 R 8.174826537 8.190936874 21 TRT 1 T 8.120727864 8.167847324 21 TRT 2 R 7.934564680 8.013054162 21 TRT 3 T 7.882511503 7.908029742 23 RTR 1 R 7.358441651 7.473058861 23 RTR 2 T 7.252320394 7.329537863 23 RTR 3 R 7.488031030 7.535879021 25 RTR 1 R 7.602494691 7.697937968 25 RTR 2 T 7.460390761 7.612326456 25 RTR 3 R 7.387379977 7.469852557 26 TRT 1 T 8.329193303 8.344287565 26 TRT 2 R 8.037593588 8.060771153 26 TRT 3 T 7.519507463 7.544075461 27 TRT 1 T 7.842873653 7.906903707 27 TRT 2 R 7.905330655 7.935414562 27 TRT 3 T 8.082943041 8.290801370 28 TRT 1 T 7.738586593 7.774590185 28 TRT 2 R 7.859365663 7.889316764 28 TRT 3 T 7.974813947 8.117347067 29 TRT 1 T 7.952082082 8.030115018 29 TRT 2 R 7.888838125 8.040802810 29 TRT 3 T 7.779364076 7.821842385 30 RTR 1 R 6.982335711 7.543216142 30 RTR 2 T 6.969401496 7.626733448 30 RTR 3 R 6.697153994 6.829441243 31 TRT 1 T 7.747856304 7.809290887 31 TRT 2 R 7.560504854 7.797942664 31 TRT 3 T 7.375110404 7.425354311 32 TRT 1 T 7.854046269 7.909554125 32 TRT 2 R 7.759783505 7.817275946 32 TRT 3 T 7.356405667 7.515982167 33 RTR 1 R 8.368864630 8.379766221 33 RTR 2 T 8.180100682 8.240037917 33 RTR 3 R 8.216466676 8.276196466 34 TRT 1 T 8.258608715 8.287758895 34 TRT 2 R 7.998750632 8.069192434 34 TRT 3 T 8.498685373 8.507850080 35 RTR 1 R 7.574446055 7.644724111 35 RTR 2 T 7.689511601 7.753216597 35 RTR 3 R 7.149663272 7.258526933 36 TRT 1 T 7.697065912 7.731297188 36 TRT 2 R 8.249022200 8.258958847 36 TRT 3 T 7.327066091 7.535905700 37 RTR 1 R 8.139238284 8.195958196 37 RTR 2 T 7.743985836 7.768691851 37 RTR 3 R 7.634118159 7.675717682 38 TRT 1 T 8.444112046 8.468275809 38 TRT 2 R 8.725257703 8.835573174 38 TRT 3 T 8.860378559 8.90071176 39 TRT 1 T 7.649143902 7.760258994 39 TRT 2 R 7.265019149 7.428235744 39 TRT 3 T 7.083417210 7.144682596 40 TRT 1 T 8.089394434 8.723463832 40 TRT 2 R 8.215514150 8.256136432 40 TRT 3 T 8.178085044 8.269703924 42 TRT 1 T 7.699680257 8.477479957 42 TRT 2 R 7.876135868 8.032802077 42 TRT 3 T 7.828881103 7.976931205 44 TRT 1 T 8.469052606 8.546889654 44 TRT 2 R 8.410692915 8.428963108 44 TRT 3 T 8.589966402 8.605777165 45 TRT 1 T 8.525890857 8.539325065 45 TRT 2 R 8.458267051 8.472018404 45 TRT 3 T 8.734765894 8.773995092 46 TRT 1 T 7.438072894 7.502691878 46 TRT 2 R 7.296383438 7.439829543 46 TRT 3 T 7.502625684 8.280353272 47 TRT 1 T 8.195452589 8.244132241 47 TRT 2 R 8.349083368 8.362756337 47 TRT 3 T 7.702201940 7.870368371 48 RTR 1 R 8.176094027 8.205685530 48 RTR 2 T 8.326805753 8.371119472 48 RTR 3 R 7.815589152 7.870187719 49 RTR 1 R 8.186195741 8.240900150 49 RTR 2 T 8.236498120 8.251025744 49 RTR 3 R 8.046571451 8.140032993 51 TRT 1 T 8.890424460 8.926528534 51 TRT 2 R 9.073040287 9.097873779 51 TRT 3 T 9.039707436 9.101568489 52 TRT 1 T 7.856079031 7.917417221 52 TRT 2 R 8.921983405 8.936348427 52 TRT 3 T 8.319488803 8.645876426 53 TRT 1 T 8.254585577 8.267455890 53 TRT 2 R 8.047640980 8.076713878 53 TRT 3 T 8.439900183 8.497614983 54 RTR 1 R 8.082637012 8.110054083 54 RTR 2 T 8.774547697 8.830803941 54 RTR 3 R 8.632410093 8.658085799 55 TRT 1 T 8.226321237 8.316787172 55 TRT 2 R 8.105905766 8.171993299 55 TRT 3 T 7.695208954 7.756027950 56 RTR 1 R 8.215087328 8.369842175 56 RTR 2 T 8.088395381 8.173364805 56 RTR 3 R 7.985378156 8.083406692 57 TRT 1 T 7.721135381 7.754655632 57 TRT 2 R 7.865161850 7.899924985 57 TRT 3 T 7.706186822 7.809824158 59 RTR 1 R 7.771470210 7.842954906 59 RTR 2 T 7.624704432 7.649123891 59 RTR 3 R 7.423932557 7.452101942
Now I had a fully replicated design (TRT|RTR). No warnings, no problems with convergence. Unless some genius comes up with a model for the partial replicate which shows no problems with convergence I would avoid it in the future. Alternative: Send your data to [email protected].

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2013-05-02 12:00
(2975 d 12:30 ago)

@ jag009
Posting: # 10522
Views: 18,136

## Third opinion

Hi John,

» Detlew, need help!

Here I am! But don't know if I can help anyway. The whole story "Use Proc MIXED code for Partial replicate design" is mysterious to me.

» Here is the covariance output from SAS on ln AUCt
»
» Covariance Parameter Estimates
» Cov Parm  Subject       Group         Estimate
» FA(1,1)   subject                     0.4427
» FA(2,1)   subject                     0.4236
» FA(2,2)   subject                     0.2481
» Residual  subject   formulation Ref   0.05301
» Residual  subject   formulation Test  0.02648
»
» Question, what does the residual "formulation Test" represent? Is it the residual attributed to both test and ref

No.
As Helmut already pointed out: an ambiguous attempt of the REML algo to obtain the within-subject variance of the Test formulation. But IMHO the model is over-specified (s2D + s2wT not separable, see below) and therefore there is no guarantee that the value obtained is reasonable.

» while residual "formulation ref" is attributed to the ref (since it was given 2x)?

Correct. Unambiguously identifiable.

» which one would one use to compute the 90% geometric CI then?

Not clear to me what a 90% geometric CI is .

The difference µT-µR has as standard error associated with it for the partial replicate design

sd = sqrt((s2D + s2wT + s2wR/2)*sum(1/ni)/seq^2)
where s2D is the variance of the subject-by-formulation interaction, ni are the number of subjects in the sequence groups, seq is the number of sequences.

s2D can be obtained from the G-matrix according to
s2D = g11+g22-2*g12
(see for more details this post).

Since the model seems over-specified try to use a simple model, f.i. neglect s2D which in turn results in a CS variance-covariance structure for the random part. Sometimes this helps.

BTW: @Helmut, asking the FDA seems a very good idea!

Regards,

Detlew
Helmut
★★★

Vienna, Austria,
2013-05-02 14:41
(2975 d 09:49 ago)

@ d_labes
Posting: # 10524
Views: 18,268

## Compound Symmetry

Hi Detlew & John,

» […] try to use a simple model, f.i. neglect s2D which in turn results in a CS variance-covariance structure for the random part. Sometimes this helps.

Nice – no warnings in PHX any more (only two iterations with the default settings).

log AUCt

PE: 94.7377 (90% CI: 87.4898 – 102.586) Final variance parameter estimates: csDiag_11                           0.00843516 csBlock_11                          0.187534 Var(Period*Formulation*Subject)_21  0.0530102 Var(Period*Formulation*Subject)_22  0.0715068

log AUCi

PE: 95.2967 (90% CI: 88.4014 – 102.730) Final variance parameter estimates: csDiag_11                          -0.00942047 csBlock_11                          0.147924 Var(Period*Formulation*Subject)_21  0.0624828 Var(Period*Formulation*Subject)_22  0.0898627

What do you SASians get?

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2013-05-02 16:26
(2975 d 08:03 ago)

@ Helmut
Posting: # 10529
Views: 18,543

## Compound Symmetry - SASian (1)

Dear Helmut,

» What do you SASians get?

Aye, aye Sir, here I am .

log AUCt
     90% CI      PE=94.74%  [ 87.49% ... 102.59%]     Covariance Parameter Estimates   Cov Parm     Subject    Group    Estimate   Variance     subject              0.008436   CS           subject              0.1875   Residual     subject    tmt A     0.07150   Residual     subject    tmt B     0.05300 NOTE: Convergence criteria met. 

log AUCinf
     90% CI      PE=95.30%  [ 88.32% ... 102.82%]      Covariance Parameter Estimates   Cov Parm     Subject    Group    Estimate   Variance     subject                    0 (!)   CS           subject              0.1441   Residual     subject    tmt A     0.07465   Residual     subject    tmt B     0.06088 NOTE: Convergence criteria met. NOTE: Estimated G matrix is not positive definite. NOTE: Asymptotic variance matrix of covariance parameter estimates has been found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=SATTERTH.

Please note the 'Variance' parameter in case of AUCinf.
CS Covariance matrix parameterized in Proc MIXED as:
( CS+var  CS   CS      CS+var)

See this post to notice that we really need 'Variance'=0 in our model (Type CS is only an approximation to that end, hoping the 'Variance' parameter is fitted with a value near zero).
So we eventually have the optimizer to tell that for AUCt. Couldn't figure out in a hurry at moment how to do that.

Regards,

Detlew
Helmut
★★★

Vienna, Austria,
2013-05-03 16:07
(2974 d 08:22 ago)

@ d_labes
Posting: # 10535
Views: 18,061

## Variance=0

Dear Detlew!

» Please note the 'Variance' parameter in case of AUCinf.

… and the negative one obtained in PHX. Seems that SAS’ RLME-engine forces negative values to zero.

» See this post to notice that we really need 'Variance'=0 in our model (Type CS is only an approximation to that end, hoping the 'Variance' parameter is fitted with a value near zero).

Ooh – that one.

» So we eventually have the optimizer to tell that for AUCt.

I’m not very optimistic whether this is possible in PHX at all; I will ask Pharsight. In PHX for linear mixed effects models the initial estimates are derived by the method of moments:
log AUCt

Starting estimates of variance parameters: csDiag_11                           0.00881650 csBlock_11                          0.188267 Var(Period*Formulation*Subject)_21  0.0519883 Var(Period*Formulation*Subject)_22  0.0703479

log AUCi

Starting estimates of variance parameters: csDiag_11                          -0.00961080 csBlock_11                          0.148549 Var(Period*Formulation*Subject)_21  0.0620715 Var(Period*Formulation*Subject)_22  0.0894799

Alternatively one can state initial variances. If I set csDiag_11 (PHX’ terminology) to zero (whilst keeping the others), the optimizer iterates happily around (four iterations instead of two) – only to end up with the same final estimates…

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes