yicaoting
★    

NanKing, China,
2011-10-03 16:37
(4950 d 09:39 ago)

Posting: # 7409
Views: 16,800
 

 Least Square Means (LSM) for unequal sequence [Regulatives / Guidelines]

Dear all,

As far as I know, when number of subjects in sequence RT equals to that of TR, LSM of R or T equals to Geometric Mean of all PKmetrics of R or T.

I want to calculate Least Square Means(LSM) for unequal sequence 2*2 Xover BE for further calculation of Diff(T-R) of LSM, and for further estimation of 90% CI of Diff(T-R) of LSM, and for further estimation of 90%CI of GMR(geometric means ratio).

Let me shown my calculation in details:

My data is extracted from Hauschke D's
Bioequivalence Studies in Drug Development: Methods and Applications
Page 70, (date of Subject 16 and 18 ws deleted to generate unequal sequences)

sub sequence   auc  period formulation ln(auc)
 5     1     249.94    1       1       5.521221
 2     1     339.03    1       1       5.826088
 8     1     184.32    1       1       5.216673
 4     1     242.64    1       1       5.491579
15     1     239.84    1       1       5.479972
13     1     207.4     1       1       5.334649
12     1     209.3     1       1       5.343769
 7     2     235.89    2       1       5.463366
17     2     169.7     2       1       5.134032
10     2     245.48    2       1       5.503215
11     2     134.89    2       1       4.904459
 1     2     288.79    2       1       5.6657
14     2     223.39    2       1       5.408919
 3     2     343.37    2       1       5.838809
 9     2     215.14    2       1       5.371289
 6     2     225.77    2       1       5.419517
 7     2     133.13    1       2       4.891326
11     2     163.93    1       2       5.09944
 6     2     217.97    1       2       5.384357
 1     2     228.04    1       2       5.429521
 3     2     288.21    1       2       5.66369
10     2     248.98    1       2       5.517373
 9     2     213.78    1       2       5.364947
14     2     245.92    1       2       5.505006
17     2     188.05    1       2       5.236708
12     1     231.98    2       2       5.446651
 2     1     329.76    2       2       5.798365
 8     1     249.64    2       2       5.52002
 5     1     201.56    2       2       5.306087
 4     1     258.19    2       2       5.553696
15     1     241.25    2       2       5.485834
13     1     234.19    2       2       5.456133
-----------------------------------------------

in which Treatment 1 was cconsidered as Reference.

First, I used the original data (no transformation) to do BE analysis with WinNonlin 5.2, some of the the results are:
Treatment   LSM      SE    CI level T_critical LowerCI  UpperCI
1         235.1521 12.7463   90       1.727    213.1332 257.1711
2         231.8667 12.7463   90       1.727    209.8477 253.8856

  Diff  Diff_SE
-3.2855 10.3873


My questions are:
1. How to calculate LSM and SE for each Treatment? I want to manually calculate it, can anyone give me some exact equation?
I have read WNL's user guide, it didn't tell me the exact equation. Only reads "(computed by LinMix)," on page 332 of WNL 5.1's User Guide
2. Why SE of the two treatments are identical?

Second, I setted "Ln(x)" transformation in WinNonlin's BE Wizard, some of the resuls are:
Treatment   LSM      SE    CI level T_critical LowerCI  UpperCI
1           5.4356  0.0563   90       1.721     5.3387   5.5326
2           5.4266  0.0563   90       1.721     5.3296   5.5235

LSMDiff    0.0091
Diff_SE    0.0513


I have the same questions as above mentioned.

I have tried to analysis this unequal sequences data in SAS and Stata, both were failed.

In SAS, I used the following code:
proc glm data=dose_equivalence;
class subject sequence period formulation;
model logAUC=sequence subject(sequence) period formulation;
random subject(sequence) / test;
lsmeans formulation/pdiff cl alpha=0.1;
run;
quit;


It returned:
ERROR: One or more variables are missing or freq or weight is zero on every observation.
182  quit;


In Stata, I used the following code:
pkequiv auc formulation period sequence sub
It returned:
must specify an equivalence comparison /*
r(198);


Is SAS or Stata not able to deal with unequal sequences, or I didn't use the right code?

Thank you very much for your help.


Edit: Category changed. Please don’t paste tabs into your post; use the Preview to check. [Helmut]
ElMaestro
★★★

Denmark,
2011-10-03 16:51
(4950 d 09:24 ago)

@ yicaoting
Posting: # 7410
Views: 15,285
 

 Least Square Means (LSM) for unequal sequence

Dear yicaoting,

❝ 1. How to calculate LSM and SE for each Treatment? I want to manually calculate it, can anyone give me some exact equation?

❝ I have read WNL's user guide, it didn't tell me the exact equation. Only reads "(computed by LinMix)," on page 332 of WNL 5.1's User Guide

❝ 2. Why SE of the two treatments are identical?


1. You take the Mean of T in Seq TR and the mean of T in Seq RT. You add them and divide by two. Do the same for R.

2. The crucial variability for a 2,2,2-BE design from which the 90% CI is derived is the residual sigma (which is your [pseudo-]within variability). Since there is no true replication of neither T nor R you cannot derive a within-subject variability separately for T or R. Think of it in matrix terms: Your error matrix is just a bunch of zeros and with the common sigma on the diagonal.

Pass or fail!
ElMaestro
yicaoting
★    

NanKing, China,
2011-10-03 17:28
(4950 d 08:47 ago)

@ ElMaestro
Posting: # 7412
Views: 14,478
 

 Least Square Means (LSM) for unequal sequence

Dear ElMaestro,

❝ 1. You take the Mean of T in Seq TR and the mean of T in Seq RT. You add them and divide by two. Do the same for R.


Thank you for your guide, your equation works well to calculate LSM for unequal sequences data (both for original and Ln-transformed data), thank you again.
But how to calculate SE in WinNonlin? The same SEs for R and T, why?

❝ 2. The crucial variability for a 2,2,2-BE design from which the 90% CI is derived is the residual sigma (which is your [pseudo-]within variability). Since there is no true replication of neither T nor R you cannot derive a within-subject variability separately for T or R. Think of it in matrix terms: Your error matrix is just a bunch of zeros and with the common sigma on the diagonal.


Thank you for your explanation on residual sigma.
ElMaestro
★★★

Denmark,
2011-10-03 17:36
(4950 d 08:39 ago)

@ yicaoting
Posting: # 7413
Views: 14,484
 

 Least Square Means (LSM) for unequal sequence

Dear yicaoting,

❝ But how to calculate SE in WinNonlin? The same SEs for R and T, why?


I don't use WinNonlin so can't be of help. Answer to the other aspect (same SEs for R and T) is in my pt. 2 above.

best regards,
EM.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-03 18:14
(4950 d 08:01 ago)

@ yicaoting
Posting: # 7415
Views: 14,441
 

 Least Square Means (LSM) for unequal sequence

Dear yicaoting!

❝ But how to calculate SE in WinNonlin?


After reading the entire LinMix-Section of the manual I’m confused as well. Consider registering at Pharsight’s Extranet and ask there. Maybe they come up with a more comprehensible description. ;-)
BTW, you are not alone. A similar question did not get a single answer…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2011-10-03 22:10
(4950 d 04:05 ago)

@ Helmut
Posting: # 7418
Views: 14,471
 

 Least Square Means (LSM) for unequal sequence

Dear Both,

perhaps the apps mean to calculate the between-subject SE's (can be calculated for both T and R and assumed to be equal [not a word about Welch!])?


EM
Pankaj Bhangale
☆    

India,
2011-10-04 16:52
(4949 d 09:23 ago)

@ ElMaestro
Posting: # 7422
Views: 14,396
 

 Least Square Means (LSM) for unequal sequence

❝ Dear ElMaestro,


❝ ❝ 1. How to calculate LSM and SE for each Treatment? I want to manually calculate it, can anyone give me some exact equation?


❝ 1. You take the Mean of T in Seq TR and the mean of T in Seq RT. You add them and divide by two. Do the same for R.:confused:


For Example:
Suppose, n1(no. of TR Seq)=8 & n2(no. of RT Seq.)=6 (i.e. unequal sequences)

M1=Mean of T in Seq TR (n1=8) = 86.62
M2=Mean of T in Seq RT (n2=6) = 67.73

According to your formula, Combine mean is (86.62 + 67.73)/2 = 77.17 (This formula is correct when n1=n2) but here n1 & n2 are different.

I suggest Formula for Combine Mean=((n1*M1)+(n2*M2))/(n1+n2)

Mean of T in Seq TR & RT = ((8*86.62)+(6*67.73))/(8+6)=78.52


Thanks,
Best Regards,
Pankaj Bhangale
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-05 03:53
(4948 d 22:22 ago)

@ Pankaj Bhangale
Posting: # 7425
Views: 14,346
 

 Weighted Means?

Dear Pankaj,

personally I have great sympathy for your procedure (suggesting a weighted mean). With yicaoting’s data of formulation 1 we would get:

(238.92·7 + 231.38·9)/(7+9) = 234.68

Unfortunately both SAS and Phoenix/WinNonlin report only the LSM (238.92 + 231.38)/2 = 235.15 (which is required in many guidelines). No big deal if sequences are not too unbalanced.
I think your suggestion is reasonable but you open a can of worms (see also this post and ElMaestro’s reply).

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-10-04 11:39
(4949 d 14:36 ago)

@ yicaoting
Posting: # 7420
Views: 15,349
 

 LSM for unbalanced sequences -SAS-

Dear yicaoting,

❝ I have tried to analysis this unequal sequences data in SAS and Stata, both were failed.


❝ In SAS, I used the following code:

❝ ...

❝ It returned:

ERROR: One or more variables are missing or freq or weight is zero on every observation.

❝ 182  quit;


Sorry but I can't reproduce your Error(neous) result utilizing SAS 9.2.
Using your data and your code (exactly as given above by you) all goes fine and I get for the LSM section:
Least Squares Means

                               H0:LSMean1=
                     logAUC      LSMean2
formulation          LSMEAN       Pr > |t|
1                5.43564048         0.8623
2                5.42656865


                     logAUC
formulation          LSMEAN      90% Confidence Limits
1                  5.435640        5.371701     5.499580
2                  5.426569        5.362629     5.490508


     Least Squares Means for Effect formulation

            Difference
               Between    90% Confidence Limits for
i    j           Means       LSMean(i)-LSMean(j)
1    2        0.009072       -0.081353     0.099496


So, what did you do here :confused:
Check your variables. The error occurred hints to a left hand variable in the model which has only missing values everywhere.

BTW: We had already several discussions here on unbalanced / incomplete data. See for instance this thread. But be warned: Rather lengthy and nitpicking :-D.

Regards,

Detlew
yicaoting
★    

NanKing, China,
2011-10-04 21:37
(4949 d 04:38 ago)

@ d_labes
Posting: # 7423
Views: 14,475
 

 LSM for unbalanced sequences -SAS-

Dear d_labes,
Thanks for your test of my SAS code.

Now, I can manually calculate LSM to obtain the same result of WinNonlin and SAS. But new problem arises.

Both WNL and SAS give us the same SE for R and T, why?
The SE and 90%CI for R and T are not equal within WNL and SAS, which will be reliable? (Even for equal sequences data)

Let's take my results for my unequal sequences data as example:

In WNL:
For original data, I get:
Treatment   LSM      SE    CI level T_critical LowerCI  UpperCI
1         235.1521 12.7463   90       1.727    213.1332 257.1711
2         231.8667 12.7463   90       1.727    209.8477 253.8856


For "Ln(x)" transformed data, I get:
Treatment   LSM      SE    CI level T_critical LowerCI  UpperCI
1           5.4356  0.0563   90       1.721     5.3387   5.5326
2           5.4266  0.0563   90       1.721     5.3296   5.5235


In SAS:
For original data, I get:
                                                                           H0:LSMean1=
                                  logAUC        Standard    H0:LSMEAN=0      LSMean2
             formulation          LSMEAN           Error       Pr > |t|       Pr > |t|
             1                235.152143        7.344914         <.0001         0.7564
             2                231.866667        7.344914         <.0001


                                           logAUC
                      formulation          LSMEAN      90% Confidence Limits
                      1                235.152143      222.215471   248.088815
                      2                231.866667      218.929995   244.803339


For "Ln(x)" transformed data, I get:
                                        Least Squares Means

                                                                           H0:LSMean1=
                                  logAUC        Standard    H0:LSMEAN=0      LSMean2
             formulation          LSMEAN           Error       Pr > |t|       Pr > |t|
             1                5.43564054      0.03630235         <.0001         0.8623
             2                5.42656860      0.03630235         <.0001


                                           logAUC
                      formulation          LSMEAN      90% Confidence Limits
                      1                  5.435641        5.371701     5.499580
                      2                  5.426569        5.362629     5.490508


It's really puzzled me, I am eagerly need help from d_labes, HS, yjlee168, ElMaestro or others who are expertised at this problem.

Give my greatest thanks to you.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-05 03:01
(4948 d 23:14 ago)

@ yicaoting
Posting: # 7424
Views: 14,510
 

 SE for unbalanced sequences SAS vs. WinNonlin

Dear yicaoting,

yeah, that’s funny. I’ve compared WNL to SAS numerous times – but only looking at LSM, the PE and CI. Never checked the SE / CI of the formulations.
I’ll send Simon an invitation to join the party.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-10-05 14:45
(4948 d 11:30 ago)

@ yicaoting
Posting: # 7426
Views: 14,324
 

 LS Means mean?

Dear yicaoting!

❝ Both WNL and SAS give us the same SE for R and T, why?

I as a bloody statistical layperson can't explain with understandable words what LSMeans really are. Try to :google: 'standard error least square means' OR 'lsmeans standard error' and hope to find an explanation which satisfies you.
I myself have accepted this as a fact within the 2x2 crossover design evaluation without really understanding the 'Why'.
Since the term 'Least square mean' is ascribed to [image][image] this is a hornets' nest similar to the 'Type III Sum of squares'.

❝ The SE and 90%CI for R and T are not equal within WNL and SAS, which will be reliable? (Even for equal sequences data)


Wow! That's curious.
But it does not matter too much because we are not interested in the LSMeans itself but in the treatment difference (in the log-transformed domain) and the 90% CI for that.
As long as these results are identical all is right with the world I think :cool:.

Regards,

Detlew
ElMaestro
★★★

Denmark,
2011-10-05 15:23
(4948 d 10:52 ago)

@ d_labes
Posting: # 7427
Views: 14,315
 

 LS Means mean?

Very well said, d_labes.

LSMeans and type III SS borderline a little on religion.
For LSMeans, one potential argument for using them is that they are directly extracable as the model effects (b) from the y=Xb standard linear model when it has been fit with least squares and if proper contrasts were used in X. The difference is directly extractable regardless of the contrasts used.
Least squares effects might be a better term?

EM
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-08 04:21
(4945 d 21:54 ago)

@ d_labes
Posting: # 7449
Views: 14,031
 

 PHX/WNL vs. SAS

Comrades!

❝ ❝ The SE and 90%CI for R and T are not equal within WNL and SAS, which will be reliable? (Even for equal sequences data)


❝ Wow! That's curious.


Mistery solved (?) after this post.
  • untransformed
    PHX/WNL 6.2
    fixed:  sequence+formulation+period
    random: sub(sequence)
    Treatment   LSM      SE     LowerCI  UpperCI
    1         235.1521 12.7463 213.1332 257.1711
    2         231.8667 12.7463 209.8477 253.8856
    --------------------------------------------------------
            Estimate  StdError  P_value LowerCI    UpperCI
    1 - 2  3.2854762 10.387277  0.75644 -15.009737 21.580689

    fixed:  sequence+formulation+period+sub(sequence)
    Treatment   LSM      SE     LowerCI  UpperCI
    1         235.1521 7.34491 222.2155 248.0888
    2         231.8667 7.34491 218.9300 244.8033
    --------------------------------------------------------
            Estimate  StdError  P_value LowerCI    UpperCI
    1 - 2  3.2854762 10.387277  0.75644 -15.009737 21.580689


    SAS
    fixed:  sequence+formulation+period
    random: sub(sequence)
    Treatment   LSM      SE     LowerCI  UpperCI
    1         235.1521 7.34491 222.2155 248.0888
    2         231.8667 7.34491 218.9300 244.8033
    --------------------------------------------------------
            Estimate  StdError  P_value LowerCI    UpperCI
    1 - 2  3.2855    10.3873    0.7564  -15.0097   21.5807


  • ln-transformed
    PHX/WNL 6.2
    fixed:  sequence+formulation+period
    random: sub(sequence)
    Treatment   LSM      SE         LowerCI  UpperCI
    1         5.4356405 0.0563377 5.3386689 5.5326121
    2         5.4265687 0.0563377 5.3295970 5.5235403
    ------------------------------------------------------
            Estimate  StdError  P_value  LowerCI  UpperCI
    1 - 2   0.009072 0.0513393  0.8623  -0.081353 0.099496

    fixed:  sequence+formulation+period+sub(sequence)
    Treatment   LSM      SE       LowerCI  UpperCI
    1         5.435641 0.0363024 5.371701 5.499580
    2         5.426569 0.0363024 5.362629 5.490508
    ------------------------------------------------------
            Estimate  StdError  P_value  LowerCI  UpperCI
    1 - 2   0.009072 0.0513393  0.8623  -0.081353 0.099496


    SAS
    fixed:  sequence+formulation+period
    random: sub(sequence)
    Treatment   LSM      SE       LowerCI  UpperCI
    1         5.435641 0.0363024 5.371701 5.499580
    2         5.426569 0.0363024 5.362629 5.490508
    ------------------------------------------------------
            Estimate  StdError  P_value  LowerCI  UpperCI
    1 - 2   0.009072 0.0513393  0.8623  -0.081353 0.099496

If you want to get in PHX/WNL the same LSMs/SE/CIs of treatments as in SAS you have to set all effects fixed.

Maybe you can try SAS’ Proc MIXED instead of Proc GLM and tweak it in order to give you LSMEANS similar to PHX/WNL.
But I’m not equipped with [image]

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2011-10-10 17:46
(4943 d 08:29 ago)

@ Helmut
Posting: # 7459
Views: 14,272
 

 PHX/WNL vs. SAS Proc Mixed

Dear Helmut, dear All!

Here the results with real mixed model software:
Proc mixed data=dose_equivalence;
class subject sequence period formulation;
  model logAUC= formulation period sequence;
  random subject(sequence);
  lsmeans formulation / diff cl alpha=0.1;
run;


(only the least square means part shown for the ln-transformed data)
                               The Mixed Procedure

                               Least Squares Means

                                      Standard
Effect      formulation  Estimate     Error    DF  t Value  Pr > |t|   Alpha
formulation  1             5.4356   0.05634    14    96.48    <.0001     0.1
formulation  2             5.4266   0.05634    14    96.32    <.0001     0.1

                               The Mixed Procedure

                               Least Squares Means

                  Effect       formulation     Lower       Upper
                  formulation  1              5.3364      5.5349
                  formulation  2              5.3273      5.5258


                       Differences of Least Squares Means

                                                       Standard
Effect      formulation   _formulation   Estimate      Error     DF   t Value
formulation   1             2            0.009072    0.05134     14      0.18

                        Differences of Least Squares Means

Effect      formulation   _formulation   Pr > |t|    Alpha    Lower     Upper
formulation   1             2             0.8623      0.1   -0.08135    0.0995


Seems there is a perfect consistency of the results.

Remember (from our previous discussions about this topic) that SAS Proc GLM fits all effects as fixed effects. The random statement handles the named effects as random only in a "post-hoc manner", what ever this really means.

That's the reason why our captain calls this statement within Proc GLM bogus.
WNL on the other hand uses in his default evaluation obviously the real mixed model solution.
So far so good.

To be conform with the holy scripture (term invented by EM), page 15
"The terms to be used in the ANOVA model are usually sequence, subject within sequence, period and formulation. Fixed effects, rather than random effects, should be used for all terms."
the PHX/WNL user should leave the default and define all effects as fixed.

That answers Yicoating's question "Which is reliable?"
For the great oracle EMA: the SAS results using Proc GLM (random statement or not) or WNL results after setting all effects fixed :smoke:.

From a scientific point of view there are good reasons to use the real mixed effects solution.
Fortunately the outcome we are interested in at the end - the difference between formulations and their 90% CI - doesn't depend on the choice :cool:. But only if we talk simple 2x2 crossover without missing values. See here.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2011-10-10 18:30
(4943 d 07:45 ago)

@ d_labes
Posting: # 7460
Views: 13,910
 

 PHX/WNL vs. SAS Proc Mixed

Dear D. Labes,

THX for the comparison! I guess Pharsight will add a note to the manual. PHX/WNL users will learn to life with many lines of "not estimable" in the output if all effects are fixed. ;-)

   Effect:Level  Estimate   StdError Denom_DF T_stat   P_value  Conf T_crit  Lower_CI Upper_CI
----------------------------------------------------------------------------------------------
           int   5.21057    0.108907    14   47.8442    0.0000   95   2.145   4.977   5.444
    sequence:1   0.297533   0.144071    14    2.06519   0.0579   95   2.145  -0.01147 0.6065
    sequence:2    Not estimable
 formulation:1   0.00907183  0.0513393  14    0.176703  0.8623   95   2.145  -0.101   0.1192
 formulation:2    Not estimable
      period:1    -0.0594768 0.0513393  14   -1.1585    0.2660   95   2.145  -0.1696  0.05064
      period:2    Not estimable
sequence:1 sub:1  Not estimable
sequence:1 sub:2   0.329324  0.144071   14    2.28585   0.0384   95   2.145   0.02032 0.6383
sequence:1 sub:3   Not estimable
sequence:1 sub:4   0.0397345 0.144071   14    0.275799  0.7867   95   2.145  -0.2693  0.3487
sequence:1 sub:5  -0.069249  0.144071   14   -0.48066   0.6382   95   2.145  -0.3782  0.2398
sequence:1 sub:6  Not estimable
sequence:1 sub:7  Not estimable
sequence:1 sub:8  -0.114557 0.144071    14   -0.795141  0.4398   95   2.145  -0.4236  0.1944
sequence:1 sub:9  Not estimable
sequence:1 sub:10  Not estimable
sequence:1 sub:11  Not estimable
sequence:1 sub:12 -0.087693 0.144071    14   -0.60868   0.5525   95   2.145  -0.3967  0.2213
sequence:1 sub:13 -0.087512 0.144071    14   -0.607424  0.5533   95   2.145  -0.3965  0.2215
sequence:1 sub:14 Not estimable
sequence:1 sub:15 Not estimable
sequence:1 sub:17 Not estimable
sequence:2 sub:1   0.362241 0.144071    14    2.51432   0.0248   95   2.145   0.05324 0.6712
sequence:2 sub:2  Not estimable
sequence:2 sub:3   0.56588  0.144071    14    3.92779   0.0015   95   2.145   0.2569  0.8749
sequence:2 sub:4  Not estimable
sequence:2 sub:5  Not estimable
sequence:2 sub:6   0.216567 0.144071    14    1.5032    0.1550   95   2.145  -0.09243 0.5256
sequence:2 sub:7  -0.008024 0.144071    14   -0.0556949 0.9564   95   2.145  -0.317   0.301
sequence:2 sub:8  Not estimable
sequence:2 sub:9   0.182748 0.144071    14    1.26846   0.2253   95   2.145  -0.1263  0.4917
sequence:2 sub:10  0.324924 0.144071    14    2.25531   0.0406   95   2.145   0.01592 0.6339
sequence:2 sub:11 -0.18342  0.144071    14   -1.27313   0.2237   95   2.145  -0.4924  0.1256
sequence:2 sub:12 Not estimable
sequence:2 sub:13 Not estimable
sequence:2 sub:14  0.271593 0.144071    14    1.88513   0.0803   95   2.145  -0.03741 0.5806
sequence:2 sub:15  Not estimable
sequence:2 sub:17  Not estimable

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2011-10-10 21:53
(4943 d 04:23 ago)

@ Helmut
Posting: # 7463
Views: 14,070
 

 PHX/WNL vs. SAS Proc Mixed

Dear HS,

❝ THX for the comparison! I guess Pharsight will add a note to the manual. PHX/WNL users will learn to life with many lines of "not estimable" in the output if all effects are fixed. ;-)


Fortunately you can get rid of a lot of those "not estimable"s. There are two kinds of "not estimable" in play. Those effects which exist but whose addition does not increase the rank of the model matrix (e.g. column for treatment 2 is the intercept minus treatment 1; hence loss of a treatment df) and those which simply don't exist. The former is trivial, the latter should be trivial and easily avoided because subjects are uniquely coded but it seems certain bad habits are difficult to get rid of :-D

EM.
yicaoting
★    

NanKing, China,
2011-10-13 20:39
(4940 d 05:36 ago)

@ Helmut
Posting: # 7481
Views: 14,335
 

 Real mixed effects in WNL, SAS, GLM or Mixed (1)

Dear HS, d_labes and ElMaestro,

Thank all of you for your great discussion on LSM comparison between WNL and SAS. The concept of "real mixed effects" in WNL or SAS is key factor that leads to different SE, and thus 90%CI for LSM.

Now let me go on with the story of Proc GLM and Proc Mixed in SAS.
Let me begin the story with my test results in SAS.

All tests are performed using untransformed data.

----GLM 1 and GLM 2----Start--------------------------------------------------------
proc glm data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence subject(sequence) period formulation;
random subject(sequence) / test;

lsmeans formulation/stderr pdiff cl alpha=0.1;
run;
quit;

proc glm data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence subject(sequence) period formulation;
lsmeans formulation/stderr pdiff cl alpha=0.1;
run;
quit;
                                  The GLM Procedure

                                 Least Squares Means

                                                                    H0:LSMean1=
                                         Standard    H0:LSMEAN=0      LSMean2
      formulation      AUC LSMEAN           Error       Pr > |t|       Pr > |t|
      1                235.152143        7.344914         <.0001         0.7564
      2                231.866667        7.344914         <.0001


      formulation      AUC LSMEAN      90% Confidence Limits
      1                235.152143     222.215471   248.088815
      2                231.866667     218.929995   244.803339


                     Least Squares Means for Effect formulation

                             Difference
                                Between    90% Confidence Limits for
                 i    j           Means       LSMean(i)-LSMean(j)
                 1    2        3.285476      -15.009741    21.580693

----GLM 1 and GLM 2---End---------------------------------------------------------

----GLM 3 and GLM 4----Start--------------------------------------------------------
proc glm data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence period formulation;
random subject(sequence) / test;

lsmeans formulation/stderr pdiff cl alpha=0.1;
run;
quit;

proc glm data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence period formulation;
lsmeans formulation/stderr pdiff cl alpha=0.1;
run;
quit;
                                  The GLM Procedure
                                 Least Squares Means

                                                                    H0:LSMean1=
                                         Standard    H0:LSMEAN=0      LSMean2
      formulation      AUC LSMEAN           Error       Pr > |t|       Pr > |t|
      1                235.152143       12.746266         <.0001         0.8567
      2                231.866667       12.746266         <.0001


      formulation      AUC LSMEAN      90% Confidence Limits
      1                235.152143     213.469076   256.835210
      2                231.866667     210.183600   253.549734


                      Least Squares Means for Effect formulation

                             Difference
                                Between    90% Confidence Limits for
                 i    j           Means       LSMean(i)-LSMean(j)
                 1    2        3.285476      -27.379011    33.949964

----GLM 3 and GLM 4---End---------------------------------------------------------

----Mixed 1--Start------------------------------------------------------------------------

proc mixed data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence subject(sequence) period formulation;
random subject(sequence) / subject=subject;

lsmeans formulation/cl diff alpha=0.1;
run;
quit;

                                 The Mixed Procedure

                                 Least Squares Means
                  Effect         formulation       Lower       Upper

                  formulation    1                222.22      248.09
                  formulation    2                218.93      244.80


                         Differences of Least Squares Means

                                               Standard
Effect      formulation _formulation Estimate    Error DF t Value Pr > |t| Alpha
formulation 1           2              3.2855  10.3873 14   0.32   0.7564    0.1

                          Differences of Least Squares Means

             Effect       formulation  _formulation     Lower       Upper
             formulation  1            2             -15.0097     21.5807


----Mixed 1--End------------------------------------------------------------------------
yicaoting
★    

NanKing, China,
2011-10-13 20:52
(4940 d 05:23 ago)

@ Helmut
Posting: # 7482
Views: 13,912
 

 Real mixed effects in WNL, SAS, GLM or Mixed (2)

----Mixed 2--Start------------------------------------------------------------------------
proc mixed data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence subject(sequence) period formulation;
lsmeans formulation/cl diff alpha=0.1;
run;
quit;

                              The Mixed Procedure

                            Type 3 Tests of Fixed Effects

                                      Num     Den
                Effect                 DF      DF    F Value    Pr > F
                period                  1      14       1.78    0.2031
                formulation             1      14       0.10    0.7564


                                 Least Squares Means

                                   Standard
Effect      formulation  Estimate     Error    DF  t Value  Pr > |t|   Alpha
formulation 1            235.15      7.3449    14    32.02    <.0001     0.1
formulation 2            231.87      7.3449    14    31.57    <.0001     0.1

                  Effect         formulation       Lower       Upper
                  formulation    1                222.22      248.09
                  formulation    2                218.93      244.80


                         Differences of Least Squares Means

                                              Standard
Effect      formulation _formulation Estimate    Error DF t Value Pr > |t| Alpha
formulation 1           2              3.2855  10.3873 14    0.32   0.7564   0.1

             Effect       formulation  _formulation     Lower       Upper
             formulation  1            2             -15.0097     21.5807


----Mixed 2--End------------------------------------------------------------------------

----Mixed 3--Start------------------------------------------------------------------------
proc mixed data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence period formulation;
random subject(sequence) / subject=subject;

lsmeans formulation/cl diff alpha=0.1;
run;
quit;

                                     The Mixed Procedure

                                        Fit Statistics

                             AICC (smaller is better)        305.1
                             BIC (smaller is better)         306.1


                                 Type 3 Tests of Fixed Effects

                                        Num     Den
                        Effect           DF      DF    F Value    Pr > F
                        sequence          1      14       0.85    0.3732
                        period            1      14       1.78    0.2031
                        formulation       1      14       0.10    0.7564


                                      Least Squares Means

                                 Standard
Effect      formulation Estimate   Error  DF  t Value Pr > |t| Alpha
formulation 1             235.15 12.7463  14    18.45   <.0001   0.1
formulation 2             231.87 12.7463  14    18.19   <.0001   0.1

                       Effect         formulation       Lower       Upper
                       formulation    1                212.70      257.60
                       formulation    2                209.42      254.32


                              Differences of Least Squares Means

                                              Standard
Effect      formulation _formulation Estimate    Error  DF t Value Pr > |t| Alpha
formulation 1           2              3.2855  10.3873  14    0.32   0.7564   0.1

                  Effect       formulation  _formulation     Lower       Upper
                  formulation  1            2             -15.0097     21.5807


----Mixed 3--End------------------------------------------------------------------------

----Mixed 4--Start------------------------------------------------------------------------
proc mixed data=dose_equivalence;
class subject sequence period formulation;
model AUC=sequence period formulation;
lsmeans formulation/cl diff alpha=0.1;
run;
quit;

                                      The Mixed Procedure

                                 Type 3 Tests of Fixed Effects

                                        Num     Den
                        Effect           DF      DF    F Value    Pr > F
                        formulation       1      28       0.03    0.8567


                                      Least Squares Means

                                      Standard
Effect       formulation  Estimate       Error    DF   t Value   Pr > |t|   Alpha
formulation  1              235.15     12.7463    28     18.45     <.0001     0.1
formulation  2              231.87     12.7463    28     18.19     <.0001     0.1

                       Effect         formulation       Lower       Upper
                       formulation    1                213.47      256.84
                       formulation    2                210.18      253.55


                              Differences of Least Squares Means

                                              Standard
Effect      formulation _formulation Estimate    Error  DF t Value Pr > |t| Alpha
formulation 1           2              3.2855  18.0259  28    0.18   0.8567   0.1

                  Effect       formulation  _formulation     Lower       Upper
                  formulation  1            2             -27.3790     33.9500


----Mixed 4--End------------------------------------------------------------------------
yicaoting
★    

NanKing, China,
2011-10-13 21:05
(4940 d 05:10 ago)

@ Helmut
Posting: # 7485
Views: 14,077
 

 Real mixed effects in WNL, SAS, GLM or Mixed (3)

untransformed

PHX/WNL 6.2

fixed:  sequence+formulation+period

❝ random: sub(sequence)

❝ Treatment   LSM      SE     LowerCI  UpperCI

❝ 1         235.1521 12.7463 213.1332 257.1711

❝ 2         231.8667 12.7463 209.8477 253.8856

❝ --------------------------------------------------------

❝         Estimate  StdError  P_value LowerCI    UpperCI

❝ 1 - 2  3.2854762 10.387277  0.75644 -15.009737 21.580689


❝ fixed:  sequence+formulation+period+sub(sequence)

❝ Treatment   LSM      SE     LowerCI  UpperCI

❝ 1         235.1521 7.34491 222.2155 248.0888

❝ 2         231.8667 7.34491 218.9300 244.8033

❝ --------------------------------------------------------

❝         Estimate  StdError  P_value LowerCI    UpperCI

❝ 1 - 2  3.2854762 10.387277  0.75644 -15.009737 21.580689


As HS calculated, since WNL's
fixed: sequence+formulation+period
random: sub(sequence)

and
fixed: sequence+formulation+period+sub(sequence)
generates identical result on Diff 1-2 and it's SE 10.387277 and 90% CI -15.009737 21.580689, and this is what BE analysis TRUELY concerning about, so let's use this result as a temp "Golden Standard".

it can be seen:

when Proc GLM (GLM 3 and GLM 4) is used, never use
model AUC=sequence period formulation
even
random subject(sequence) / test;
is added.

However,
when Proc Mixed (Mixed 3) is used, you can use
model AUC=sequence period formulation;
but remeber to specify
random subject(sequence) / subject=subject;
as random effect.

Now, let's consider WNL's
fixed:  sequence+formulation+period+sub(sequence)
SE 7.34491

as a TRUE fixed effect analysis.

It can be seen that both GLM 1 and GLM 2 are in fixed mode even
random subject(sequence) / test;
is added (see GLM 2)
this is previously discussed as so called "… post hoc fashion …"

When Proc Mixed is used, once you specified
model AUC=sequence subject(sequence) period formulation;
SAS will consider subject(sequence) as random effect.(Mixed 1 and 2), regardless of specifying
random subject(sequence) / subject=subject;
or not, (Mixed 2) the results are identical.

Since all results of SAS's GLM 1, GLM 2, Mixed 1 and Mixed 2, identical
SEs for LSM of R and T are both = 7.344914

90% CI for 1 and 2 are
222.215471   248.088815
218.929995   244.803339


90% CI for diff is
-15.009741    21.580693

can we concluded that this result is reliable?

Besides, as shown is Mixed 2, can we manually obtain SE = 7.344914 from the result of
Type 3 Tests of Fixed Effects?
I have tried, but failed.

Mixed 3 gets right 90% CI for diff, but strange 90% CI for LSMs of R 212.70 257.60 and T 209.42 254.32.

WNL's default
fixed:  sequence+formulation+period
random: sub(sequence)
Treatment   LSM      SE     LowerCI  UpperCI
1         235.1521 12.7463 213.1332 257.1711
2         231.8667 12.7463 209.8477 253.8856
--------------------------------------------------------
        Estimate  StdError  P_value LowerCI    UpperCI
1 - 2  3.2854762 10.387277  0.75644 -15.009737 21.580689

are never obtained by SAS's any trying of Proc GLM or Proc Mixed with many optional settings. So may be it is time to suspect WNL's default setting in BE Wizard? Do you agree?

Thanks to HS, d_labes and ElMaestro for your kind patience on this topic.


Edit: Sorry yicaoting, I tried to edit your post in order to get a more compact style. [Helmut]
ElMaestro
★★★

Denmark,
2011-10-14 02:02
(4940 d 00:13 ago)

@ yicaoting
Posting: # 7489
Views: 13,741
 

 Must admit I am lost

Dear yicaoting,

impressive amount of work.
I must admit you lost me completely quite early here. The reason might be that I do not speak WNL or SAS and/or that my brain is walnut-sized.
My problem is I cannot see what you are trying to achieve.

For my learning purposes and/or your consideration:
1. Why would you use PROC MIXED when not specifying a random effect?
2. Do you know what the documented behaviour of PROC MIXED is when no random effect is specified? There is an example in the online manual but it does not tell what the general behaviour is.
3. Fitting a mixed model with just a lone sigma2 on the diagnonal in the covariance matrix (=sigma2I) is conceptually similar to the linear model, the difference being just that missing values does not mean discarded subjects with PROC MIXED.
4. What is the documented behaviour of PROC MIXED when you specify the same effect as both random and fixed? If I get you right in this case [subject(sequence)] it just defaulted to sigma2 on the diagonal in the covariance matrix, but would anyone really specify a mixed model that way? I speculate, the inner works might simply skip the (or better: a) random effect if it has been already specified as fixed, which in your case just leads to pt. 2. above. If it does exactly the opposite (skips the fixed effect when it is specified as random) it would lead to the same.

Pass or fail!
ElMaestro
yicaoting
★    

NanKing, China,
2011-10-14 08:30
(4939 d 17:46 ago)

@ ElMaestro
Posting: # 7491
Views: 13,702
 

 Must admit I am lost

Dear ElMaestro,
Thank you for your persistent concerning on my post.

❝ My problem is I cannot see what you are trying to achieve.


My only and final purpose is to manually calculate SEs and CIs of R and T in BE analysis both for equal and unequal balances data (situation of incomplete data is beyongd my ability), may be we can called it "post-BE estimation".

❝ For my learning purposes and/or your consideration:

❝ 1. Why would you use PROC MIXED when not specifying a random effect?


I used PROC MIXED without a random effect with only purpose of try-and-see what will we get with such a unusual method. But not to recommend this in BE analysis.

BTW: After my tries, I know that if we use WNL's default settings in BE Wizard, it is impossible get identical result from SAS, here so called identical result includes: LSMs and its SEs and CIs for R and T, and 90% CI of difference. Thus, maybe it is time to modify WNL's default setting or to suspect SAS? I am really :confused:

❝ 2. Do you know what the documented behaviour of PROC MIXED is when no random effect is specified? There is an example in the online manual but it does not tell what the general behaviour is.


I really want to know, but Google gives me no concrete answer. May be it is too complex to list all the calculation steps in Proc Mixed, but I really want to manually calc SE and CI. You know many programs are able to handle matrix and var-covar matrix, so I think it is possible but might be time-consuming, and unfortunately, no one can tell me how to do it.

All in one, even if let's stop the game of Proc Mixed or Proc GLM, is it possible to manually calculate CI of PKmetrics for R and T in 2*2 crossover design?
Without use of WNL or SAS or other software with similar function, we cann't get it?

Another Issue: I know that in NSCC 2007's TOST analysis, SEs for R and T are different, and I have derived it's calculation step, it uses pooled SE from datesets of two sequences for each treatment. Although it does not give out CI, it can be easily calculated using LSMean+/- T*SE. From a personal view, I think different SEs for R and T is more reasonable than the same SEs. What's your opinion?

Thus, which CI is true or acceptable? SAS's (same as WNL's) or NCSS's?

Again, thank you for your attention.
ElMaestro
★★★

Denmark,
2011-10-14 15:23
(4939 d 10:53 ago)

@ yicaoting
Posting: # 7494
Views: 13,776
 

 Must admit I am lost - edited

Hi yicaoting,

❝ BTW: After my tries, I know that if we use WNL's default settings in BE Wizard, it is impossible get identical result from SAS, here so called identical result includes: LSMs and its SEs and CIs for R and T, and 90% CI of difference. Thus, maybe it is time to modify WNL's default setting or to suspect SAS? I am really :confused:


Helmut informed us that WNL used a default Mixed model even for a 2,2,2-BE evaluation. That could be why you don't get the same result. At the end you need to ask yourself: Do I wish to include subjects in my 2,2,2-BE analysis which have a missing period? If you answer is yes, then use a mixed model. If your answer is no, use a linear model (or delete the subjects in question and do the mixed, same thing).

❝ All in one, even if let's stop the game of Proc Mixed or Proc GLM, is it possible to manually calculate CI of PKmetrics for R and T in 2*2 crossover design?


No, it is perfectly possible to calculate a CI 'manually', as long as you want to reproduce the CI you get from GLM (but not MIXED, case of missing values in a period makes the difference). Look up the equations in Chow & Liu's book; I don't have it here.
Edit: You can also look up the equations in Potvin et al. Pharm. Stat. 7:245–262.

❝ Another Issue: I know that in NSCC 2007's TOST analysis, SEs for R and T are different, and I have derived it's calculation step, it uses pooled SE from datesets of two sequences for each treatment. Although it does not give out CI, it can be easily calculated using LSMean+/- T*SE. From a personal view, I think different SEs for R and T is more reasonable than the same SEs. What's your opinion?


My opinion is that in a 2,2,2-design we do not have true replication of Test or Reference. Therefore we cannnot calculate within-subject-variabilities for T or R. We can calculate it for the difference. We can also derive the variability for the between-subject variability for T and R, and we can possibly even do that individually if we have a reason to do so (but don't ask me how; I don't know). So if you are looking for SE's corresponding to intra-subject variability for T or R, look no further until you have a replicated study.

❝ Thus, which CI is true or acceptable? SAS's (same as WNL's) or NCSS's?


There is no true or false. As another example, look up on the www the heated discussions around type I, II and III sums of squares. There is a lot of personal preference and religion involved here. Some of it is written in the form of guidelines. As you saw from an earlier post by Helmut, the EU guideline now asks for subject as fixed factor. If you can accept that as your golden standard then there's your answer.

Pass or fail!
ElMaestro
UA Flag
Activity
 Admin contact
23,424 posts in 4,927 threads, 1,670 registered users;
22 visitors (0 registered, 22 guests [including 3 identified bots]).
Forum time: 02:16 CEST (Europe/Vienna)

The difference between a surrogate and a true endpoint
is like the difference between a cheque and cash.
You can get the cheque earlier but then,
of course, it might bounce.    Stephen Senn

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5