Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-07-11 23:34
(2451 d 19:01 ago)

Posting: # 17535
Views: 5,525
 

 only subjects with TR? [RSABE / ABEL]

Dear all,

I’m asking myself how to interpret this part of the BE-GL in the section Subject accountability:

Ideally, all treated subjects should be included in the statistical analysis. However, subjects in a crossover trial who do not provide evaluable data for both of the test and reference products […] should not be included.

Funny that the first sentence describes an “ideal” situation which can be handled only with a mixed effects model – which at least for conventional crossovers is taboo.
I would say that the second sentence was written having nonreplicated crossovers in mind. We discussed in the forum whether subjects with only RR-data (say due to dropouts in the 3rd period of a partial replicate design in sequence RRT) should be included for the estimation of CVwR and the consensus was: yes.
I simulated a small partial replicate with swT = swR = 0.3, swT = swR = 1 and removed the last observation of sub­ject 18 in sequence RRT:

subject period sequence treatment response
   1       1     TRR        T       75.76
   1       2     TRR        R       59.90
   1       3     TRR        R       91.15
   2       1     TRR        T       24.57
   2       2     TRR        R       19.52
   2       3     TRR        R       21.44
   3       1     TRR        T       53.38
   3       2     TRR        R       43.84
   3       3     TRR        R       32.97
   4       1     TRR        T      157.08
   4       2     TRR        R      115.90
   4       3     TRR        R      141.98
   5       1     TRR        T       76.29
   5       2     TRR        R      104.03
   5       3     TRR        R      169.86
   6       1     TRR        T       85.57
   6       2     TRR        R       49.44
   6       3     TRR        R       70.97
   7       1     RTR        R      453.94
   7       2     RTR        T      408.99
   7       3     RTR        R      452.39
   8       1     RTR        R      109.63
   8       2     RTR        T      128.59
   8       3     RTR        R      153.43
   9       1     RTR        R       83.15
   9       2     RTR        T       87.02
   9       3     RTR        R       62.62
  10       1     RTR        R       46.59
  10       2     RTR        T       26.13
  10       3     RTR        R       89.98
  11       1     RTR        R      131.53
  11       2     RTR        T      176.23
  11       3     RTR        R      124.95
  12       1     RTR        R       14.94
  12       2     RTR        T       13.03
  12       3     RTR        R       13.41
  13       1     RRT        R       75.16
  13       2     RRT        R       39.72
  13       3     RRT        T       94.14
  14       1     RRT        R      109.21
  14       2     RRT        R      131.66
  14       3     RRT        T      109.69
  15       1     RRT        R      121.48
  15       2     RRT        R      192.08
  15       3     RRT        T      143.01
  16       1     RRT        R      122.11
  16       2     RRT        R      127.09
  16       3     RRT        T      116.77
  17       1     RRT        R       59.09
  17       2     RRT        R       37.46
  17       3     RRT        T      105.95
  18       1     RRT        R      500.99
  18       2     RRT        R      155.44
  18       3     RRT        T       NA   

I compared Methods A, B, and C (yes!). If one uses the EMA’s code in SAS (or similar software) and clicks the button that’s the “all in” situation. Below that what happens if we exclude subject 18.

        Method  DF    CVwR    L      U       90% CI       PE     CI   GMR mixed   log-½ 
all in    A    32.0  30.52  79.71 125.46  92.47 125.36  107.67  pass pass  pass  0.15218
          B    32.0  30.52  79.71 125.46  92.13 124.89  107.27  pass pass  pass  0.15211
          C    15.6  29.74  80.00 125.00  90.89 125.91  106.98  fail pass  fail  0.16298
only TR   A    31.0  30.52  79.71 125.46  93.59 123.86  107.67  pass pass  pass  0.14008
          B    31.0  30.52  79.71 125.46  93.59 123.86  107.67  pass pass  pass  0.14008
          C    15.7  24.44  80.00 125.00  91.59 126.57  107.67  fail pass  fail  0.16174


The log half-width (log-½) is useful in comparison methods, where a higher value points to a more conservative decision (wider CI). As usual Method C is the most conservative one but not ”compatible with the guideline”. The Q&A states “… it will generally give wider [sic] confidence intervals than those produced by methods A and B.”. Applicants love it when an agency recommends a liberal method.
Further down in the Q&A:

For replicate designs the results from the two approaches [A and B] will differ if there are subjects included in the analysis who do not provide data for all treatment periods. Either approach is considered scientifically acceptable, but for regulatory consistency it is considered desirable to see the same type of analysis across all applications.
A simple linear mixed model, which assumes identical within-subject variability (Method B), may be acceptable as long as results obtained with the two methods do not lead to different regulatory decisions. However, in borderline cases and when there are many included subjects who only provide data for a subset of the treatment periods, additional analysis using method A might be required.

What does that mean? If we use only subjects with TR-data, results of A and B are identical. If we opt for “all in” an open question is what a “borderline case” is and “many included subjects who only provide data for a subset of the treatment periods” are.
Reading between the lines I got the impression that the EMA believes that Method A is always more conservative than Method B. This is not correct. Data set upon request. ;-)

What do you think? What do you do?

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
zizou
★    

Plzeň, Czech Republic,
2017-07-13 01:12
(2450 d 17:22 ago)

@ Helmut
Posting: # 17536
Views: 4,826
 

 only subjects with TR?

Dear Helmut,

❝ I’m asking myself how to interpret this part of the BE-GL in the section Subject accountability:

Ideally, all treated subjects should be included in the statistical analysis. However, subjects in a crossover trial who do not provide evaluable data for both of the test and reference products […] should not be included.

Funny that the first sentence describes an “ideal” situation which can be handled only with a mixed effects model – which at least for conventional crossovers is taboo.


When subject is vomiting in the first period early after administration and then has e.g. flu. Neither mixed can't help.

❝ I would say that the second sentence was written having nonreplicated crossovers in mind.


Who knows...
EMA mentioned it in Q&A (page 15) also:

The question of whether to use fixed or random effects is not important for the standard two period, two sequence (2×2) crossover trial. In section 4.1.8 of the guideline it is stated that “subjects in a crossover trial who do not provide evaluable data for both of the test and reference products should not be included.” Provided this is followed the confidence intervals for the formulation effect will be the same regardless of whether fixed or random effects are used.

So maybe not. With interest in intra-subject variability I think that test and reference formulations will differ more than only two references in the most of the subjects. Hence the exclusion of subject without T and R data concurrently (i.e. the exclusion of subject with only RR data) will have the impact that the intra-subject CV of all (pooled) data would be higher (more probably). It's because we exclude subject with two RR which probably don't differ so much as T versus R. At least I guess and I see the point in that. To be on the safe side - exclude all these RR subjects from T/R evaluation (as these subjects could affect the intra-subject CV to be lower "incorrectly" based on RR differences of such individual subjects, i.e. affect the 90% CI to be narrower).
Of course it can happen as in your data example that CVW will be lower (i.e. 90% CI narrower) after exclusion of only RR subject(s) but I would expect the opposite really more often.

❝ We discussed in the forum whether subjects with only RR-data (...) should be included for the estimation of CVwR and the consensus was: yes.


Yes from me too if the subject is not outlier - which is other discussed topic with no guideline with definition of the outlier.

(I am thinking in method A as the simplest method to think about and as EMA.)
The comparison of methods is over my capability (especially method C which is mixed with specific settings which is not available in many statistical softwares(?) and I didn't give much time to method C myself to get all points in that method mainly because of method A is preffered by EMA).

Btw. almost everytime I open the FDA draft of progesterone guidance my eyes notice the line (I highlighted it to red) with a little wow:
proc glm data=scavbe;
class seq;
model ilat=seq/clparm alpha=0.1;
estimate 'average' intercept 1 seq 0.3333333333 0.3333333333 0.3333333333;
ods output overallanova=iglm1;
ods output Estimates=iglm2;
ods output NObs=iglm3;
title1 'scaled average BE';
run;

Note: Red color doesn't mean wrong here.

With the fact that FDA do not round off the 90% CI limits it is quite shocking that precision on 10 decimal places is enough here! :-D

I think more beautiful and more accuracy would be:
estimate 'average' intercept 3 seq 1 1 1 / divisor=3;
(I don't have SAS power so if I am wrong please don't shame me.)

A little fun.
I know that in mixed methods there are parameters for convergence criteria (e.g. 0.0000000001) or maximum count of iterations which maybe have also little influence on the precision (?) when 90% CI limit is on the board of 80 or 125 %. (Not so easy as EMA method A.)
What if with default settings BE fails and with more precision we get into 80-125%. When BE is recalculated by regulatory (e.g. FDA) with default settings and the result is fail, then BE is challenge. x)

Best regards,
zizou
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2017-07-13 22:32
(2449 d 20:03 ago)

@ zizou
Posting: # 17543
Views: 4,611
 

 only subjects with TR?

Hi zizou,

partly answering; have to chew on some other points…

❝ The comparison of methods is over my capability (especially method C which is mixed with specific settings which is not available in many statistical softwares(?)


SAS (and JMP = poor man’s SAS), Phoenix WinNonlin, STaTa, …

❝ Btw. almost everytime I open the FDA draft of progesterone guidance my eyes notice the line (I highlighted it to red) with a little wow:

estimate 'average' intercept 1 seq 0.3333333333 0.3333333333 0.3333333333;


Yes, it hurts.

❝ With the fact that FDA do not round off the 90% CI limits …


Oh, rounding is the FDA’s requirement for ages. Actually the EMA followed this bad practice.

❝ I think more beautiful and more accuracy would be:

estimate 'average' intercept 3 seq 1 1 1 / divisor=3;

❝ (I don't have SAS power so if I am wrong please don't shame me.)


I don’t speak SAS either but I’m sure you looked it up in the online manual. :-D

❝ I know that in mixed methods there are parameters for convergence criteria (e.g. 0.0000000001) or maximum count of iterations which maybe have also little influence on the precision (?) when 90% CI limit is on the board of 80 or 125 %. (Not so easy as EMA method A.)

❝ What if with default settings BE fails and with more precision we get into 80-125%. When BE is recalculated by regulatory (e.g. FDA) with default settings and the result is fail, then BE is challenge.


Correct & good point! Never seen a failure in practice* but it is interesting what the Canadians have to say:

By definition [!] the cross-over design is a mixed effects model [!] with fixed and random effects. The basic two period cross-over can be analysed according to a simple fixed effects model and least squares means estimation. Identical results will be obtained from a mixed effects analysis such as Proc Mixed in SAS®. If the mixed model approach is used, parameter constraints should be defined in the protocol. Higher order models must be [!] analysed with the mixed model approach in order to estimate random effects properly.

(my emphases)


  • Except sometimes with the f**g partial replicate design. But this is due to the over-specified model (and not related to software). Tweaking the convergence criteria and/or increasing the number of iterations never helped.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,638 registered users;
90 visitors (0 registered, 90 guests [including 6 identified bots]).
Forum time: 17:35 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5