acfalcao ● 20070916 22:34 Posting: # 1082 Views: 15,041 

Dear All My name is Amilcar Falcão and I am professor at the University of Coimbra, Portugal. I am looking for help regarding the execution and interpretation of threeway crossover bioequivalence trials (for academic purposes). I would like to have an example (explicit) where a threeway crossover study is appropriately analysed. Namely, I would like to know what kind of ANOVA should be performed and how the 90%CI should be calculated for the different combinations of study formulations (i.e. A vs B and A vs C). Perhaps this is a naive question, but in fact I am trying to look for it in literature and I have seen different approaches always without the adopted statistical pathway. Therefore, I would like to be sure regarding the correct way to deal with. I strongly appreciate if you could help me about this matter. Thank you for your attention. Best regards; Amílcar Falcão Laboratory of Pharmacology Faculty of Pharmacy University of Coimbra Coimbra  Portugal 
Helmut ★★★ Vienna, Austria, 20070917 20:47 @ acfalcao Posting: # 1088 Views: 14,283 

Dear Amilcar! » I would like to have an example (explicit) where a threeway crossover study is appropriately analysed. You have to use a Williams’ design (three period, six sequences); the topic was covered in previous threads. A 6×3 design is needed in order to ‘extract’ two 2×2 tables, which are also balanced. Although the full 6×3 table will be used in the analysis of AUC and C_{max}, you will need these 2×2s for the nonparametric analysis of t_{max} (unfortunately there’s no confidence interval based nonparametric method available for more than two formulations/periods). The asterisks * denote pseudosequences and pseudoperiods, e.g. P1* means only that the treatment was given in a period prior to P2* – irrespective of the true study period:+++ > +++ and +++ A common mistake is to design the study as a set of 3×3 latin squares, which will lead (especially if the sample size is small and in the case of drop outs) to extremely imbalanced data sets: +++ > +++ and +++ » Namely, I would like to know what kind of ANOVA should be performed and how the 90%CI should be calculated for the different combinations of study formulations (i.e. A vs B and A vs C). For the design see Chapter 10 of Chow SC, Liu Jp. Jones B, Kenward MG. Patterson S, Jones B. You may download zipped datasets and programs (SAS/S+) from CRC’s website. If you don’t have access to commercial software, S+ code will run with opensource R with minor modifications. Patterson/Jones give results in Table 4.12 (p. 105) with: T/R (% Test vs. Reference 1) ++++ T/S (% Test vs. Reference 2) ++++ WinNonlin 5.2 comes up with: T/R (% Test vs. Reference 1) ++++ T/S (% Test vs. Reference 2) ++++ Results are slightly different (although both WinNonlin and SAS use GLM – not ANOVA; implementation, rounding, etc. is different – and of course ‘proprietary information’ and not documented). I assume Patterson/Jones' results were produced by SAS; I will check the results from their S+ code the next days. » Perhaps this is a naive question,... Not at all; little is published  and no worked examples at all. There’s another point which is a little bit tricky: multiplicity. If you are testing one test (A) against two references (B, C), any impression of ‘data dredging’ must be avoided, e.g., calculation 90% CIs of A/B and A/C  and only picking out the best getting an approval. Since in the EU the Innovator's product from any European country may be used as the reference, you may run into problems (A vs B is BE, whereas A vs C is not BE). It may be wise to use 95% CIs instead (overall Bonferronicorrected patient’s risk: α = 1–(1–0.05/k)^{k}, where k is the number of simultaneous comparisons). 95% CI should also be applied in testing for dose proportionality of three dose levels (or 96.67% for four levels). IMHO the only case where 90% CIs should be used is the comparison of two test formulations against one reference, and only one of them will be further used in the approval process. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
acfalcao ● 20070917 23:02 @ Helmut Posting: # 1089 Views: 13,485 

Dear Helmut Thank you very much for your help. The correct codification of "sequences/periods" is the key for this problem. Unfortunately the link to download the SAS files was not working. I hope it can be solve in a near future. Best regards; Amílcar Edit: Full quote removed. Please delete everything from the text of the original poster which is not necessary in understanding your answer; see also this post #5! [Helmut] 
Helmut ★★★ Vienna, Austria, 20070918 12:47 @ acfalcao Posting: # 1092 Views: 13,624 

Dear Amilcar! » Thank you very much for your help. The correct codification of "sequences/periods" is the key for this problem. Unfortunately the link to download the SAS files was not working. I hope it can be solve in a near future. Please try it again (it worked yesterday, and today as well): CRC’s website Having downloaded the file C5300.zip, unzip it keeping folder's structure; in folder [chapter4] you will find: Example4.5.sas (the SAS code) Example45.SSC (the S+ code) exam45.dat (the raw data for S+) As I get from the SAScode a mixed model (degrees of freedom according to Kenward) was used, whereas WinNonlin uses Satterthwaite's method. The S+ code does not calculate BE, but is used for the example’s figures only. Edit: Wrong link corrected in both posts. [Helmut] — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Nirali ★ India, 20070921 06:49 @ Helmut Posting: # 1106 Views: 13,458 

Dear Helmut, Which hypothesis Winnonlin will use while running BE for 2 test (T1 & T2) vs a reference (R)? Would it be "MEAN(T1)=MEAN(T2)=MEAN(R)"? In threeway crossover study, does we try to compare Equivalence of all three product together? Thanks & Regards, Nirali 
Helmut ★★★ Vienna, Austria, 20070921 13:04 @ Nirali Posting: # 1109 Views: 13,492 

Dear Nirali, » Which hypothesis Winnonlin will use while running BE for 2 test (T1 & T2) vs a reference (R)? » Would it be "MEAN(T1)=MEAN(T2)=MEAN(R)"? » In threeway crossover study, does we try to compare Equivalence of all three product together? Comparisons are based on paired differences; IMHO a simultaneous test of T_{1}=T_{2}=R is not possible. WinNonlin always uses one reference in a run, i.e., T_{1}=R (one test), and T_{2}=R (the second test). If you are interested in T_{1}=T_{2} you have to start a second run (defining T_{2} as the reference), and will obtain two tests (T_{1}=T_{2} and R=T_{2}) within. BTW, why are you interested in T_{1}=T_{2}? — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Nirali ★ India, 20070925 08:05 @ Helmut Posting: # 1127 Views: 13,436 

Dear Helmut, Actully, I was under impression that WinNonlin is using the hypo: "MEAN(Test1)=MEAN(Test2)=MEAN(Ref)". As I understood from your reply (A) suppose we have two REFERENCES and single TEST [R1, R2, T]for threeway comparison: if we select R1 as REFERENCE, WNL will calculate BE with hypo: (1) R1=T and (2) R1=R2. same way, (B) for two TEST and single REFERENCE [T1, T2, R]: if we select R as standard, hypo will be: (1) R=T1 and (2)R=T2 pls do correnct me if i am wrong to understand. Thanks & Regards, Nirali 
Helmut ★★★ Vienna, Austria, 20070925 13:15 @ Nirali Posting: # 1132 Views: 13,549 

Dear Nirali! » Actully, I was under impression that WinNonlin is using the hypo: » "MEAN(Test1)=MEAN(Test2)=MEAN(Ref)". Again, we obtain only pairwise comparisons, not simultaneous ones. » (A) suppose we have two REFERENCES and single TEST [R1, R2, T]for threeway comparison: if we select R1 as REFERENCE, WNL will calculate BE with hypo: » (1) R1=T and (2) R1=R2. Not quite, but very close. If you run the BE wizard (R_{1}=reference), you will get (1a) T vs. R_{1}, and (1b) R_{2} vs. R_{1}. You have to run the BE wizard again (this time with R_{2} as reference), getting (2a) T vs. R_{2}, and (2b) R_{1} vs. R_{2}. In order to get comparisons of T=R_{1} and T=R_{2}, you have to run the wizard twice (results 1a and 2a), omitting 1b/2b. Of course PE and CL of 2b=1/1b. » same way, » (B) for two TEST and single REFERENCE [T1, T2, R]: if we select R as standard, hypo will be: (1) R=T1 and (2)R=T2 Yes this is correct; in this case only one run of the BE wizard is necessary, getting: (1a) T_{1} vs. R, and (1b) T_{2} vs. R. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Irene_I ☆ Indonesia, 20180607 11:09 @ Helmut Posting: # 18861 Views: 5,692 

Dear Helmut, I'm currently looking for statistics methods for William Design in Bioequivalence Study and I found this thread. I tried to download the SAS code from the link you've attached, but apparently the link doesn't work for now. » Please try it again (it worked yesterday, and today as well): CRC's website » Having downloaded the file C5300.zip, unzip it keeping folder's structure; in folder [chapter4] you will find: » Example4.5.sas (the SAS code) » Example45.SSC (the S+ code) » exam45.dat (the raw data for S+) I know that this topic was discussed years ago, but I think I really need to download these files for my further understanding about this topic. I would really appreciate if you could help me with this. Thank you. Best Regards, Irene I 
Helmut ★★★ Vienna, Austria, 20180607 13:04 @ Irene_I Posting: # 18862 Views: 5,692 

Hi Irene, » I know that this topic was discussed years ago, but I think I really need to download these files for my further understanding about this topic. I would really appreciate if you could help me with this. The files of the 2005 edition are gone. However, here is the link for the 2016 edition. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Irene_I ☆ Indonesia, 20180608 11:09 @ Helmut Posting: # 18869 Views: 5,656 

Dear Helmut, Thanks a lot! It is really a great help to me.. Regards, Irene I 
Irene_I ☆ Indonesia, 20180612 09:21 @ Helmut Posting: # 18886 Views: 5,572 

Dear Helmut, Could I ask you some questions about this example? After I tried to run the SAS code (Example 4.5), I got the CI, but there is only one Residual value. I guess I suppose to calculate the intrasubject CV from this value, right? So, My question is, Do we only get one Intrasubject CV from threecomparison of the treatment (TR, ST, SR) while we got three CI (TR, ST, SR)? I am looking forward for your help. Thank you very much. Best Regards, Irene I 
Helmut ★★★ Vienna, Austria, 20180612 12:42 @ Irene_I Posting: # 18888 Views: 5,598 

Hi Irene, » After I tried to run the SAS code (Example 4.5), I got the CI, but there is only one Residual value. I guess I suppose to calculate the intrasubject CV from this value, right? So, My question is, Do we only get one Intrasubject CV from threecomparison of the treatment (TR, ST, SR) while we got three CI (TR, ST, SR)? I’m not equipped with … Hence, my understanding of the code is limited. You are right, in the model you get only one (pooled) estimate of the variance. That’s not a good idea, since it will “work” only if intrasubject variances would be identical and the treatment differences at least very similar. Otherwise, the treatment estimates will be biased and the type I error is not controlled. See also this post. At the 2^{nd} Conference of the Global Bioequivalence Harmonisation Initiative (Rockville, September 2016) Pina D’Angelo gave a presentation “Testing for Bioequivalence in Higher‐Order Crossover Designs: Two‐at‐a‐Time Principle Versus Pooled ANOVA” showing exactly that. Here some of her slides: Purpose
Introduction Statistical Concerns:
Simulated Data: Summary
Closing Remarks
If you are interested in all pairwise comparisons, generate three data sets (values or T&R, S&R, T&S, i.e., exclude all values of the respective other treatment S, T, R) whilst keeping the codes for sequence and period. This will give you data sets which represent an IBD (incomplete block design). Run the usual model on them. This approach is also recommended in the EMA’s BEGL: In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA […], the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question. (my emphasis)Example 4.5 (Phoenix/WinNonlin 8, mixed effects, Satterthwaite’s degrees of freedom): PE (%) 90% CI (%) s²_{w} CV_{w} (%) It’s clear that the variances are not identical. The ones of T/R are smaller than the ones of S/R. If we apply the pooled model the CIs of T/R will be wider than in the IBD model and the CIs of S/R narrower. BTW, I don’t understand what the purpose of the carry variable in Byron’s code is.— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Irene_I ☆ Indonesia, 20180613 05:02 @ Helmut Posting: # 18896 Views: 5,466 

Dear Helmut, Thank you for your explanation. I am sure this will be helpful not only for me, but also others who looking for information about this topic too. Best Regards, Irene I 
Helmut ★★★ Vienna, Austria, 20180613 15:02 @ Irene_I Posting: # 18898 Views: 5,407 

Hi Irene, » Thank you for your explanation. You are welcome. A picture tells more than a thousand words (AUC only). The PE and its CI of T/R are similar. The small bias is caused by the fact that the average PE of S/R and T/S (√141.09×83.12=108.29) is close to the T/R with 116.05. The other comparisons show negative biases. The CI of S/R is narrower in the pooled model and the CI of T/S is wider. We don’t want to go there. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes ★★★ Berlin, Germany, 20180613 15:31 @ Helmut Posting: # 18899 Views: 5,377 

Dear Helmut, » BTW, I don’t understand what the purpose of the carry variable in Byron’s code is.As far as I see it: It serves for nothing because it is not used except in the class statement of Proc Mixed .The class statement serves the purpose of declaring factors even if a variable is numeric. Nothing more. The carry variable seems to me a left over from old times where carryover was an issue.— Regards, Detlew 
ElAlumno ☆ 20190314 23:40 @ Helmut Posting: # 20036 Views: 2,046 

Dear Helmut, In your old but stilluseful post (excerpt below), you describe generating pseudosequences and pseudoperiods when assessing ABE. » A 6×3 design is needed in order to ‘extract’ two 2×2 tables, which are also balanced. Although the full 6×3 table will be used in the analysis of AUC and C_{max}, you will need these 2×2s for the nonparametric analysis of t_{max} (unfortunately there’s no confidence interval based nonparametric method available for more than two formulations/periods). The asterisks * denote pseudosequences and pseudoperiods, e.g. P1* means only that the treatment was given in a period prior to P2* – irrespective of the true study period:Your post preceded the EMA's Guideline on the Investigation of Bioequivalence by a few years, but it appears to agree with their stipulation: In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA, or a four period study including test and reference in fed and fasted states), the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question. I completely understand the use of pseudosequences in this situation. But what about pseudoperiods? The EMA guidelines aren't too specific. On the one hand, if you wish to display your data with pseudosequences, you would want to use pseudoperiods too (so that the timing of treatments is consistent within pseudosequence). On the other hand, the ANOVA doesn't care whether your pseudosequences are logically consistent with your period. If there is no period effect, the difference between using period and pseudoperiod will be minimal (use a few more df in the ANOVA table with period than pseudoperiod). But what if there is a serious period effect? You would miss it (wrong sum of squares for period) by using pseudoperiods. Opinions? 
Helmut ★★★ Vienna, Austria, 20190315 00:47 @ ElAlumno Posting: # 20037 Views: 2,020 

Hi ElAlumno (nice nick!), » In your old but stilluseful post (excerpt below), you describe generating pseudosequences and pseudoperiods when assessing ABE. » » » A 6×3 design is needed in order to ‘extract’ two 2×2 tables, [] » » Your post preceded the EMA's Guideline on the Investigation of Bioequivalence by a few years, but it appears to agree with their stipulation: […] Not quite. » I completely understand the use of pseudosequences in this situation. But what about pseudoperiods? The EMA guidelines aren't too specific. The EMA means the ‘Two‐at‐a‐Time Principle’ (see above) and not my recoding (which I abandoned a good while ago as well). David Brown (MHRA, member the EMA’s Biostatistics Working Party) gave a clarification at the 3^{rd} EGA Symposium on Bioequivalence (London, June 2010). The Q&A document states: However, the treatment, groups, sequences and periods should have their original values maintained in the analysis, and not have the values modified. For example an observation made in period 3 should still be coded as period 3, not have the period changed to “2” because the results for that subject in one of the earlier periods has now be removed. (my emphasis)» On the one hand, if you wish to display your data with pseudosequences, you would want to use pseudoperiods too … Nope. Relevant slides: GL — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElAlumno ☆ 20190322 21:59 @ Helmut Posting: # 20069 Views: 1,622 

Thank you Helmut for your quick and detailed response! I'm glad to hear that pseudoperiods have gone the way of the dodo. I agree with the EMA Biostatistics Working Party that it makes more sense (statistically) to maintain the original codings when leaving out data from irrelevant treatments (not to mention being less prone to human error than recoding). Also, I love their name; I bet the EMA Biostatistics Working Party could form a coalition government with the Slightly Silly Party, the Surprise Party, and the Rent Is Too Damn High Party. But I digress. I don't have WinNonlin, but I tried analyzing Patterson & Jones' Example 4.5 with the Two‐at‐a‐Time Principle using R 1.1.456. My results were similar to the results you posted above, but not as close as I would have expected. I will post the code & table below. Do you see anything I am doing wrong or any obvious explanation of the differences? I don't know how WinNonlin handles missing values, so I tried a few variations (subjects with complete cases in the full dataset, and subjects with complete cases in each of the 3 incompleteblock subsets), but the differences persisted.
# 3treatment design from Patterson & Jones (2017) This produced the following table:
Test PE LCI UCI CV 
Helmut ★★★ Vienna, Austria, 20190323 01:03 @ ElAlumno Posting: # 20070 Views: 1,633 

Hi ElAlumno, » I agree with the EMA Biostatistics Working Party […]. Also, I love their name; I bet the EMA Biostatistics Working Party could form a coalition government with the Slightly Silly Party, the Surprise Party, and the Rent Is Too Damn High Party. It is really existing though David Brown will be brexited (is this already a word?) unceremoniously. » But I digress. That’s fine. Adds some spice to that dry stuff. » […] I tried analyzing Patterson & Jones' Example 4.5 with the Two‐at‐a‐Time Principle using R 1.1.456. What? 1.1.456? 1.1.4 was the first one I used in 2002. Current is 3.5.3! » My results were similar to the results you posted above, but not as close as I would have expected. I will post the code & table below. Do you see anything I am doing wrong or any obvious explanation of the differences? I don't know how WinNonlin handles missing values,… That’s not the point. The devil is in the details. I wrote above: » » » » Phoenix/WinNonlin 8, mixed effects Why? Well, Patterson & Jones analyzed the data with SAS Proc MIXED and I wanted to reproduce it in PHX/WNL (restricted maximum likelihood). OK, they used in SAS ddfm=KENWARDROGER ^{1} and yours truly Satterthwaite’s df.When I switched in PHX/WNL to a fixed effects model (i.e., like lm() in R and SAS Proc GLM ) I got exactly your results. Hence, your code is fine though you can streamline it (specify NAs and classes):
I would not use option(..., digits=4) cause IMHO, this may affect also the precision of internal calculations. Better to round yourself at the end, f.i.
So what do we have (T/R of AUC)?
If Satterthwaite’s df are used, the mixed effects model in SAS and PHX/WNL agree. R lm() agrees with PHX/WNL’s fixed effects model. A little bit surprising to me that Proc MIXED with ddfm=KENWARDROGER^{3} gives the same result as a fixed effects model. OK, but the FDA recommends Satterthwaite’s df for ages.^{4} A mixed effects model may give a lower residual variance – like in this case – because information of incomplete data is recovered. R lmer() is set up according to the EMA’s ‘Method B’ which treats subject as a random effect but is not a true mixed effects model. Given only for completeness (for the code see below). Mixed effects in R are a swamp. The closest I could get was this:
Extracting the required stuff is beyond me. We had many discussions already in the forum and the verdict was “mixed effects models for BE are not possible in R”. So if you are happy with a fixed effect model (many jurisdictions like the EMA), fine. If you have to deal with the FDA or Health Canada, maybe not.
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
mittyri ★★ Russia, 20190323 23:13 @ Helmut Posting: # 20073 Views: 1,518 

Dear Helmut, » A request to the SASians of the forum: Please run Proc MIXED with DDFM=SATTERTHWAITE of the pooled data and both DDFMoptions on the data where S is excluded. Here's my (naive) attempt <...> Results:
Option Label Estimate StdErr DF tValue Probt Alpha Lower Upper pe lowerCL upperCL — Kind regards, Mittyri 
Helmut ★★★ Vienna, Austria, 20190323 23:36 @ mittyri Posting: # 20074 Views: 1,512 

Hi mittyri, » » A request to the SASians of the forum: Please run Proc MIXED with DDFM=SATTERTHWAITE of the pooled data and both DDFMoptions on the data where S is excluded. » Here's my (naive) attempt THX a lot. I added your result to the comparison above. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
mittyri ★★ Russia, 20190323 23:50 @ Helmut Posting: # 20076 Views: 1,508 

Dear Helmut, since you are very quick with answers even Saturday night, I'm not editing my previous post but commenting it here » Extracting the required stuff is beyond me. We had many discussions already in the forum and the verdict was “mixed effects models for BE are not possible in R”. Hmmm... If I remember correctly we were struggling with FDA model where sophisticated 'repeated' statement exists. We successfully crosschecked EMA method B (simple mixed effects model with a Subject as random effect). So linear mixed effects models are possible until FDAstyle is required . — Kind regards, Mittyri 
Helmut ★★★ Vienna, Austria, 20190324 01:23 @ mittyri Posting: # 20077 Views: 1,501 

Hi mittyri, » If I remember correctly we were struggling with FDA model where sophisticated 'repeated' statement exists. » We successfully crosschecked EMA method B (simple mixed effects model with a Subject as random effect). Are you reminding Detlew and me obout our TODOlist? » So linear mixed effects models are possible until FDAstyle is required . Yes, but that’s the point. However, recycled our code: library(lmerTest) # (requires lme4, Matrix) Gives:
Similar ≠ same. Though the DFs are slightly different, the CIs look only identical due to rounding. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 