jag009 Hero NJ, 20130724 17:30 Posting: # 11051 Views: 10,509 

Hi all, I have a question about BE study results... Since both SAS and WinNonlin results are submittable for FDA (EMA?), what if my results are borderline passing the 90% CI with SAS and borderline failing the 90% CI with WinNonlin? Just an example: SAS gives 79.85%  99.93%, while WinNonlin gives 80.15%  100.52%. Thanks John 
Helmut Hero Vienna, Austria, 20130724 17:55 @ jag009 Posting: # 11052 Views: 9,099 

Hi John, » Since both SAS and WinNonlin results are submittable for FDA (EMA?), … EMA: » … what if my results are borderline passing the 90% CI with SAS and borderline failing the 90% CI with WinNonlin? » Just an example: SAS gives 79.85%  99.93%, while WinNonlin gives 80.15%  100.52%. Some ideas:
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
jag009 Hero NJ, 20130724 22:56 @ Helmut Posting: # 11055 Views: 9,051 

Hi Helmut, » Some ideas: » • Conventional 2×2 crossover: Since Phoenix/WinNonlin only speaks REML, you have to exclude incomplete data. Unlike in SAS GLM (which ignores such cases) you have to filter for complete data in Phoenix first. Otherwise you are in mixedeffects model limbo. For EMA you have to treat all effects fixed. In Phoenix that means: Delete subject(sequence) in the variance structure / random1 tab and add it to the fixed effects.» • Higherorder Xover (say two references). Same as above, but for EMA you have to exclude the “uninteresting” treatment, while for FDA I guess (!) you keep all in the model and report only the relevant pair. That’s independent from the software you use. It is a 3way 2 test vs reference study (tada! You guessed it! We have common/pool variance!) Well the data I have is not complete, one subject is missing period 3 (withdrew) but I elected to keep his periods 1 and 2 data since period 2 was the reference arm. I used GLM in SAS and then for my own amusement I ran the data in Phoenix... John 
Helmut Hero Vienna, Austria, 20130724 23:53 @ jag009 Posting: # 11058 Views: 9,019 

Hi John, » It is a 3way 2 test vs reference study (tada! You guessed it! We have common/pool variance!) » Well the data I have is not complete, one subject is missing period 3 (withdrew) but I elected to keep his periods 1 and 2 data since period 2 was the reference arm. I used GLM in SAS and then for my own amusement I ran the data in Phoenix... If you are walking the fun road already: You can feed the incomplete data to PROC MIXED and the other way ’round the complete to PHX. You should see again different results, but in reversed order. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
jag009 Hero NJ, 20130725 00:56 @ Helmut Posting: # 11060 Views: 9,005 

Hi helmut, » If you are walking the fun road already: You can feed the incomplete data to PROC MIXED and the other way ’round the complete to PHX. You should see again different results, but in reversed order. Yup, did that all. I love chaos and to confuse my direct reports... Here is question though but I think I might have asked you (and others) before. With a three way xover study, what if the 1st formulation fails on 90% CI due to the pool variance being inflated by the 2nd formulation? Can I file something to the agency to have them to reconsider the fact that the 1st formulation actually passes BE if the study was conducted as a two way? How? By collapsing the data into a 2way study while preserving the order of the T and R treatments are per sequence (i.e., ABC → CB, BAC → AC, CBA → CA etc etc) and run stats? Or remove the 2nd formulation data completely and run stats? So that the width of the 90% CI is attributed to intrasubject CV from both 1st formulation and reference. Just curious that all. I know I know its a bad thing to do. But I am not altering any data or randomness of the treatments (Am I?). Thanks John 
d_labes Hero Berlin, Germany, 20130725 09:13 @ jag009 Posting: # 11061 Views: 8,975 

Dear John, let me snap in. » ... How? By collapsing the data into a 2way study while preserving the order of the T and R treatments are per sequence (i.e., ABC → CB, BAC → AC, CBA → CA etc etc) and run stats? Or remove the 2nd formulation data completely and run stats? So that the width of the 90% CI is attributed to intrasubject CV from both 1st formulation and reference. If you aimed for an EMA submission you have to do some sort of that. To cite the EMA guideline again (we had this already occasionally discussed here, also by yourself!): "In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA, or a four period study including test and reference in fed and fasted states), the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question." Practically this means to me: Don't change anything in the dataset. Keep periods and sequences as they are. Only forgot 1/3 of your data per comparison under consideration. I'm not a friend of this crippled approach . But ... On the other hand the full analysis is crucially depending on the assumption of a common error variance for the three treatments. What happens and what are the implications if this assumption is violated is not really clear to me. Is the case of heteroscedastic variances in BE studies an issue? Don't ask me . See a quote of Steven Senn here. — Regards, Detlew 
jag009 Hero NJ, 20130725 15:38 (edited by jag009 on 20130725 15:52) @ d_labes Posting: # 11062 Views: 8,889 

Hi Detlew, » If you aimed for an EMA submission you have to do some sort of that. » To cite the EMA guideline again (we had this already occasionally discussed here, also by yourself!): » "In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA, or a four period study including test and reference in fed and fasted states), the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question." Sorry I was going to mention that in my post but I got carried away But you get my point correct? There is no alteration of the study data or the method of analysis. I can present the study data into two sets of results by removing each of the test formulation data from the dataset (yes keeping the period and sequence as a 3way study) and run stats on each. The point is to show that one formulation actually fails because of the pool variance effect attributed to the high CV from the other formulation and removing the "bad" formulation data results in passing BE. Of course I have multiple parameters (Cmax and AUCs) to deal with... What if one parameter passes and one (or two) fails after doing the split data analysis... We actually spoke with FDA about the pool variance effect (at a conference) and they didn't seem to care. Our question was "Would you raise a question about the results of a 3way '2 formulation vs 1 reference' study because of the pool variance effect?". I wasn't there but if I was then I would have asked "What if the study fails marginally due to pool variance? Can I reevaluate the data by removing each formulation and run stats to show that one formulation actually passes BE?" John 
Helmut Hero Vienna, Austria, 20130810 13:52 @ jag009 Posting: # 11260 Views: 8,724 

Hi John, » We actually spoke with FDA about the pool variance effect (at a conference) and they didn't seem to care. […] I found the abstract of a 15 minutes presentation from the 2004 ENAR Spring Meeting of the IBS. Since it is so short below in all its splendor: TWO AT A TIME? OR ALL AT ONCE? Suppose we have a bioequivalence study with three treatments – A, B, and C – and the objective of the study is to make pairwise comparisons among the treatments. Suppose further that treatment C is different in kind from A and B, so that the assumption of homogeneous variance among the three treatments is questionable. One way to do the analyses, under normality assumptions, is Two at a Time – e.g., to test hypotheses about A and B, use only the data from A and B. Another way is All at Once – include the data from all three treatments in a single analysis, making pairwise comparisons within this analysis. If the assumption of homogeneous variance is correct, the All at Once approach will provide more d.f. for estimating the common variance, resulting in increased power. If the variance of C differs from that of A and B, the All at Once approach may have reduced power or an inflated type I error rate, depending on the direction of the difference in variances. I will attempt to quantify the difference between the two approaches for both the comparison of A to B and the comparison of A or B to C. Both parallel and crossover designs will be considered. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Helmut Hero Vienna, Austria, 20130727 18:30 @ d_labes Posting: # 11076 Views: 8,766 

Dear Detlew, » I'm not a friend of this crippled approach . Me not either. On the other hand, remember your own simulations? » On the other hand the full analysis is crucially depending on the assumption of a common error variance for the three treatments. What happens and what are the implications if this assumption is violated is not really clear to me. » » Is the case of heteroscedastic variances in BE studies an issue? Don't ask me . See a quote of Steven Senn here. Stephen and Andy were taking about study planning, and IMHO they erred. We all know that f.i. generic formulations of PPIs show lower CVs since many references are lousy products (mainly problems with occasionally failing gastroresistant coating). Should we not perform such studies because we expect heteroscedasticity? Doesn’t make sense to me. Although the author of famous PowerTOST is not entirely happy with the underlying maths, the outcome (assuming different CV_{W}s in replicate designs) matches common sense: A reward in terms of power / sample size if CV_{WT} < CV_{WR} and a penalty if CV_{WT} > CV_{WR}.Another story are parallel designs ( ElMaestro). Here we deal with the pooled total variance, which will be influenced by different formulation CVs as well, but to a much lesser extent than in Xovers. Differences between groups end up straight in the treatment comparison. Therefore, it is of paramount importance to standardize studies in a parallel design. Though matched pairs are sometimes used in phase III I don’t know whether such an approach was ever seriously* tried in BE. It would need some a priori knowledge of influential covariates on PK. BMI, bodyweight and/or fat, sex, and renal clearance should perform better than eye’s color.
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
jag009 Hero NJ, 20130727 05:59 @ Helmut Posting: # 11066 Views: 8,757 

Hi Helmut and others, » With the correct coding I never saw different results from SAS and PHX. Is this a hypothetical example or from the ‘real world’? On the other hand different results in higherorder Xovers are possible (again independent from software)... Here is a strange one I just experience. 100% final plasma data from a 3way (2 formulation vs reference) study. Both myself and the CRO used SAS Proc GLM to compute A vs C, B vs C (alpha=0.05 yes). Results? My lower 90% CI for formulation 1 was 80.24, CRO result showed 79.65 at the lower limit of the 90% CI. I checked my individual parameter values with theirs and they all matched! Okay their report only show 2 decimal places and I use 4 decimals. The values are in tenths (? I mean numbers like 20.50, etc, not into hundredths). The difference in arithmetic means are like .5 for some reason. Geometric (least square) means are also off Mines are 31.82 (T), 35.50 (R), Intra CV = 24.44 Theirs are 31.8 (T), 35.55 (R), Intrasub CV = 25.8 ??? My suspicion is the number of decimal places they used. The plasmatime data is 3 decimal places. No time deviation (024 hrs sampling, all in house) John P.S. I tried running WinNonlin and the output confirmed my numbers as well. 
Helmut Hero Vienna, Austria, 20130727 14:53 @ jag009 Posting: # 11075 Views: 8,842 

Hi John, » […] My lower 90% CI for formulation 1 was 80.24, CRO result showed 79.65 at the lower limit of the 90% CI. I checked my individual parameter values with theirs and they all matched! Okay their report only show 2 decimal places and I use 4 decimals. » ??? My suspicion is the number of decimal places they used. The plasmatime data is 3 decimal places. Reasonable assumption. It might well be that – despite concentrations in the report are given with three decimals – the CRO used results in full precision. That’s not uncommon if a CRO runs both the analytical and statistical part of the study or if analytical results are transfer to another site in electronic form (f**ing Excel or SAS transport format). Ask the CRO to repeat the evaluation with data rounded to three decimal places (as given in the report). If they don’t get your results, something is wrong. BTW, data transfer should always be done to the same precision as given in the analytical report. Imagine a case where a study passes with data in full precision and fails with results given in the report. Yes, regulators sometimes (especially if the CI is borderline) recalculate studies. Will ring the bell. In Excel there is an option to export to a CSVfile with “Precision as displayed”. That’s the way to go if you want to avoid discrepancies. » P.S. I tried running WinNonlin and the output confirmed my numbers as well. Pharsight will be happy to read this. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
jag009 Hero NJ, 20130727 22:02 @ Helmut Posting: # 11077 Views: 8,699 

Hi Helmut, The bioanalytical work was performed by another CRO (contract Lab). The datafile I received was the exact copy that the CRO (one who ran the stats) received. I even spoke with the contract lab people and they confirmed that they reported only 3 decimal places and it was an excel file. The other strange part is that my formulation B outcome was exactly the same as what they have (down to the 2nd decimal place for ratios, 90%CI and Least square means). My broadsword is ready for Monday's telecon. John 
mittyri Senior Russia, 20140110 08:05 @ Helmut Posting: # 12153 Views: 7,988 

Hi Helmut and All, Sorry for outofdate question, but I need to clarify one issue. » Some ideas: Conventional 2×2 crossover: Since Phoenix/WinNonlin only speaks REML, you have to exclude incomplete data. Unlike in SAS GLM (which ignores such cases) you have to filter for complete data in Phoenix first. Otherwise you are in mixedeffects model limbo. In the study we have 2 volunteers who were discontinued the study after 1st period (both from same sequence) Do I understand correct, that I shouldn't put the data from these subjects to WNL? I tried both options (with and without them) and recieved unequal CIs — Kind regards, Mittyri 
Helmut Hero Vienna, Austria, 20140110 13:41 @ mittyri Posting: # 12155 Views: 8,085 

Hi mittyri, » » […] Since Phoenix/WinNonlin only speaks REML, you have to exclude incomplete data. […] you have to filter for complete data in Phoenix first. Otherwise you are in mixedeffects model limbo. » » In the study we have 2 volunteers who were discontinued the study after 1st period (both from same sequence) » Do I understand correct, that I shouldn't put the data from these subjects to WNL? Yes. See also EMA’s GL: […] subjects in a crossover trial who do not provide evaluable data for both of the test and reference products […] should not be included. » I tried both options (with and without them) and recieved unequal CIs Correct. PHX/WNL in its default setting (incomplete data included) will recover information of the betweensubject variance (REML estimate of a mixedeffects model). In SASspeak that’s Proc MIXED instead of Proc GLM . In other words, the variance will be differently split into their between, within, and residual components. Therefore, the CI (and CV_{intra}) will be affected as well – though in my experience the difference generally is quite small.See also an example in SAS 9.2 and WinNonlin 5.2.1. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro Hero Denmark, 20130724 18:06 @ jag009 Posting: # 11053 Views: 8,959 

Hi John, apart from what Hötzi just said, I think the issue is of no practical importance since you will submit the data analysed in accordance with the protocol and SAP, which in turn specify the software and mode of analysis — if (3) 4 Best regards, ElMaestro "(...) targeted cancer therapies will benefit fewer than 2 percent of the cancer patients they’re aimed at. That reality is often lost on consumers, who are being fed a steady diet of winning anecdotes about miracle cures." New York Times (ed.), June 9, 2018. 
jag009 Hero NJ, 20130724 22:59 @ ElMaestro Posting: # 11056 Views: 8,969 

Hi ElMaestro, » ...in accordance with the protocol and SAP, which in turn specify the software and mode of analysis I see! Thanks! PS. Oh get this. There was one CRO who used both WinNonlin and SAS for PK and stats. WinNonlin to compute the PK parameters, SAS to do the stats. John 