Log in
Register|
jag009 ★★★ NJ, 2013-07-24 19:30 (4289 d 18:29 ago) Posting: # 11051 Views: 21,103 |
|
|
Hi all, I have a question about BE study results... Since both SAS and WinNonlin results are submittable for FDA (EMA?), what if my results are borderline passing the 90% CI with SAS and borderline failing the 90% CI with WinNonlin? Just an example: SAS gives 79.85% - 99.93%, while WinNonlin gives 80.15% - 100.52%. Thanks John |
|
Helmut ★★★   Vienna, Austria, 2013-07-24 19:55 (4289 d 18:04 ago) @ jag009 Posting: # 11052 Views: 18,294 |
|
|
Hi John, ❝ Since both SAS and WinNonlin results are submittable for FDA (EMA?), … EMA: ![[image]](img/uploaded/image28.png) ❝ … what if my results are borderline passing the 90% CI with SAS and borderline failing the 90% CI with WinNonlin? ❝ Just an example: SAS gives 79.85% - 99.93%, while WinNonlin gives 80.15% - 100.52%. Some ideas:
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! ![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png) Helmut Schütz ![[image]](https://static.bebac.at/img/CC by.png) The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
jag009 ★★★ NJ, 2013-07-25 00:56 (4289 d 13:03 ago) @ Helmut Posting: # 11055 Views: 18,126 |
|
|
Hi Helmut, ❝ Some ideas: ❝ • Conventional 2×2 cross-over: Since Phoenix/WinNonlin only speaks REML, you have to exclude incomplete data. Unlike in SAS GLM (which ignores such cases) you have to filter for complete data in Phoenix first. Otherwise you are in mixed-effects model limbo. For the EMA you have to treat all effects fixed. In Phoenix that means: Delete ❝ • Higher-order Xover (say two references). Same as above, but for the EMA you have to exclude the “uninteresting” treatment, while for the FDA I guess (!) you keep all in the model and report only the relevant pair. That’s independent from the software you use. It is a 3-way 2 test vs reference study (tada! You guessed it! We have common/pool variance!) Well the data I have is not complete, one subject is missing period 3 (withdrew) but I elected to keep his periods 1 and 2 data since period 2 was the reference arm. I used GLM in SAS and then for my own amusement I ran the data in Phoenix... John |
|
Helmut ★★★ Vienna, Austria, 2013-07-25 01:53 (4289 d 12:06 ago) @ jag009 Posting: # 11058 Views: 17,894 |
|
|
Hi John, ❝ It is a 3-way 2 test vs reference study (tada! You guessed it! We have common/pool variance!) ❝ Well the data I have is not complete, one subject is missing period 3 (withdrew) but I elected to keep his periods 1 and 2 data since period 2 was the reference arm. I used GLM in SAS and then for my own amusement I ran the data in Phoenix... If you are walking the fun road already: You can feed the incomplete data to PROC MIXED and the other way ’round the complete to PHX. You should see again different results, but in reversed order. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
jag009 ★★★ NJ, 2013-07-25 02:56 (4289 d 11:03 ago) @ Helmut Posting: # 11060 Views: 17,957 |
|
|
Hi helmut, ❝ If you are walking the fun road already: You can feed the incomplete data to PROC MIXED and the other way ’round the complete to PHX. You should see again different results, but in reversed order. Yup, did that all. I love chaos and to confuse my direct reports... Here is question though but I think I might have asked you (and others) before. With a three way x-over study, what if the 1st formulation fails on 90% CI due to the pool variance being inflated by the 2nd formulation? Can I file something to the agency to have them to re-consider the fact that the 1st formulation actually passes BE if the study was conducted as a two way? How? By collapsing the data into a 2-way study while preserving the order of the T and R treatments are per sequence (i.e., ABC → CB, BAC → AC, CBA → CA etc etc) and run stats? Or remove the 2nd formulation data completely and run stats? So that the width of the 90% CI is attributed to intrasubject CV from both 1st formulation and reference. Just curious that all. I know I know its a bad thing to do. But I am not altering any data or randomness of the treatments (Am I?). Thanks John |
|
d_labes ★★★ Berlin, Germany, 2013-07-25 11:13 (4289 d 02:46 ago) @ jag009 Posting: # 11061 Views: 17,879 |
|
|
Dear John, let me snap in. ❝ ... How? By collapsing the data into a 2-way study while preserving the order of the T and R treatments are per sequence (i.e., ABC → CB, BAC → AC, CBA → CA etc etc) and run stats? Or remove the 2nd formulation data completely and run stats? So that the width of the 90% CI is attributed to intrasubject CV from both 1st formulation and reference. If you aimed for an EMA submission you have to do some sort of that. To cite the EMA guideline again (we had this already occasionally discussed here, also by yourself!): "In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA, or a four period study including test and reference in fed and fasted states), the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question." Practically this means to me: Don't change anything in the dataset. Keep periods and sequences as they are. Only forgot 1/3 of your data per comparison under consideration. I'm not a friend of this crippled approach  . But ... On the other hand the full analysis is crucially depending on the assumption of a common error variance for the three treatments. What happens and what are the implications if this assumption is violated is not really clear to me. Is the case of heteroscedastic variances in BE studies an issue? Don't ask me  . See a quote of Steven Senn here.— Regards, Detlew |
|
jag009 ★★★ NJ, 2013-07-25 17:38 (4288 d 20:21 ago) @ d_labes Posting: # 11062 Views: 17,856 |
|
|
Hi Detlew, ❝ If you aimed for an EMA submission you have to do some sort of that. ❝ To cite the EMA guideline again (we had this already occasionally discussed here, also by yourself!): ❝ "In studies with more than two treatment arms (e.g. a three period study including two references, one from EU and another from USA, or a four period study including test and reference in fed and fasted states), the analysis for each comparison should be conducted excluding the data from the treatments that are not relevant for the comparison in question." Sorry I was going to mention that in my post but I got carried away  But you get my point correct? There is no alteration of the study data or the method of analysis. I can present the study data into two sets of results by removing each of the test formulation data from the dataset (yes keeping the period and sequence as a 3-way study) and run stats on each. The point is to show that one formulation actually fails because of the pool variance effect attributed to the high CV from the other formulation and removing the "bad" formulation data results in passing BE. Of course I have multiple parameters (Cmax and AUCs) to deal with... What if one parameter passes and one (or two) fails after doing the split data analysis... We actually spoke with FDA about the pool variance effect (at a conference) and they didn't seem to care. Our question was "Would you raise a question about the results of a 3-way '2 formulation vs 1 reference' study because of the pool variance effect?". I wasn't there but if I was then I would have asked "What if the study fails marginally due to pool variance? Can I re-evaluate the data by removing each formulation and run stats to show that one formulation actually passes BE?" John |
|
Helmut ★★★ Vienna, Austria, 2013-08-10 15:52 (4272 d 22:07 ago) @ jag009 Posting: # 11260 Views: 18,176 |
|
|
Hi John, ❝ We actually spoke with FDA about the pool variance effect (at a conference) and they didn't seem to care. […] I found the abstract of a 15 minutes presentation from the 2004 ENAR Spring Meeting of the IBS. Since it is so short below in all its splendor: TWO AT A TIME? OR ALL AT ONCE? Suppose we have a bioequivalence study with three treatments – A, B, and C – and the objective of the study is to make pairwise comparisons among the treatments. Suppose further that treatment C is different in kind from A and B, so that the assumption of homogeneous variance among the three treatments is questionable. One way to do the analyses, under normality assumptions, is Two at a Time – e.g., to test hypotheses about A and B, use only the data from A and B. Another way is All at Once – include the data from all three treatments in a single analysis, making pairwise comparisons within this analysis. If the assumption of homogeneous variance is correct, the All at Once approach will provide more d.f. for estimating the common variance, resulting in increased power. If the variance of C differs from that of A and B, the All at Once approach may have reduced power or an inflated type I error rate, depending on the direction of the difference in variances. I will attempt to quantify the difference between the two approaches for both the comparison of A to B and the comparison of A or B to C. Both parallel and crossover designs will be considered. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
Helmut ★★★ Vienna, Austria, 2013-07-27 20:30 (4286 d 17:29 ago) @ d_labes Posting: # 11076 Views: 17,700 |
|
|
Dear Detlew, ❝ I'm not a friend of this crippled approach Me not either. On the other hand, remember your own simulations? ❝ On the other hand the full analysis is crucially depending on the assumption of a common error variance for the three treatments. What happens and what are the implications if this assumption is violated is not really clear to me. ❝ ❝ Is the case of heteroscedastic variances in BE studies an issue? Don't ask me Stephen and Andy were taking about study planning and IMHO, they erred. We all know that f.i. generic formulations of PPIs show lower CVs since many references are lousy products (mainly problems with occasionally failing gastroresistant coating). Should we not perform such studies because we expect heteroscedasticity? Doesn’t make sense to me. Although the author of famous PowerTOST is not entirely happy with the underlying maths, the outcome (assuming different CVWs in replicate designs) matches common sense: A reward in terms of power / sample size if CVWT < CVWR and a penalty if CVWT > CVWR.Another story are parallel designs (  ElMaestro). Here we deal with the pooled total variance, which will be influenced by different formulation CVs as well, but to a much lesser extent than in Xovers. Differences between groups end up straight in the treatment comparison. Therefore, it is of paramount importance to standardize studies in a parallel design. Though matched pairs are sometimes used in phase III I don’t know whether such an approach was ever seriously* tried in BE. It would need some a priori knowledge of influential covariates on PK. BMI, body-weight and/or -fat, sex, and renal clearance should perform better than eye’s color. ![[image]](https://upload.wikimedia.org/wikipedia/commons/0/04/Miniauge2.gif)
— Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
jag009 ★★★ NJ, 2013-07-27 07:59 (4287 d 06:00 ago) @ Helmut Posting: # 11066 Views: 17,801 |
|
|
Hi Helmut and others, ❝ With the correct coding I never saw different results from SAS and PHX. Is this a hypothetical example or from the ‘real world’? On the other hand different results in higher-order Xovers are possible (again independent from software)... Here is a strange one I just experience. 100% final plasma data from a 3-way (2 formulation vs reference) study. Both myself and the CRO used SAS Proc GLM to compute A vs C, B vs C (alpha=0.05 yes). Results? My lower 90% CI for formulation 1 was 80.24, CRO result showed 79.65 at the lower limit of the 90% CI. I checked my individual parameter values with theirs and they all matched! Okay their report only show 2 decimal places and I use 4 decimals. The values are in tenths (? I mean numbers like 20.50, etc, not into hundredths). The difference in arithmetic means are like .5 for some reason. Geometric (least square) means are also off Mines are 31.82 (T), 35.50 (R), Intra CV = 24.44 Theirs are 31.8 (T), 35.55 (R), Intrasub CV = 25.8 ??? My suspicion is the number of decimal places they used. The plasma-time data is 3 decimal places. No time deviation (0-24 hrs sampling, all in house) John P.S. I tried running WinNonlin and the output confirmed my numbers as well. |
|
Helmut ★★★ Vienna, Austria, 2013-07-27 16:53 (4286 d 21:06 ago) @ jag009 Posting: # 11075 Views: 17,795 |
|
|
Hi John, ❝ […] My lower 90% CI for formulation 1 was 80.24, CRO result showed 79.65 at the lower limit of the 90% CI. I checked my individual parameter values with theirs and they all matched! Okay their report only show 2 decimal places and I use 4 decimals. ❝ ??? My suspicion is the number of decimal places they used. The plasma-time data is 3 decimal places. Reasonable assumption. It might well be that – despite concentrations in the report are given with three decimals – the CRO used results in full precision. That’s not uncommon if a CRO runs both the analytical and statistical part of the study or if analytical results are transfer to another site in electronic form (f**ing Excel or SAS transport format). Ask the CRO to repeat the evaluation with data rounded to three decimal places (as given in the report). If they don’t get your results, something is wrong. BTW, data transfer should always be done to the same precision as given in the analytical report. Imagine a case where a study passes with data in full precision and fails with results given in the report. Yes, regulators sometimes (especially if the CI is borderline) recalculate studies. Will ring the bell. In Excel there is an option to export to a CSV-file with “Precision as displayed”. That’s the way to go if you want to avoid discrepancies. ❝ P.S. I tried running WinNonlin and the output confirmed my numbers as well. Pharsight will be happy to read this.  — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
jag009 ★★★ NJ, 2013-07-28 00:02 (4286 d 13:57 ago) @ Helmut Posting: # 11077 Views: 17,558 |
|
|
Hi Helmut, The bioanalytical work was performed by another CRO (contract Lab). The datafile I received was the exact copy that the CRO (one who ran the stats) received. I even spoke with the contract lab people and they confirmed that they reported only 3 decimal places and it was an excel file. The other strange part is that my formulation B outcome was exactly the same as what they have (down to the 2nd decimal place for ratios, 90%CI and Least square means). My broadsword is ready for Monday's telecon.  John |
|
mittyri ★★ Russia, 2014-01-10 09:05 (4120 d 03:54 ago) @ Helmut Posting: # 12153 Views: 16,841 |
|
|
Hi Helmut and All, Sorry for out-of-date question, but I need to clarify one issue. ❝ Some ideas: Conventional 2×2 cross-over: Since Phoenix/WinNonlin only speaks REML, you have to exclude incomplete data. Unlike in SAS GLM (which ignores such cases) you have to filter for complete data in Phoenix first. Otherwise you are in mixed-effects model limbo. In the study we have 2 volunteers who were discontinued the study after 1st period (both from same sequence) Do I understand correct, that I shouldn't put the data from these subjects to WNL? I tried both options (with and without them) and recieved unequal CIs — Kind regards, Mittyri |
|
Helmut ★★★ Vienna, Austria, 2014-01-10 14:41 (4119 d 22:18 ago) @ mittyri Posting: # 12155 Views: 17,287 |
|
|
Hi mittyri, ❝ ❝ […] Since Phoenix/WinNonlin only speaks REML, you have to exclude incomplete data. […] you have to filter for complete data in Phoenix first. Otherwise you are in mixed-effects model limbo. ❝ ❝ In the study we have 2 volunteers who were discontinued the study after 1st period (both from same sequence) ❝ Do I understand correct, that I shouldn't put the data from these subjects to WNL? Yes. See also the EMA’s GL: […] subjects in a crossover trial who do not provide evaluable data for both of the test and reference products […] should not be included. ❝ I tried both options (with and without them) and recieved unequal CIs Correct. PHX/WNL in its default setting (incomplete data included) will recover information of the between-subject variance (REML estimate of a mixed-effects model). In SAS-speak that’s Proc MIXED instead of Proc GLM. In other words, the variance will be differently split into their between-, within-, and residual components. Therefore, the CI (and CVintra) will be affected as well – though in my experience the difference generally is quite small.See also an example in SAS 9.2 and WinNonlin 5.2.1. — Dif-tor heh smusma 🖖🏼 Довге життя Україна! Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. 🚮 Science Quotes |
|
ElMaestro ★★★ Denmark, 2013-07-24 20:06 (4289 d 17:53 ago) @ jag009 Posting: # 11053 Views: 18,133 |
|
|
Hi John, apart from what Hötzi just said, I think the issue is of no practical importance since you will submit the data analysed in accordance with the protocol and SAP, which in turn specify the software and mode of analysis  — Pass or fail! ElMaestro |
|
jag009 ★★★ NJ, 2013-07-25 00:59 (4289 d 13:00 ago) @ ElMaestro Posting: # 11056 Views: 17,890 |
|
|
Hi ElMaestro, ❝ ...in accordance with the protocol and SAP, which in turn specify the software and mode of analysis I see! Thanks! PS. Oh get this. There was one CRO who used both WinNonlin and SAS for PK and stats. WinNonlin to compute the PK parameters, SAS to do the stats. John |
Ing. Helmut Schütz