Helmut Hero Vienna, Austria, 20100622 18:53 Posting: # 5551 Views: 19,345 

Dear all, according to rumours at the recent Workshop in Budapest (235 participants!) EMA started an initiative involving the PKDrafting Group and the Biostatistics Drafting Group of EWP to clarify/correct some open issues of the new BEGL. Points under discussion areWe can expect not only SAScode, but example datasets in order to validate other software. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20100623 10:09 @ Helmut Posting: # 5552 Views: 17,509 

Dear Helmut, seems this are good news. Hopefully the results are not according to:
— Regards, Detlew 
Helmut Hero Vienna, Austria, 20100623 10:11 @ d_labes Posting: # 5553 Views: 17,545 

Dear D. Labes! » seems this are good news. Well, let’s see and cross our fingers. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Helmut Hero Vienna, Austria, 20110205 18:48 @ Helmut Posting: # 6562 Views: 17,036 

Dear all, the document is expected to be published in midFebruary 2011. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Helmut Hero Vienna, Austria, 20110316 13:44 @ Helmut Posting: # 6762 Views: 16,859 

Dear all, the clarification is part of the recent version of the “Questions & Answers: Positions on specific questions addressed to the Pharmacokinetics Working Party” EMA/618604/2000 Rev. 3, dated 26 January 2011, published 14 March 2011. The interesting part is Section 11. Clarification on the recommended statistical method for the analysis of a bioequivalence study (pages 21–32). — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
ElMaestro Hero Denmark, 20110316 14:20 @ Helmut Posting: # 6763 Views: 16,740 

Hehe, A simple linear mixed model, which assumes identical withinsubject variability (Method B), may be acceptable as long as results obtained with the two methods do not lead to different regulatory decisions. However, in borderline cases and when there are many included subjects who only provide data for a subset of the treatment periods, additional analysis using method A might be required. At the time of protocol writing you do not know if there will be data gaps. Therefore, you cannot write which evaluation method you will ultimately be using. This means (??) that one has to apply both methods, check if the BE conclusions are the same and then, in case there are differences, discard a method which gives unbiased variance estimates. An advantage of Method C is that it directly calculates s^{2}_{wr} However, sometimes the algorithm fails to converge. No, the Al Gore Rhythm will converge if the guy doing the statistics knows what he is doing in terms of controlling the inivalues and other optimizer settings (flat multidimensional likelihood surfaces are hypothetical). 
Helmut Hero Vienna, Austria, 20110317 04:23 @ Helmut Posting: # 6766 Views: 16,919 

Dear all, I could reproduce EMA’s results in Phoenix. Interesting points for Dataset II:
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20110317 11:00 (edited by d_labes on 20110317 14:06) @ Helmut Posting: # 6770 Views: 16,744 

Dear All, that makes me dumbfound! Seems the ANOVA fraction of EMA statisticians has triumphed all along the line. Thus our crossing fingers was of no effect. To summarize my understanding of this socalled "clarification":
The rationale behind that all I can't and will not discuss seriously . I thank my God that I'm only a quantumtheoretical chemist educationally and not a statistician. Thus I must not understand . BTW: The CV_{WT} of Method C from SAS for dataset II is 3.87%. Proc MIXED is complaining: The Mixed Procedure This is a strong sign of an overspecified model. That may be one of the sources of wider CIs compared to the simpler models. — Regards, Detlew 
Helmut Hero Vienna, Austria, 20110319 02:59 @ d_labes Posting: # 6779 Views: 16,684 

Bless you, sir, and all your house, unto the seventh generation! You are a most noble and shining example of all that's right and good and true here. » Do you think I have got their points? Yes. Exactly. Edit: Data Set I is funny. Some subjects have missing data in one period, but data in a subsequent one (e.g., subject 11’s third period is missing). Obviously not data from the ‘real world’. Did you have a look at box plots and/or QQplots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)? Quote from last June’s Q&A: On a case by case basis, a study could be acceptable if the bioequivalence requirements are met both including the outlier subject (using the scaled average bioequivalence approach and the withinsubject CV with this subject) and after exclusion of the outlier (using the withinsubject CV without this subject). Excluding subjects 45 and 52 CV_{WR} (EMA’s method) drops from 47.0% to 32.2%. Maybe that’s a hidden trick question to us:
Scaled AR width The 90% CIs of the full data set of 107.11  124.89 (Method A) and 107.17  124.97 (Method B) are also within the narrower scaled AR – but it’s a pretty close shave!— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20110324 11:55 @ Helmut Posting: # 6803 Views: 16,371 

Dear Helmut, » "Bless you, sir, and all your house, unto the seventh generation! » You are a most noble and shining example of all that's right and good and true here." This quote, or is it your own poetry, is really resistant. First time that I noticed the Net is not omniscient! » ... Did you have a look at box plots and/or QQplots of residuals? » What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)? Which residuals did you analyse? Which model? Please enlighten me. What do you think: which type of boxplot should we apply? Best for our sponsors would be a simple boxplot with whiskers up to minimum/maximum not showing any 'outlier' . — Regards, Detlew 
Helmut Hero Vienna, Austria, 20110324 15:09 @ d_labes Posting: # 6806 Views: 16,623 

Dear D. Labes! » This quote, or is it your own poetry, … I borrowed it from a personal discussion page at Wikipedia. » … is really google resistant. First time that I noticed the Net is not omniscient! Well, personal discussion pages of WP are blocked from being indexed by Google. I have blocked some pages (i.e., the download section and the latest posts) from Google as well. Requires two lines in the domain’s robots.txt …Useragent: * … and adding rel="nofollow" to respective links within other pages (the same is done here; look at the HTML code).» » … Did you have a look at box plots and/or QQplots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)? » » Which residuals did you analyse? Which model? Please enlighten me. I used EMA’s crippled model (test removed), Sequence+Subject(Sequence)+Period and threw away one period’s residuals (same values, but different in signs to the respective other one).» What do you think: which type of boxplot should we apply? Best for our sponsors would be a simple boxplot with whiskers up to minimum/maximum not showing any 'outlier' . I have chosen ±3×IQR following the convention (!) that values within 1.53×IQR are ‘mild’ outliers and outside 3×IQR are ‘severe’ outliers. I’m exploring full replicate datasets right now – outliers almost in all of them. Don’t know whether residuals make any sense at all (see also this post: period ratios instead?). I would end up with different numbers of outliers, depending on the method of calculation of the IQR (see Rmanual). Period ratios (second / first administration): Type 3: SAS according to Rdoc  2 outliers (#46: 3.524, #45: 26.08) Type 5: or is this SAS?  2 outliers Type 6: Minitab, SPSS, Phoenix/WinNonlin  2 outliers Type 7: default in S, R  3 outliers (as above + #13: 3.349) CVWR Scaled AR width Don’t know what to do. Suggestions? Maybe Q&A is an abbreviation for Questions and Ambiguities. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20110328 14:16 @ Helmut Posting: # 6822 Views: 16,261 

Dear Helmut, » Period ratios (second / first administration): » Type 3: SAS according to Rdoc  2 outliers (#46: 3.524, #45: 26.08) » Type 5: or is this SAS?  2 outliers » Type 6: Minitab, SPSS, Phoenix/WinNonlin  2 outliers » ... SAS has 5 different percentile definitions. The default is: Let n*p=j+g where j is the integer part, g is the fractional part, n is the number of values, x the ordered values. Let y denote the percentile. Then (SAS PCTLDEF=5)
y = 0.5*(x_{j}+x_{j+1}) if g=0 This corresponds to R's Type 2 I think. » Don’t know what to do. Suggestions? If the 'outlier' considerations based on the crippled EMA model makes any sense at all, which I'm not convinced at all , I would vote for an analysis of the residuals, not the period ratios also they appeared a natural choice on the first view. Pro's:
To add more ambiguities:
— Regards, Detlew 
d_labes Hero Berlin, Germany, 20110404 08:53 @ Helmut Posting: # 6855 Views: 16,496 

Dear Helmut, dear All! FYI: Robert Schall, Laszlo Endrenyi, Arne Ring Residuals and Outliers in Replicate Design Crossover Studies Journal of Biopharmaceutical Statistics, 20: 4, 835 — 849 Online accessible here as preprint or here. IMHO this fits excellent into the framework of scaled ABE evaluated via suitable intrasubject contrasts as outlined in the Progesterone guidance . — Regards, Detlew 
ElMag Junior 20110324 12:45 (edited by ElMag on 20110324 14:44) @ Helmut Posting: # 6805 Views: 16,511 

» Did you have a look at box plots and/or QQplots of residuals? What about subjects 45 and 52 (studentized residuals outside ±1.96 and outside 3×IQR)? Hello there!! Can you tell me if you have created the boxplots based on the ratio of Cmax between the two periods? If not, please indicate which data have you selected and how did you use them in order to create the boxplot. Thank you! 
Helmut Hero Vienna, Austria, 20110324 16:50 @ ElMag Posting: # 6808 Views: 16,384 

— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Priyanka_S Junior 20110321 14:28 @ Helmut Posting: # 6789 Views: 16,578 

Dear all, We tried the SAS code given by EMA for replicated designs by using method A with data set II (three period data). But it is showing the following error. Please clarify this. WARNING: ADJUST=T implies no adjustment for simultaneous inference. — Best Regards Priyanka S 
d_labes Hero Berlin, Germany, 20110321 16:08 (edited by d_labes on 20110321 16:21) @ Priyanka_S Posting: # 6790 Views: 17,369 

Dear Priyanka, » ... But it is showing the following error ... » WARNING: ADJUST=T implies no adjustment for simultaneous inference. as the log states this is not an ERROR:, but a WARNING:. And this warning is nonsense in the context. You request with the option ADJUST=T confidence intervals using the tquantile without eventually necessary multiplicity adjustments. But multiplicity adjustment is not necessary in comparing only two possible LSMeans. It would be eventually necessary if there are more than two. Proc GLM had could that figured out, but SAS in his great wisdom had decided nevertheless to throw a warning. Note that this warning disappears if you omit the ADJUST=T option in the LSMeans statement. Doing so Proc GLM reverts to the default method for CI calculation: confidence intervals using the tquantile without eventually necessary multiplicity adjustments, i.e. the same as with ADJUST=T , but without a warning. What a stroke of genius .This annoying warningophylistic behaviour is notorious in SAS 9.2. You can observe it in numerous instances. Check if SAS does what you intend and forget the warnings if it does. BTW: The WARNING: has nothing to do with the data set used. You will get it also with data set I. — Regards, Detlew 
Helmut Hero Vienna, Austria, 20110321 22:38 @ d_labes Posting: # 6791 Views: 16,529 

Dear D. Labes! I'm not gifted with _{}, but I'm wondering how EMA got their results for 'Method C' with this statement: estimate 'testref' formulation 1 1/ CL alpha=0.10; @ Priyanka: If you want to repeat results from any type of software, first use the code exactly as it is given. Only if your results don't match, try to modify your code (sometimes published code contains typos...). ADJUST=T is not stated by FDA (2001), EMA's Q&A (2011), and FDA's Progesterone Guidance (2010, 2011).— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
d_labes Hero Berlin, Germany, 20110322 09:24 @ Helmut Posting: # 6794 Views: 16,583 

Dear Helmut! » ... but I'm wondering how EMA got their results for 'Method C' with this statement: estimate 'testref' formulation 1 1/ CL alpha=0.10; This is one of the hidden gems in SAS coding . But it is correct if you use the codes 'R' and 'T' for the formulations. SAS orders them lexically 'R' coming first and expects the coefficients for the difference in the estimate statement in that order. » @ Priyanka: » ... ADJUST=T is not stated by FDA (2001), EMA's Q&A (2011), and FDA's Progesterone Guidance (2010, 2011).Here the Great Admin err's. It is stated in EMA's Q&A Method 1:
proc glm data=replicate; But as I said above, it is not necessary here. Also the pdiff=control("R") is not necessary. But it does here the job of ordering T first and thus giving the LSMeans difference TR because again the LSMean of 'R' comes first and the difference 12 is calculated. Moreover the whole lsmeans statement is superfluous if you code the option /CLparm alpha=0.1 in the model statement. Then the estimate statement will give the CI like Proc MIXED does as default.BTW: I'm not sure if the test of the sequence effect as coded from Great Oracle EMA is appropriate in case of missings. Using the 'Capt'n EM calls me bogus' Random statement I get a mixture of MS_{error} and MS_{subject(sequence)} as denominator of the corresponding Ftest. The degrees of freedom are also adapted according to Satterthwaite.BTW2: The code for obtaining the intrasubject variance taken literally will bring us directly to the Type III hell. Output (I have named their DATA as AUC):  GLMANOVA Analysis of REF. withinsubject var. for log(AUC)  As you see: Subtleties and flaws in coding, questions over questions after that 'Clarification'. But what could we expect other if considering the scientific foundation of that all? — Regards, Detlew 
Helmut Hero Vienna, Austria, 20110327 20:35 @ d_labes Posting: # 6818 Views: 16,310 

Dear D. Labes! » Source DF Type III SS Mean Square [...] » » sequence 0 0.0000000 . [...] » [...] Zero degrees of freedom is not a lot. I love SAS‘s ‘.’  Phoenix/WinNonlin spits out ‘Not estimable’, but I can trim it to ‘.’ or even ‘0’… Open issues IMHO:
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
The user Junior Russia, 20170314 10:16 (edited by The user on 20170314 12:44) @ Helmut Posting: # 17151 Views: 5,988 

» I love SAS‘s ‘.’  Phoenix/WinNonlin spits out ‘Not estimable’, but I can trim it to ‘.’ or even ‘0’… Dear Helmut, I decided to continue the topic (moreover Simon kindly directed me here). Regarding "Not estimable". Could you please share your expirience with following issue? When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapears. Comments: it is BE cross over study in 3 groups and all effects are fixed. BR 
ElMaestro Hero Denmark, 20170314 11:02 @ The user Posting: # 17152 Views: 5,989 

Hi The user, » I decided to continue the topic (moreover Simon kindly directed me here). » Regarding "Not estimable". Could you please share your expirience with following issue? When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapeares. Comments: it is BE cross over study in 3 groups and all effects are fixed. Hmmmm... thinking loud here... In a type III model Group will have in your case up to three columns in the model matrix, but every column is equally represented by the subjects in the respective groups. Hence Group has zero df's and no effect for any level of Group can be estimated. Try a type I model with Group before Subjects, then you'll get something that is estimable, I believe. — “A tenyear, doubleblind study from the Mayo Clinic concluded that even in late stages of dementia, the last to go is the lobe of the brain in charge of cafeteria layout.” (Serge Storms/Tim Dorsey). Best regards, ElMaestro  Bootstrapping is a relatively new hobby of mine. I am only 30 years late to the party. 
The user Junior Russia, 20170314 12:13 @ ElMaestro Posting: # 17154 Views: 5,944 

» Try a type I model with Group before Subjects, then you'll get something that is estimable, I believe. Thank you for your reply. Sorry, but I did not catch the idea. Should I try this model: Group+Patient(Sequence)+Sequence+Period+Treatment? One more comment: the groups are unbalanced. Type I is suiatable for balanced groups as I undertood. Looks like I could not use the Type I model. 
ElMaestro Hero Denmark, 20170314 12:41 @ The user Posting: # 17155 Views: 5,974 

Hi, » Sorry, but I did not catch the idea. Should I try this model: Group+Patient(Sequence)+Sequence+Period+Treatment? Yes try that with type I. I think type III may give the same as before if you are not using SAS. » One more comment: the groups are unbalanced. Type I is suiatable for balanced groups as I undertood. Looks like I could not use the Type I model. Your CI will be the same. Type I vs Type III is generally a topic that is of a much more sensitive nature to some people than e.g.religion or venereal diseases. People who grew up with SAS stick to type III, and type III only, because that is all they know and therefore they seem to be resistant to common sense. Besides, SAS invented the term "Least Squares Means" and that sounds so good that no reasonable alternative could ever exist, right? Type I is not better or worse than type III. LS Means are no better than model effects. Depending on contrasts, model effects are LS Means and vice versa. And so forth... — “A tenyear, doubleblind study from the Mayo Clinic concluded that even in late stages of dementia, the last to go is the lobe of the brain in charge of cafeteria layout.” (Serge Storms/Tim Dorsey). Best regards, ElMaestro  Bootstrapping is a relatively new hobby of mine. I am only 30 years late to the party. 
Helmut Hero Vienna, Austria, 20170318 21:59 @ The user Posting: # 17164 Views: 5,851 

Hi BR, » When I use such a model: Sequence+Period+Treatment+Group+Patient(Sequence) I get "Not estimable" in the results of ABE in WNL. Model is wrong? When I delete the "group" than error dissapears. Comments: it is BE cross over study in 3 groups and all effects are fixed. As Simon noted already at Certara’s Forum, its the fixed effects model – not the software. Even without the groupterm (i.e., the EMA’s sequence, subject(sequence), period, formulation) you will get an endless list of “Not estimables” simply because the requested combination does not exist in the data set. Example: All subjects are uniquely coded (1, 2, …, n) and subject 1 is in sequence RT. You will get an estimate. Fine. But the model tries to estimate subject 1 in sequence TR as well. Not estimable! That’s correct because a datum with such a coding does not exist. Since you are posting from Russia what do you want to achieve? Satisfy the «Экспертами» (see this post)? Every time I was in Moscow we had endless & fruitless debates about it… All relevant documents (2008 GL, 2013 “Red Book”, 2015 EEU GL) are more or less translations of the EMA’s GL (the EEU GL spiced with some parts of the WHO’s GL). What does the EMA’s GL say about groups (or more important sites)? Nothing! Only: The precise model to be used for the analysis should be prespecified in the protocol. The statistical analysis should take into account sources of variation that can be reasonably assumed to have an effect on the response variable. Is it reasonable to assume such an effect if a study was performed in multiple groups due to logistic reasons (e.g., limited capacity of the clinical site)? I don’t think so. Hence, in the EU generally data are simply pooled and the common model (without a group term) is used.Different sites are much more problematic. I recently saw a multisite study where the sites clearly showed different results (averages differed tenfold). It was a cancer drug and some sites were pretty small. If (if!) all sites would have balanced sequences it would have been still no problem but this was not the case. Actually there was a highly significant (p <0.001) sitebytreatment interaction. If one would naïvely pool the sites the treatment effect would be biased. OK, back to the EEU GL (the last paragraph of section 94): Если предполагается проведение исследования в нескольких группах из логистических соображений, об этом необходимо явно указать в протоколе исследования; при этом, если в отчете отсутствуют результаты статистического анализа, учитывающие многогрупповой характер исследования, необходимо представить научное обоснование отсутствия таких результатов. My interpretion:
#2 can be nasty! Start with “Model 1” (fixed effects* in Phoenixnotation): Group + Sequence + Sequence(Group) + Period(Group) + Treatment +
Group + Sequence + Sequence(Group) + Period(Group) + Treatment + Yes, you will see an awful lot of “Not estimables”.
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 