Elena777 ☆ Belarus, 20190909 19:34 Posting: # 20564 Views: 645 

Dear all, I would be pleased to get your opinion on the following. We are planning to conduct several BE studies with adaptive design using the drugs with uncertain intraCV. We have decided to use method C described by Potvin and included the description of the model C in the protocols (the same as in the corresponding scheme presented in Potvin's article). But it seems it's not enough.
Post number 20,000. [Helmut] 
ElMaestro ★★★ Belgium?, 20190909 21:39 @ Elena777 Posting: # 20565 Views: 619 

Hello Elena777, » 1. Should we include the information that evaluation after stage 1 completion should be performed assuming GMR=0.95? I would do so. » 2. Should we describe the maximum number of subjects who can be included in whole or in stage 2? I would only put a cap on it if you can refer to simulations having done exactly so (having done so in exactly your way of capping). » 3. Any other information that should be clearly stated in order to be accurate and to satisfy regulatory authorities? Exact decision tree, and exact values for alphas, desired power level, and power being calculated using GMR=0.95. » 4. What if BE criteria are met after stage 1, but estimated power is too low (e.g. 30%)? "Too low"? It is not a crime to be lucky. I don't see any issue. Regulators are generally not afraid of low power after results become available. This forum is a paradise for grumpy old men being adverse to posthoc power. I used be to a reasonably happy, cheerful bloke, but then I got a profile here and quickly I went very sour if not outright angry. I blend in nicely, I think?!? — I could be wrong, but... Best regards, ElMaestro 
Helmut ★★★ Vienna, Austria, 20190909 23:27 @ ElMaestro Posting: # 20567 Views: 609 

Hi ElMaestro, » » 1. Should we include the information that evaluation after stage 1 completion should be performed assuming GMR=0.95? » » I would do so. So would I. » » 2. Should we describe the maximum number of subjects who can be included in whole or in stage 2? » » I would only put a cap on it if you can refer to simulations having done exactly so (having done so in exactly your way of capping). From a regulatory perspective this is not necessary. Any futility rule (like max. n_{2}) decreases the chance to show BE if compared to a published method without one. Hence, if the type I error was controlled in a method without a futility rule, the TIE will always be lower with a futility rule. However, if a futility rule is too strict, you may shoot yourself in the foot since power might be compromised. To check that, sim’s are a good idea indeed. » » 3. Any other information that should be clearly stated in order to be accurate and to satisfy regulatory authorities? » » Exact decision tree, and exact values for alphas, desired power level, and power being calculated using GMR=0.95. Yep. » » 4. What if BE criteria are met after stage 1, but estimated power is too low (e.g. 30%)? » » It is not a crime to be lucky. Absolutely. As one of the grumpy old men: Forget power, doesn’t matter. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Ohlbe ★★★ France, 20190910 10:27 @ ElMaestro Posting: # 20570 Views: 570 

Dear ElMaestro, [off topic] » This forum is a paradise for grumpy old men being adverse to posthoc power. Hey, I'm not old ! [/off topic] — Regards Ohlbe 
Helmut ★★★ Vienna, Austria, 20190909 23:17 @ Elena777 Posting: # 20566 Views: 609 

Hi Elena, » We are planning to conduct several BE studies with adaptive design using the drugs with uncertain intraCV. We have decided to use method C described by Potvin … Whether Potvin’s Method C will be accepted depends on the jurisdiction you are bound to.^{a,b} More information please.
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Elena777 ☆ Belarus, 20190911 20:24 @ Helmut Posting: # 20587 Views: 496 

Dear Helmut and ElMaestro, Thank you very much for responding me shortly. We are planning to submit applications in Belarus (the country I`m from) and in Russia. Hope method C will be OK for them. Another 2 questions:

Helmut ★★★ Vienna, Austria, 20190912 01:31 @ Elena777 Posting: # 20589 Views: 488 

Hi Elena, » We are planning to submit applications in Belarus (the country I`m from) and in Russia. Hope method C will be OK for them. THX for the information. Sections 97/98 of the EEU regulations are a 1:1 translation of the corresponding section about TSDs in the EMA’s BEGL. It is difficult to predict how regulators of the EEU interpret their own guideline. Maybe other members from Belarus (4) and Russia (21) can share their experiences. My ranking (not based on scientific value but on likelihood of acceptance) in the following. To explore the empiric type I error (TIE) I recommend functions of the Rpackage Power2Stage with 1 mio simulations at theta0=1.25 . When I give locations of the maximum TIE it is based on a much narrower grid than in the publications (n_{1} 12…72, step size 2 and CV 10…80%, step size 2%).
» Another 2 questions: » 1. Is it a good idea to add the statement that, in case of conducting stage 2, data from both stages will be pooled by default, without evaluation of differences between stages? If you performed the second stage, it’s mandatory to pool the data. None of the methods contains any kind of test between stages. Furthermore, a formulationbystage interaction term in the model is considered nonsense in the EMA’s Q&A. » 2. Is there any established minimum for a number of subjects that should be included in stage 2 (e.g. at least 2, or at least 1 for each sequence (TR/RT))? Nothing in the guidelines, but mentioned in the EMA’s Q&A document. However, that’s superfluous. If you perform a sample size estimation, in all software the minimum stage 2 sample size will be 2 anyhow (if odd, rounded up to the next even to obtain balanced sequences). In the functions of Power2Stage you can used the argument min.n2=2 and will never see any difference.Only if you are a nerd, read the next paragraph. The conventional sample size estimation does not take the stageterm in the final analysis into account. If you prefer to use braces with suspenders, use the function In #5 the minimum is 4 because you perform a separate ANOVA in the second stage. One word of caution: If you have a nasty drug (dropouts due to AEs) take care that you don’t end up with <3 subjects – otherwise the ANOVA would not be possible. In designing a study I recommend to call the functions with the arguments theta0 and CV , which are your best guesses. Don’t confuse that with the argument GMR , which is fixed in most methods. Then you get an impression what might happen (chance to show BE in the first stage, probability to proceed to stage 2, average & range of total sample sizes…). n_{1} which is ~80% of a fixed sample design is a good compromise between chances to show BE in the first stage whilst keeping overall power. Example of finding a suitable futility rule of the total sample size:CV < 0.25 A futility rule of 48 looks good. Let’s explore the details: Power2Stage::power.tsd(CV=CV, n1=n1, Nmax=48) Not that bad. However, futility rules can be counterproductive because you have to come up with a “best guess” CV – which is actually against the “sprit” of TSDs. Homework: Power2Stage::power.tsd(CV=0.30, n1=24, Nmax=48) As ElMaestro wrote above you have to perform own simulations if you are outside the published methods (GMR, target power, n_{1}/CVgrid, futility rules). My basic algorithm is outlined by Molins et al.^{ 7} A final reminder: In the sample size estimation use the fixed GMR (not the observed one), unless the method allows that.
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Astea ★ Russia, 20190914 14:56 @ Helmut Posting: # 20597 Views: 302 

Dear all! Elena777, I guess I misunderstood something. Did you mean aposteriory power or interim power? If interim is 30% go to the next step by the decision tree. » It is difficult to predict how regulators of the EEU interpret their own guideline So true! "Words are chameleons, which reflect the color of their environment" (Learned Hand) Helmut, what was the final conclusions on the post? Are there any suggestions on how to deal with two metrics in adaptive trials? For example: a). Let us consider Type II design: first step  estimated power is less than target (80%) for C_{max} and more than target for AUC, besides 90%CI for AUC is OK. 1). We calculate 100(12α_{adj}) CI for C_{max}, should we also do it for AUC? It can fail. 2). Suppose further we go to the 2^{nd} stage. Should we use data from the 2^{nd} stage to estimate CI for AUC the second time? If yes, it possibly can fail, if not  how to explain the fact that we do not use the data? Another example: b). First step  estimated power is less than target (80%) for both metrics and adjusted level CI is outside the range. Should we use the largest observed CV to calculate the total sample size? Would the study be overpowered for the second PK metric? Would it affect the TIE? To conclude: what is the best strategy to follow in this situation in order to avoid inflation of the TIE and the loss of power? (Some mad idea: is it possible to make some hybrid monster to combine both C_{max} and AUC in the same test for adaptive designs? Something like C_{max}/AUC but with more powerful reflection of the situations (I dealt with a plenty of studies (BE and not proven BE) with C_{max}/AUC as an additional metric, only once it was outside the range) » Furthermore, a formulationbystage interaction term in the model is considered nonsense in the EMA’s Q&A. What ANOVA model should be used for the second stage? By the way, what about the code on R for the full decision tree? — "We are such stuff as dreams are made on, and our little life, is rounded with a sleep" 
Helmut ★★★ Vienna, Austria, 20190916 11:50 @ Astea Posting: # 20598 Views: 217 

Hi Nastia, » Helmut, what was the final conclusions on the post? I was wrong and we shouldn’t worry. See Detlew’s simulations. » Are there any suggestions on how to deal with two metrics in adaptive trials? » For example: a). Let us consider Type II design: first step  estimated power is less than target (80%) for C_{max} and more than target for AUC, besides 90%CI for AUC is OK. » 1). We calculate 100(12α_{adj}) CI for C_{max}, should we also do it for AUC? It can fail. » 2). Suppose further we go to the 2^{nd} stage. Should we use data from the 2^{nd} stage to estimate CI for AUC the second time? If yes, it possibly can fail, if not  how to explain the fact that we do not use the data? Think about how we design a fixed sample design. Always based on the metric with the higher CV. I would go with your 2). How likely is it that AUC (which passed already in the first stage) will fail in the second? Let’s consider the example of the other post. I assumed the best, i.e., all studies in the ‘type II’ design passed with α 0.05. Now:
» Another example: b). First step  estimated power is less than target (80%) for both metrics and adjusted level CI is outside the range. Should we use the largest observed CV to calculate the total sample size? Yes. » Would the study be overpowered for the second PK metric? Would it affect the TIE? According to Detlew’s simulations, no. Given, only ‘type I’ implemented which is more conservative anyhow.
» To conclude: what is the best strategy to follow in this situation in order to avoid inflation of the TIE and the loss of power? Estimate the sample size based on the metric with the higher CV. No inflation of the TIE and a gain in power for the other metric. » (Some mad idea: is it possible to make some hybrid monster to combine both C_{max} and AUC in the same test for adaptive designs? Take some Schützomycin? » Something like C_{max}/AUC but with more powerful reflection of the situations (I dealt with a plenty of studies (BE and not proven BE) with C_{max}/AUC as an additional metric, only once it was outside the range) As expected. C_{max}/AUC is generally less variable than C_{max}. » » Furthermore, a formulationbystage interaction term in the model is considered nonsense in the EMA’s Q&A. » » What ANOVA model should be used for the second stage? According to the Q&A: stage, sequence, sequence × stage, subject(sequence × stage), period(stage), treatment. As usual for the EMA, all effects fixed and the nested term subject(sequence × stage) superfluous.The simple model stage, sequence, sequence × stage, subject, period(stage), treatment. gives exactly the same result.I once received a deficiency letter for a ‘type 2’ study passing in the first stage (α 0.05!) where I dared to model subjects as a random effect… Interesting that there were no questions to use an adjusted α (would have passed as well but I followed my SAP which was approved by the BfArM and for “educational reasons” I didn’t show the adjusted CI). » By the way, what about the code on R for the full decision tree? Ask Detlew or inspect the sources of power.tsd() and power.tsd.2m() . — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Astea ★ Russia, 20190916 18:28 @ Helmut Posting: # 20599 Views: 162 

Dear Helmut! » I was wrong and we shouldn’t worry. See Detlew’s simulations. That's good. Sorry, I didn't realized it at first. » How likely is it that AUC (which passed already in the first stage) will fail in the second? Thank you for the example! I've puzzled whether it will be reproduced for other cases. Let us consider the situation when CV of C_{max} and AUC are very close to each other, like 21% and 20%, and for the first stage the number of subjects (n_{1}=20) was sufficient for AUC, but not for C_{max}. Calculation shows that even then the power for AUC for the second stage would be always enough.
» Take some Schützomycin? Did you patent that? I gonna make a generic » According to the Q&A: stage, sequence, sequence × stage, subject(sequence × stage), period(stage), treatment. Are there any documents to refer which mention this model (excepting the answer on the EMA's web page?) » Ask Detlew or inspect the sources of power.tsd() and power.tsd.2m() . Ok, need more tea to dive to the source... — "We are such stuff as dreams are made on, and our little life, is rounded with a sleep" 
Helmut ★★★ Vienna, Austria, 20190917 12:27 @ Astea Posting: # 20606 Views: 41 

Hi Nastia, » Let us consider the situation when CV of C_{max} and AUC are very close to each other, like 21% and 20%, and for the first stage the number of subjects (n_{1}=20) was sufficient for AUC, but not for C_{max}. » […] even then the power for AUC for the second stage would be always enough. […] Wow, you are a master of condensed Rcode! Here my version:
» » Take some Schützomycin? » » Did you patent that? I gonna make a generic Not mine. It was mentioned for the first time by ElMaestro back in 2010: » » Let's say we want to develop a generic of Schützomycin. The product is available in one strength, posology is 1 tablet daily. […] Schützomycin is a nice drug with little safety concern. My claim seems to be unfounded. » » According to the Q&A: stage, sequence, sequence × stage, subject(sequence × stage), period(stage), treatment. » Are there any documents to refer which mention this model (excepting the answer on the EMA's web page?) Made up out of thin air by the EMA. To quote myself^{1} In none of the published procedures, a test for poolability was part of the simulations. Although statistical tests could be constructed comparing variances of stages, their precision is poor in such designs and should be applied with caution. Nonetheless, in 2013, the European Medicines Agency introduced an additional term sequence×stage to the statistical model. Since both sequence and stage are betweensubject effects, the residual error (hence, the CI) should not be affected – which was recently demonstrated.^{2} Excerpt^{2} Special emphasis was also given to the significance (P value) of the additional term ‘sequence × stage’ used in the ANOVA model proposed by EMA. […]
You don’t have to be a rocket scientist to understand that. Why the EMA introduced it, remains a mystery. Maybe influenced by GarcíaArieta and Gordon?^{3} A term for the stage should be included in the ANOVA model. However, the guideline does not clarify what the consequence should be if it is statistically significant. In principle, the data sets of both stages could not be combined. Well, stage is already a factor in all published methods (didn’t they read them?). Concerning “poolability” see above. Reminds me on Grizzle’s nonsense for crossovers “if the sequenceeffect is significant, analyze data of the first period as a parallel design”.
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Elena777 ☆ Belarus, 20190916 19:48 @ Helmut Posting: # 20601 Views: 149 

Dear Helmut, Still need some clarifications on the question from Astea: » a) first step  estimated power is less than target (80%) for C_{max} and more than target for AUC, besides 90%CI for AUC is OK. To be in compliance with method C of Potvin, we recalculate CI for Cmax with α=0,0294. Let`s assume that BE criterion for Cmax is met after this step. Should we also recalculate CI for AUC with α=0,0294? Otherwise, finally we will have the following in a CSR: BE criterion was met for AUC using α=0,05 (90%CI) and BE criterion was met for Cmax using α=0,0294 (94,12% CI). Does it look OK? 
Helmut ★★★ Vienna, Austria, 20190916 23:30 @ Elena777 Posting: # 20603 Views: 122 

Hi Elena, » To be in compliance with method C of Potvin, we recalculate CI for Cmax with α=0,0294. Let`s assume that BE criterion for Cmax is met after this step. […] Otherwise, finally we will have the following in a CSR: BE criterion was met for AUC using α=0,05 (90%CI) and BE criterion was met for Cmax using α=0,0294 (94,12% CI). Does it look OK? Absolutely. The ideas behind the different alphas in Potvin C are:
» Should we also recalculate CI for AUC with α=0,0294? You could but please only “at home”. That’s against the method and what you should have laid down in the protocol. See the end of this post. My study would have passed with the adjusted α as well. But what if not? See this bizarre case study. If the sponsor would have known before that the agency will not accept Method C, they would have planned for Method B, initiated a second stage with 6 (six!) subjects and happily walked away. Stupid. What will you do with your ”homework”?
I recommended it for years. Well, no more. The small gain in power claimed by the authors is not worth the troubles:
— Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Elena777 ☆ Belarus, 20190916 19:35 @ Astea Posting: # 20600 Views: 152 

Dear Astea, » Elena777, I guess I misunderstood something. Did you mean aposteriory power or interim power? If interim is 30% go to the next step by the decision tree. tree? I meant interim power (power that we calculate after stage 1 completion). 
Astea ★ Russia, 20190916 20:39 @ Elena777 Posting: # 20602 Views: 141 

Dear Elena777! » tree? I meant decision scheme (in graph theory mathematicians call "trees" undirected graphs). See also this message. » To be in compliance with method C of Potvin, we recalculate CI for Cmax with α=0,0294. Let`s assume that BE criterion for Cmax is met after this step. Should we also recalculate CI for AUC with α=0,0294? Otherwise, finally we will have the following in a CSR: BE criterion was met for AUC using α=0,05 (90%CI) and BE criterion was met for Cmax using α=0,0294 (94,12% CI). Does it look OK? In this situation we still have to go to the next stage. As it was shown before the fail for AUC (if it has less variability than C_{max}) in the second stage is very unlikely. So it just doesn't matter what was CI for AUC after the first stage, the second stage for C_{max} should be crucial. — "We are such stuff as dreams are made on, and our little life, is rounded with a sleep" 
Helmut ★★★ Vienna, Austria, 20190916 23:37 @ Astea Posting: # 20604 Views: 117 

Hi Nastia, » » […] BE criterion was met for AUC using α=0,05 (90%CI) and BE criterion was met for Cmax using α=0,0294 (94,12% CI). Does it look OK? » » In this situation we still have to go to the next stage. Why (see above)? Both AUC and C_{max} passed already in the first stage. — Cheers, Helmut Schütz The quality of responses received is directly proportional to the quality of the question asked. ☼ Science Quotes 
Astea ★ Russia, 20190917 06:14 @ Helmut Posting: # 20605 Views: 96 

Dear Helmut and Elena777! » Why (see above)? Both AUC and C_{max} passed already in the first stage. Oops, sorry . I was wrong (thought about the case of fail C_{max}). Thank you! 