BE-proff
★★  

Russia,
2016-06-02 14:35

Posting: # 16382
Views: 7,519
 

 Regulatory acceptance of PowerTOST [Software]

Dear All,

Do you make references to calculations done by PowerTOST in documents submitted to regulatory authorities (i.e. study protocols, clarifications, etc)?
I mean phrases like "sample size was calculated with PowerTOST bla-bla".:-)

AFAIK, authorities want to see validated software while R-based scripts seem not to be validated...

How to prove officials that PowerTOST is reliable soft?:confused:

Thx.


Edit: Category changed; see also this post #1. [Helmut]
d_labes
★★★

Berlin, Germany,
2016-06-02 15:05

@ BE-proff
Posting: # 16384
Views: 6,786
 

 Regulatory acceptance of Sample size software

Dear BE-proff,

» AFAIK, authorities want to see validated software while R-based scripts seem not to be validated...
» How to prove officials that PowerTOST is reliable soft?:confused:

I'm not a friend of software validation, whatever that is.

Ask the same question to statements like:
"I have calculated the sample size with SAS, PASS, nQuery, Brain-Pencil-Paper ... (or whatever soft-/wetware you prefer ;-)). Is that SAS, PASS, nQuery, Brain-Pencil-Paper ... validated and reliable?"

See also this thread.

Then act. Others, more trained in this ugly field of software validation may give you more hints what to do in detail. In short:
  • Write a validation plan
  • Do what you have written in the validation plan
  • Write a validation report

BTW: PowerTOST is not an R-based script. It is a package providing you the possibility (via functions) to estimate power / sample size with your R-based script.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-06-02 15:39

@ BE-proff
Posting: # 16385
Views: 6,741
 

 PowerTOST/Power2Stage: no issues so far

Hi BE-proff,

» Do you make references to calculations done by PowerTOST in documents submitted to regulatory authorities (i.e. study protocols, clarifications, etc)?

Yes, I do.
In tricky cases (e.g., justification of an adjusted α in two-stage design) I give my script in an appendix to the protocol.

» How to prove officials that PowerTOST is reliable soft?:confused:

Hart is’ hart und weich is’ weich.
Aber allweil weich is’ auch hart.
    Austrian proverb


My translation:

Hard is hard and soft is soft.
But always soft is hard as well.


Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
DavidManteigas
★    

Portugal,
2016-06-03 11:05
(edited by Ohlbe on 2016-06-03 13:54)

@ BE-proff
Posting: # 16389
Views: 6,622
 

 Regulatory acceptance of PowerTOST

Hi,

As far as I understand, regulatory authorities in Europe are not that troublesome with software as FDA. They don't even ask the sponsor to send the datasets of the study or the Logs of the statistical software execution. I believe that in some countries, you could state in the Protocol and in the SAP that you will used Winonlin and SAS in your analysis and use R in all of them and never have a problem in submission's nor in inspections. For instance, regulatory inspections in Portugal care only about GCP compliance in the clinical site. They don't even check in detail eDC software or data management & analysis process. Lack of qualified resources I think.
Nevertheless, my personal opinion is that you should always "validate" yourself the results of the software. For instance, when I use a new R package to implement a new statistical method I always validate the results with SAS, being the job for regulatory purposes or not. I also do that when I'm doing an analysis I'm not very comfortable with and I store both outputs (R and SAS) for my own protection in case of questions (the main trouble with R arises from the clients, much of them used to SAS and don't want to take the risk of using R just by "superstition").

Regards,
David


Edit: Full quote removed. Please delete everything from the text of the original poster which is not necessary in understanding your answer; see also this post! [Ohlbe]
BE-proff
★★  

Russia,
2016-06-03 22:30

@ DavidManteigas
Posting: # 16392
Views: 6,614
 

 Regulatory acceptance of PowerTOST

Hi again,

I'll make some clarifications.

In Russia we have two options to prove sample size: either demonstrate calculations with formulas or provide literature reference.

The 1st option is not for me because I don't have relevant background so I'd prefer the 2nd way.

So, if somebody could share any literature reference in a journal (preferrable) which mentions PowerTOST I'd really appreciate it :-)

Thx
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2016-06-04 15:51

@ BE-proff
Posting: # 16399
Views: 6,722
 

 Regulatory acceptance of PowerTOST

Hi BE-proff,

» In Russia we have two options to prove sample size: either demonstrate calculations with formulas …

See this presentation (slide 8). The procedure is always iterative (since the t-value in the formula to calculate power depends on the degrees of freedom and hence, the sample size itself). The approximation1 by the shifted central t-distribution can be done with a pocket calculator (alternatively: paper-pencil-brain) and a t-table. The better approximation (noncentral t) already needs software, since no tables exist for a range of the noncentrality parameter. The exact method needs numeric integration of two integrals → software.

» … or provide literature reference.

What do you mean by that?

» The 1st option is not for me because I don't have relevant background so I'd prefer the 2nd way.
»
» So, if somebody could share any literature reference in a journal (preferrable) which mentions PowerTOST I'd really appreciate it :-)

Jones & Kenward2

The power and sample size for a bioequivalence trial can conveniently be calculated using the R package PowerTOST (Labes (2013)).a
The power is calculated using the R package PowerTOST (Labes (2013)) with the “exact” option.b
Using the PowerTOST (Labes (2013)) R library function, the power, for a true ratio of 1, for this estimated CV and a total sample size of 20 is just over 0.04.c


PowerTOST is mentioned in Nadarajah et al.3

In the folder \library\PowerTOST\tests you find R-scripts to validate the various sample size functions against published data. See the man-pages of ct5.1, ct5.2, ct5.3, ct5.4.1, ct9.6.2, ct9.6.4, ct9.6.6, ct9.6.8, ctCW.III, ctSJ.VIII.10, and ctSJ.VIII.20 for the references.


  1. Hauschke, D, Steinijans, VW, Diletti E, Burke M. Sample Size Determination for Bioequivalence Assessment Using a Multiplicative Model.
    J Pharmacokin Biopharm 1992;20(5);557–61. doi:10.1007/BF01061471.
  2. Jones B, Kenward MG. Design and Analysis of Cross-Over Trials.
    Boca Raton: CRC Press / Chapman & Hall; 3rd ed. 2015.
    1. p.329
    2. p.373
    3. p.379
  3. Nadarajah S, Sergey Bityukov S, Krasnikov N. Confidence distributions: A review.
    Stat Meth. 2015;22:23–46. doi:10.1016/j.stamet.2014.07.002. [image] open source.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
BE-proff
★★  

Russia,
2016-06-07 15:15

@ Helmut
Posting: # 16405
Views: 6,463
 

 Regulatory acceptance of PowerTOST

Hi Helmut,

Thank you very much for your help! :ok:
ElMaestro
★★★

Belgium?,
2016-06-04 16:12

@ BE-proff
Posting: # 16400
Views: 6,620
 

 Regulatory acceptance of PowerTOST

Hi BE-proff,

as far as I know PowerTOST and other packages will generally be accepted.
Power is usually the sponsor's problem: It is the chance of showing BE when a set of assumptions are met. It usually does not affect patient's risk and so isn't where you get the most deficiency letters. At the end of the day, your assumptions are guess and guesses tend to have a lot of uncertainty, so power is in that perspective not a question of having a boatload of decimals right. Besides there is no regulatory power minimum or maximum though 80% or thereabouts is quite common.
Regulatory questions -or inspection triggers- may arise from bad guesses; for example it may be somewhat unethical to assume a CV of 8% and a GMR of 100.00% for most drugs (you'd be including too few patients for a realistic chance of showing BE). Thus, what the industry generally needs is a healthy dose of common sense rather than validation of power-calculating software to the 17th decimal. It is difficult to validate someone's ability to guess well anyhow.

I could be wrong, but...
Best regards,
ElMaestro
BE-proff
★★  

Russia,
2016-06-07 15:17

@ ElMaestro
Posting: # 16406
Views: 6,401
 

 Regulatory acceptance of PowerTOST

Hi ElMaestro,

We will try to use PowerTOST with literature references in protocol.
Time will show if such option suits our regulators...
mittyri
★★  

Russia,
2016-06-08 07:51

@ BE-proff
Posting: # 16414
Views: 6,425
 

 Regulatory acceptance of PowerTOST

Hi BE-proff,

» Time will show if such option suits our regulators...

We used it without any questions from experts. Good luck!

Kind regards,
Mittyri
Astea
★    

Russia,
2016-06-14 17:41

@ mittyri
Posting: # 16429
Views: 6,267
 

 Regulatory acceptance of PowerTOST

I join the opinion of mittyri. Until now, there were no pretensions in this regard.

"We are such stuff as dreams are made on, and our little life, is rounded with a sleep"
Activity
 Thread view
Bioequivalence and Bioavailability Forum |  Admin contact
19,871 posts in 4,212 threads, 1,364 registered users;
online 19 (0 registered, 19 guests [including 6 identified bots]).
Forum time (Europe/Vienna): 13:41 CEST

Multivariate analysis: A means of finding the answer
when you don’t know the question.    Stephen Senn

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
HTML5