Vienna, Austria,
2019-08-28 12:58

Posting: # 20527
Views: 1,040

 R in regulated environments [R for BE/BA]

Dear all,

[image]since occasionally the question is asked whether R is accepted in regulatory environments, here the answers:
  1. Yes, if the installation and code is validated.
  2. If you are an expert and understand what #1 means with all its pitfalls,
    • stop reading the post here;
    • otherwise, get a cup of coffee and continue.

Any (!) system has to pass three levels (definitions given in ISO 9000 and by the FDA in 1999 and 2002):
  1. Installation Qualification (IQ)
       The system is compliant with appropriate codes and approved design intentions, and that vendor’s recommendations are suitably considered.
  2. Operational Qualification (OQ)
       The system is capable of consistently operating within stated limits and tolerances.
  3. Performance Qualification (PQ)
       The system is meeting all release requirements for functionality and safety and that procedures are effective and reproducible.
Whereas #1 generally is the job of the vendor, #2 can be shared between the vendor and user, the responsibility for #3 lies entirely in the hands of the user.
   Since the source code of commercial software is not accessible to the user, only a black box validation can be performed (i.e., compare results of reference data sets with published ones). Open source software (e.g., R, GNU Octave, …) allows – by definition – a white box validation but this can be tough (requires an experienced coder). Hence, in practice most users opt for a black box validation as well. See also reference data sets for various designs in BE.1,2,3

Quotes from relevant documents:
  • ICH E94
       The credibility of the numerical results of the analysis depends on the quality and validity of the methods and software (both internally and externally written) used both for data management (data entry, storage, verification, correction and retrieval) and also for processing the data statistically. Data management activities should therefore be based on thorough and effective standard operating procedures. The computer software used for data management and statistical analysis should be reliable, and documentation of appropriate software testing procedures should be available.
  • EMA, Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials5
       Records of system validation including requirements, design, installation, access and security, testing (e.g. user acceptance testing, installation, operational and performance testing), training and controlled release for use should be maintained.
  • FDA, Statistical Software Clarifying Statement6
       FDA does not require use of any specific software for statistical analyses, and statistical software is not explicitly discussed in Title 21 of the Code of Federal Regulations [e.g., in 21CFR part 11]. However, the software package(s) used for statistical analyses should be fully documented in the submission, including version and build identification.
  • WHO, TRS 996, Annex 97
       Computer systems should be qualified and validated (hardware, software, networks, data storage systems and interfaces. Qualification is the planning, carrying out and recording of tests on equipment and systems which form part of the validated process, to demonstrate that the equipment or system will perform as intended.
  • Addendum to ICH E6(R1)8
       Ensure the integrity of the data including any data that describe the context, content, and structure. This is particularly important when making changes to the computerized systems, such as software upgrades or migration of data.

When it comes to R, a lot is going on – especially with support of regulators, the academia, and innovators.9,10,11,12 Sebastian Wolf of Roche Diagnostics presented11 a 500,000+ lines Shiny application.14 ;-)
   One reason for the increasing popularity of R in the industry is that the statistical curriculum gradually shifted from SAS to R and nowadays graduates are at least “bilingual” (nerds are even proficient in FORTRAN and/or C). Young statisticians are no more willing to accept a “SAS only” working environment.

  1. Schütz H, Labes D, Fuglsang A. Reference Datasets for 2-Treatment, 2-Sequence, 2-Period Bioequivalence Studies. AAPS J. 2014; 16(6): 1292–7. doi:10.1208/s12248-014-9661-0. [image] free resource.
  2. Fuglsang A, Schütz H, Labes D. Reference Datasets for Bioequivalence Trials in a Two-Group Parallel Design. AAPS J. 2015; 17(2): 400–4. doi:10.1208/s12248-014-9704-6.[image] free resource.
  3. Schütz H, Tomashevskiy M, Labes D, Shitova A, González-de la Parra M, Fuglsang A. Reference Data­sets for Studies in a Replicate Design intended for Average Bioequivalence with Expanding Limits. Manu­script in pre­paration 2019.
  4. International Council for Harmonisation. Statistical Principles for Clinical Trials E9. 5 February 1998.
  5. European Medicines Agency, GCP Inspectors Working Group. Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. London, 9 June 2010.
  6. US FDA. Statistical Software Clarifying Statement. May 6, 2015. Study Data Standards.
  7. World Health Organization. Technical Report Series No. 996, Annex 9. Guidance for organizations performing in vivo bioequivalence studies. Geneva, May 2016.
  8. International Council for Harmonisation. Integrated Addendum to ICH E6(R1): Guideline For Good Clinical Practice E6(R2). 9 November 2016.
  9. The R Foundation for Statistical Computing. R: Regulatory Compliance and Validation Issues. A Guidance Document for the Use of R in Regulated Clinical Trial Environments. Vienna, March 25, 2018. [image] free resource.
  10. Smith D. How R is used by the FDA for regulatory compliance. June 29, 2017.
  11. R/Pharma 2018. Harvard University, 15/16th August, 2018. Program.
  12. Rickert J. Conference Report: R / Pharma 2018. R J. 2018;10(2):579–80.
  13. pharmaR. Validation Overview.
  14. Wolf S. How to Build A Shiny “Truck”! 2018-08-14.

Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Vienna, Austria,
2020-01-09 15:04

@ Helmut
Posting: # 21070
Views: 293

 FDA, again

[image] Dear all,

we know that the FDA uses R internally, e.g., in all simulations which lead to its reference-scaling method for NTIDs.

THX to ElMaestro discovering another example. Though I attended the 4th GBHI workshop last month in Bethesda, it is hidden in the backup slides.* Quote:

Q1. What is a sample size to achieve 80% power to pass bioequivalence, assuming CV as 100%, T/R true ratio as 5%?
Q2. With a fixed sample size (n=1,000), what are maximum differences in FEV1 metrics which pass BE with 80% power?

To address questions 1–2;

  • package ‘PowerTOST’ was used to estimate sample size using observed CVs (R ver 3.6.0)
  • Sample size was calculated for reference and treatment group

Of note, Robert Lionberger (Director Office of Research and Standards, Office of Generic Drugs) told me that he visits the forum regularly. :thumb up:

  • Liang Zhao (Division of Quantitative Methods and Modeling, ORS, OGD, CDER/FDA). FEV1 Based Bioequivalence Study for Inhaled Corticosteroids.

Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
 Admin contact
20,246 posts in 4,261 threads, 1,397 registered users;
online 12 (0 registered, 12 guests [including 5 identified bots]).
Forum time (Europe/Vienna): 02:59 UTC

To call the statistician after the experiment is done
may be no more than asking him to perform a postmortem examination:
he may be able to say what the experiment died of.    R.A. Fisher

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz