Helmut
★★★
avatar
Homepage
Vienna, Austria,
2019-08-28 14:58

Posting: # 20527
Views: 3,288
 

 R in regulated environments [R for BE/BA]

Dear all,

[image]since occasionally the question is asked whether R is accepted in regulatory environments, here the answers:
  1. Yes, if the installation and code is validated.
  2. If you are an expert and understand what #1 means with all its pitfalls,
    • stop reading the post here;
    • otherwise, get a cup of coffee and continue.

Any (!) system has to pass three levels (definitions given in ISO 9000 and by the FDA in 1999 and 2002):
  1. Installation Qualification (IQ)
       The system is compliant with appropriate codes and approved design intentions, and that vendor’s recommendations are suitably considered.
  2. Operational Qualification (OQ)
       The system is capable of consistently operating within stated limits and tolerances.
  3. Performance Qualification (PQ)
       The system is meeting all release requirements for functionality and safety and that procedures are effective and reproducible.
Whereas #1 generally is the job of the vendor, #2 can be shared between the vendor and user, the responsibility for #3 lies entirely in the hands of the user.
   Since the source code of commercial software is not accessible to the user, only a black box validation can be performed (i.e., compare results of reference data sets with published ones). Open source software (e.g., R, GNU Octave, …) allows – by definition – a white box validation but this can be tough (requires an experienced coder). Hence, in practice most users opt for a black box validation as well. See also reference data sets for various designs in BE.1,2,3

Quotes from relevant documents:
  • ICH E94
       The credibility of the numerical results of the analysis depends on the quality and validity of the methods and software (both internally and externally written) used both for data management (data entry, storage, verification, correction and retrieval) and also for processing the data statistically. Data management activities should therefore be based on thorough and effective standard operating procedures. The computer software used for data management and statistical analysis should be reliable, and documentation of appropriate software testing procedures should be available.
  • EMA, Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials5
       Records of system validation including requirements, design, installation, access and security, testing (e.g. user acceptance testing, installation, operational and performance testing), training and controlled release for use should be maintained.
  • FDA, Statistical Software Clarifying Statement6
       FDA does not require use of any specific software for statistical analyses, and statistical software is not explicitly discussed in Title 21 of the Code of Federal Regulations [e.g., in 21CFR part 11]. However, the software package(s) used for statistical analyses should be fully documented in the submission, including version and build identification.
  • WHO, TRS 996, Annex 97
       Computer systems should be qualified and validated (hardware, software, networks, data storage systems and interfaces. Qualification is the planning, carrying out and recording of tests on equipment and systems which form part of the validated process, to demonstrate that the equipment or system will perform as intended.
  • Addendum to ICH E6(R1)8
       Ensure the integrity of the data including any data that describe the context, content, and structure. This is particularly important when making changes to the computerized systems, such as software upgrades or migration of data.

When it comes to R, a lot is going on – especially with support of regulators, the academia, and innovators.9,10,11,12 Sebastian Wolf of Roche Diagnostics presented11 a 500,000+ lines Shiny application.14 ;-)
   One reason for the increasing popularity of R in the industry is that the statistical curriculum gradually shifted from SAS to R and nowadays graduates are at least “bilingual” (nerds are even proficient in FORTRAN and/or C). Young statisticians are no more willing to accept a “SAS only” working environment.


  1. Schütz H, Labes D, Fuglsang A. Reference Datasets for 2-Treatment, 2-Sequence, 2-Period Bioequivalence Studies. AAPS J. 2014; 16(6): 1292–7. doi:10.1208/s12248-014-9661-0. [image] free resource.
  2. Fuglsang A, Schütz H, Labes D. Reference Datasets for Bioequivalence Trials in a Two-Group Parallel Design. AAPS J. 2015; 17(2): 400–4. doi:10.1208/s12248-014-9704-6. [image] free resource.
  3. Schütz H, Tomashevskiy M, Labes D, Shitova A, González-de la Parra M, Fuglsang A.Reference Data­sets for Studies in a Replicate Design intended for Average Bioequivalence with Expanding Limits. AAPS J. 2020; 22(2): Online First 7 February 2020. doi:10.1208/s12248-020-0427-6.
  4. International Council for Harmonisation. Statistical Principles for Clinical Trials E9. 5 February 1998.
  5. European Medicines Agency, GCP Inspectors Working Group. Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. London, 9 June 2010.
  6. US FDA. Statistical Software Clarifying Statement. May 6, 2015. Study Data Standards.
  7. World Health Organization. Technical Report Series No. 996, Annex 9. Guidance for organizations performing in vivo bioequivalence studies. Geneva, May 2016.
  8. International Council for Harmonisation. Integrated Addendum to ICH E6(R1): Guideline For Good Clinical Practice E6(R2). 9 November 2016.
  9. The R Foundation for Statistical Computing. R: Regulatory Compliance and Validation Issues. A Guidance Document for the Use of R in Regulated Clinical Trial Environments. Vienna, March 25, 2018. [image] free resource.
  10. Smith D. How R is used by the FDA for regulatory compliance. June 29, 2017.
  11. R/Pharma 2018. Harvard University, 15/16th August, 2018. Program.
  12. Rickert J. Conference Report: R / Pharma 2018. R J. 2018;10(2):579–80.
  13. pharmaR. Validation Overview.
  14. Wolf S. How to Build A Shiny “Truck”! 2018-08-14.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2020-01-09 16:04

@ Helmut
Posting: # 21070
Views: 988
 

 FDA, again

[image] Dear all,

we know that the FDA uses R internally, e.g., in all simulations which lead to its reference-scaling method for NTIDs.

THX to ElMaestro discovering another example. Though I attended the 4th GBHI workshop last month in Bethesda, it is hidden in the backup slides.* Quote:

Q1. What is a sample size to achieve 80% power to pass bioequivalence, assuming CV as 100%, T/R true ratio as 5%?
Q2. With a fixed sample size (n=1,000), what are maximum differences in FEV1 metrics which pass BE with 80% power?

To address questions 1–2;

  • package ‘PowerTOST’ was used to estimate sample size using observed CVs (R ver 3.6.0)
  • Sample size was calculated for reference and treatment group

Of note, Robert Lionberger (Director Office of Research and Standards, Office of Generic Drugs) told me that he visits the forum regularly. :thumb up:


  • Liang Zhao (Division of Quantitative Methods and Modeling, ORS, OGD, CDER/FDA). FEV1 Based Bioequivalence Study for Inhaled Corticosteroids.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Achievwin
☆    

US,
2020-01-27 20:21

@ Helmut
Posting: # 21088
Views: 628
 

 R in regulated environments

I heard the same question during the BE-GBH meeting in Bethesda (and had a coffee Helmut). The short and sweet answer was you can submit your analysis whatever the software you are comfortable FDA runs their own analysis using internal tools, in the event the results differs 9by what magnitude is subjective) it triggers a review question.

Having said that I agree with the original post we need to do some kind of qualification, simple example is to do a cross comparison between R analysis and SAS/WinNonlin analysis (even better a 3 way comparison) This strategy worked in the past. It is better to have some validation than having no validation.

Regards,

Achievwin
ElMaestro
★★★

Belgium?,
2020-01-27 23:13

@ Achievwin
Posting: # 21089
Views: 619
 

 R in regulated environments

» It is better to have some validation than having no validation.

You'll certainly get my vote if you run for president, Achievwin :ok:

This week's list of things I absolutely detest: Corona virus, the which function in R, WIA-WIA interfaces for scanning under Windows 10, the Bee Gees, the smell of my fridge.

Best regards,
ElMaestro
mittyri
★★  

Russia,
2020-01-28 16:00

@ Achievwin
Posting: # 21091
Views: 605
 

 internal tools and modelling

Hi Achievwin,

» The short and sweet answer was you can submit your analysis whatever the software you are comfortable FDA runs their own analysis using internal tools, in the event the results differs 9by what magnitude is subjective) it triggers a review question.

really interesting how do they come with modelling where NONMEM is attacked by many competitors (R: nlmixr or solutions with common ODE solvers; RStan)

Kind regards,
Mittyri
Activity
 Admin contact
20,465 posts in 4,298 threads, 1,414 registered users;
online 6 (0 registered, 6 guests [including 5 identified bots]).
Forum time (Europe/Vienna): 18:18 CEST

Confidence is the result of hours and days and weeks and years
of constant work and dedication.    Roger Staubach

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5