Bioequivalence and Bioavailability Forum

Dear all,

since occasionally the question is asked whether is accepted in regulatory environments, here the answers:

1. Yes, if the installation and code is validated.
   
2. If you are an expert and understand what #1 means with all its pitfalls,
   • stop reading the post here;
     
   • otherwise, get a cup of coffee and continue.

---

Any (!) system has to pass three levels (definitions given in ISO 9000 and by the FDA in 1999 and 2002):

1. Installation Qualification (IQ)
      The system is compliant with appropriate codes and approved design intentions, and that vendor’s recommendations are suitably considered.
   
2. Operational Qualification (OQ)
      The system is capable of consistently operating within stated limits and tolerances.
   
3. Performance Qualification (PQ)
      The system is meeting all release requirements for functionality and safety and that procedures are effective and reproducible.

Whereas #1 generally is the job of the vendor, #2 can be shared between the vendor and user, the responsibility for #3 lies entirely in the hands of the user.
   Since the source code of commercial software is not accessible to the user, only a black box validation can be performed (i.e., compare results of reference data sets with published ones). Open source software (e.g., R, GNU Octave, …) allows – by definition – a white box validation but this can be tough (requires an experienced coder). Hence, in practice most users opt for a black box validation as well. See also reference data sets for various designs in BE.^1,2,3

Quotes from relevant documents:

• ICH E9⁴
     The credibility of the numerical results of the analysis depends on the quality and validity of the methods and software (both internally and externally written) used both for data management (data entry, storage, verification, correction and retrieval) and also for processing the data statistically. Data management activities should therefore be based on thorough and effective standard operating procedures. The computer software used for data management and statistical analysis should be reliable, and documentation of appropriate software testing procedures should be available.
  
• EMA, Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials⁵
     Records of system validation including requirements, design, installation, access and security, testing (e.g. user acceptance testing, installation, operational and performance testing), training and controlled release for use should be maintained.
  
• FDA, Statistical Software Clarifying Statement⁶
     FDA does not require use of any specific software for statistical analyses, and statistical software is not explicitly discussed in Title 21 of the Code of Federal Regulations [e.g., in 21CFR part 11]. However, the software package(s) used for statistical analyses should be fully documented in the submission, including version and build identification.
  
• WHO, TRS 996, Annex 9⁷
     Computer systems should be qualified and validated (hardware, software, networks, data storage systems and interfaces. Qualification is the planning, carrying out and recording of tests on equipment and systems which form part of the validated process, to demonstrate that the equipment or system will perform as intended.
  
• Addendum to ICH E6(R1)⁸
     Ensure the integrity of the data including any data that describe the context, content, and structure. This is particularly important when making changes to the computerized systems, such as software upgrades or migration of data.

When it comes to , a lot is going on – especially with support of regulators, the academia, and innovators.^9,10,11,12 Sebastian Wolf of Roche Diagnostics presented¹¹ a 500,000+ lines Shiny application.¹⁴
   One reason for the increasing popularity of in the industry is that the statistical curriculum gradually shifted from SAS to and nowadays graduates are at least “bilingual” (nerds are even proficient in FORTRAN, C, C++). Young statisticians are no more willing to accept a “SAS only” working environment.

---

1. Schütz H, Labes D, Fuglsang A. Reference Datasets for 2-Treatment, 2-Sequence, 2-Period Bioequivalence Studies. AAPS J. 2014; 16(6): 1292–7. doi:10.1208/s12248-014-9661-0. Free PMC.
   
2. Fuglsang A, Schütz H, Labes D. Reference Datasets for Bioequivalence Trials in a Two-Group Parallel Design. AAPS J. 2015; 17(2): 400–4. doi:10.1208/s12248-014-9704-6. Free PMC.
   
3. Schütz H, Tomashevskiy M, Labes D, Shitova A, González-de la Parra M, Fuglsang A. Reference Data­sets for Studies in a Replicate Design intended for Average Bioequivalence with Expanding Limits. AAPS J. 2020; 22(2):44. doi:10.1208/s12248-020-0427-6.
   
4. International Council for Harmonisation. Statistical Principles for Clinical Trials E9. 5 February 1998.
   
5. European Medicines Agency, GCP Inspectors Working Group. Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. London, 9 June 2010.
   
6. US FDA. Statistical Software Clarifying Statement. May 6, 2015. Study Data Standards.
   
7. World Health Organization. Technical Report Series No. 996, Annex 9. Guidance for organizations performing in vivo bioequivalence studies. Geneva, May 2016.
   
8. International Council for Harmonisation. Integrated Addendum to ICH E6(R1): Guideline For Good Clinical Practice E6(R2). 9 November 2016.
   
9. The R Foundation for Statistical Computing. R: Regulatory Compliance and Validation Issues. A Guidance Document for the Use of R in Regulated Clinical Trial Environments. Vienna, March 25, 2018. Open access.
   
10. Smith D. How R is used by the FDA for regulatory compliance. June 29, 2017.
    
11. R/Pharma 2018. Harvard University, 15/16^th August, 2018. Program.
    
12. Rickert J. Conference Report: R / Pharma 2018. R J. 2018;10(2):579–80.
    
13. pharmaR. Validation Overview.
    
14. Wolf S. How to Build A Shiny “Truck”! 2018-08-14.