## Software validation – again and again [Power / Sample Size]

Hi sury,

» we have not yet finalized the SOP, and moreover we use the SAS Software procedure "PROC POWER" for the estimation of the sample size.

OK.

» But however we wanted it to be more accurate sample size for that we want to include the deletti tables.

I wonder why you were asking here for the Diletti tables. The reference is readily available in the manual of Proc POWER
Proc POWER employs Owen’s Q-function1 – as does PowerTOST by default. AFAIK, OwenQ is a still undocumented function in SAS… Proc POWER is available in SAS since v9.1 (2003). Hence, Diletti et al. wrote in 1991:

The exact form of these integrals and an algorithm for their calculation has been provide by Owen; this algorithm has been implemented to create the figures and tables in this paper.

We don’t know which software and code they used. Edgar Diletti and Volker Steinijans are retired; ask Dieter Hauschke.

» » These tables […] are hopelessly behind state of the art!
» » » NB, if you are interested in reference-scaling […]»
» Yes

For all reference-scaling methods you need simulations and have to write your own SAS-code. Expect run-times of many, many hours. The two Lászlós wrote their code in MATLAB, which is only slightly faster than SAS. Hence, for years they recommend PowerTOST themselves.

» » Hence, I strongly suggest […] package PowerTOST.
»
» yes we are thinking to opt this package. But however we have a concern whether regulatory will accept it or not.
» By then we are doubtfull of using this package.

Since PowerTOST is open source you (or an -expert) can perform a white-box validation. Additionally, PowerTOST contains not only reference data sets from the literature but also scripts to validate your installation. See also this thread.
Maybe you are interested in this document.2

» Sometime ago I have faced the concern regarding the Software validation from the regulatory.

Any software used in a GCP-environment has to be validated. Have you done that with SAS?* No, I don’t mean just IQ, OQ, PQ! Good luck with commercial software where the source code (allowing white-box validation) is not accessible. You could only perform cross-validation (black-box) with reference data sets. If you don’t have more than one software at hand, you have to believe (!) that what is reported for the reference data set is correct.

1. Owen DB. A special case of a bivariate non-central t-distribution. Biometrika. 1965;52(3/4):437–46. doi:10.2307/2333696.
2. The R Foundation for Statistical Computing. R: Regulatory Compliance and Validation Issues. A Guidance Document for the Use of R in Regulated Clinical Trial Environments. Vienna, March 2018. open access.

• Stephen Senn told me this story:
He discovered that after an update of SAS the Welch-Satterthwaite’s degrees of freedom (required in parallel designs with unequal group sizes and/or unequal variances) were faulty. Scary, cause such data sets are the rule in phase III. He phoned the SAS-Institute and got the responsible software engineer on the line. After inspecting the source within minutes he confirmed the bug. However, it took SAS half a year to roll out a patch…
So much for that.

Dif-tor heh smusma 🖖
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes