Software validation [🇷 for BE/BA]
❝ Try to put yourself in the shoes of the software writer.
Oh, I took these boots on many years ago. And they got muddy almost immediatelly. I tried to get a numerical approximation of the cdf of the t-distribution which lead me to the gamma-function and headaches. Finally I ended up with numerical approximations.[1,2]
❝ […] whatever the algo does it is not exact. It is approximate, but how good that approximation actually is is unknown.
Sure.
❝ […] we have absolutely no way of telling which one is the right one until some clever guy works out the integrals exactly.
Definitely not me. I’m getting more stupid every day.
❝ I'll buy you a Mozartkugel if you do it.
THX, no needs.
❝ Decompile your recent acquisition and check if they one way or another use the ASA243.
I guess the SW was built using the Borland C++ compiler. You know that reverse engineering of software is a breach of the license?
BTW, according to the manual the ‘Illinois method’ (Algorithm AS 184, FORTRAN source) is used. I assume AS 243[3] is ‘better’ than AS 184.[4] But to which extent? Is it my job to go through pros and cons of algorithms?
❝ If they do, why validate?
First, different algos are used. Second, I have to validate software - or not?
At the product's site I found a nice statement:
These sets of solutions were reviewed by Janet Elashoff, who checked for consistency, face validity, and for computational accuracy against other sources.
Face validity?What I actually wanted to do was to check my R code (which is also part of bear) with a piece of SW of high reputation. Checking the sample size I wasn’t satisfied (only integers), therefore I opted to do it the other way 'round (power). It made me angry that there is no (easy) way to obtain more than 4 significant digits. By this external validation is effectively prevented.
Another example from the dirt track of SW validation: Diletti et al.[5] published exact samples size tables for BE ranges of 0.9-11 and 0.7–1.43. When trying to validate my code against these tables I found small discrepancies (only at low CVs and close to the acceptance range). By trial-and-error I got the solution: I could reproduce tables only if setting the upper acceptance limit not to the reciprocal of the lower AL (0.9-1, 0.7-1) but to exactly 1.1111 or 1.4286…
BTW, nitpicking Diletti’s Table 1[6] (power 70%, T/R=1, CV=7.5%) and nQuery 7 give n=4, whilst my R code, StudySize 2.0.1, and FARTSSIE 1.6 come up with n=6. Now what? Go with a democratic vote of 3:2 for 6?
- Gardiner DA and BF Bombay
An Approximation to Student's t
Technometrics 7(4), 71–2 (1965)
- Abramowitz M and IA Stegun
Handbook of mathematical functions
National Bureau of Standards, Applied Mathematics Series – 55, Washington, D.C., pp 948–9 (10th Printing with corrections 1972)
online resource
- RV Lenth
Statistical Algorithms. Algorithm AS 243: Cumulative Distribution Function of the Non-Central t Distribution
Applied Statistics 38(1), 185–9 (1989)
- Bohrer R, Schervish M, and J Sheft
Statistical Algorithms. Algorithm AS 184: Non-Central Studentized Maximum and Related Multiple-t Probabilities
Applied Statistics 31(1), 309–17 (1982)
- Diletti E, Hauschke D, and VW Steinijans
Sample size determination: Extended tables for the multiplicative model and bioequivalence ranges of 0.9 to 1.11 and 0.7 to 1.43
Int J Clin Pharmacol Ther Toxicol 30/Suppl.1, S59-62 (1992)
- Diletti E, Hauschke D, and VW Steinijans
Sample size determination for bioequivalence assessment by means of confidence intervals
Int J Clin Pharmacol Ther Toxicol 29(1), 1-8 (1991)
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- R-Code for Power in 2×2 Cross-over Helmut 2006-12-28 21:33 [🇷 for BE/BA]
- Power in Diletti's sample size table Helmut 2007-01-01 23:59
- Approximate Power Helmut 2007-01-06 15:17
- Power Calculation for replicate crossover design mathews 2008-04-03 13:19
- Power Calculation for replicate crossover design Helmut 2008-04-03 13:49
- Power Calculation for replicate crossover design mathews 2008-04-04 13:21
- Power Calculation for replicate crossover design Helmut 2008-04-03 13:49
- Exact Power Helmut 2009-09-25 19:20
- Exact Power ElMaestro 2009-09-27 23:27
- Software validationHelmut 2009-09-28 15:21
- All the important answers are here! ElMaestro 2009-09-28 19:12
- All the important answers are here! Helmut 2009-09-28 19:52
- All the important answers are here! ElMaestro 2009-09-28 20:18
- All the important answers are here! yjlee168 2009-09-28 21:30
- All the important answers are here! Helmut 2009-09-28 23:18
- 42! SASophylistic power oracle d_labes 2009-09-29 13:37
- Al Gore Rhythms Helmut 2009-09-29 14:52
- Al Gore Rhythms ElMaestro 2009-09-29 18:14
- Al Gore Rhythms yjlee168 2009-09-29 23:55
- Al Gore Rhythms Helmut 2009-09-29 14:52
- 42! SASophylistic power oracle d_labes 2009-09-29 13:37
- All the important answers are here! yjlee168 2009-09-29 23:49
- All the important answers are here! Helmut 2009-09-28 23:18
- All the important answers are here! Helmut 2009-09-28 19:52
- All the important answers are here! ElMaestro 2009-09-28 19:12
- Software validationHelmut 2009-09-28 15:21
- Exact Power ElMaestro 2009-09-27 23:27
- Power Calculation for replicate crossover design mathews 2008-04-03 13:19
- Approximate Power Helmut 2007-01-06 15:17
- R-Code for Power in 2×2 Cross-over KDA 2019-01-23 04:24
- R-Code for Power in 2×2 Cross-over Ohlbe 2019-01-27 18:24
- R-Code for Power in 2×2 Cross-over Brus 2021-12-02 15:00
- R-Code for Power in 2×2 Cross-over ElMaestro 2021-12-02 15:15
- R-Code for Power in 2×2 Cross-over Helmut 2021-12-02 17:30
- R-Code for Power in 2×2 Cross-over Helmut 2021-12-02 16:57
- R-Code for Power in 2×2 Cross-over ElMaestro 2021-12-02 15:15
- Power in Diletti's sample size table Helmut 2007-01-01 23:59