Challenging data sets [Software]
Dear Yung-jin & Elba,
I’m a little bit short in time, but I give you some ideas on what I have done back in the dark-ages validating my 60,000-lines Pascal code.
A very important point in validation is to challenge the software with extreme data. Are errors trapped, are the results reasonable, etc.
I’ll split it up into two parts: NCA and BE.
NCA
❝ ❝ I was wondering if there is any initiative in this forum to develop "public data reference" in order to evaluate the performance of BEAR (or another package).
❝ Sounds do-able. May need more opinions and time to plan it. There are plenty BE/BA guru members in this Forum from worldwide. They should be able to figure out how-to sooner or later.
I’m a little bit short in time, but I give you some ideas on what I have done back in the dark-ages validating my 60,000-lines Pascal code.

I’ll split it up into two parts: NCA and BE.
NCA
► Data-limits
- Generate a dataset containing only the headers, but do data. Does the software crash or trap the error and come up with a useful (!) error-message?
- Dataset with only one line (i.e, time+concentration).
- Dataset with empty lines or – in R-terms – ‘NAs’.
- Dataset with negative or non-numeric values.
► Cmax/tmax
- Two ore more identical values. Is the first one identified as Cmax/tmax?
► λz-estimation
- Less than three decreasing values.
- Cz ≥ Cz-1.
- 0 or NA within the decreasing values.
- Dataset with known λz.
Example:t(0, 1, 2)
,C(1, 0.5, 0.25)
should give exactly λz=ln(2) and t½=1.
- Simple dataset (use the AUC-example below) and calculate λz from the last three values manually.
- Yi=ln(Ci)=
(4.04305, 2.94444, 1.38629)
- t=(∑ti)/3=
13.33333
and Y=(∑Yi)/3=2.79126
- λz=-∑(ti-t) (Yi-Y)/(ti-t)²=
0.13260
and
t½=ln(2)/λz=0.69315/0.13260
=5.22729
.
- If you want to calculate AUC0-∞ based on the estimated Ĉz (instead of the observed one) you need also the regression’s intercept:
ln(C0)=Y-λzt=2.79126-0.13260×13.33333
=4.55928
All calculations in full precision; final result rounded to the software’s precision.
A note for cross-validators: In PHX/WNL you can request this value in
NCA > Options > Model Settings > [×] Intermediate Output
.
Results in theCore output
:
Intermediate Output
-------------------
Value for Lambda_z: 0.1326, and intercept: 4.5593
► Trapezoidal rules
- Dataset with only one concentration >0.
- Dataset with two identical concentrations (lin-up/log-down should switch to linear).
- Dataset which can easily be calculated manually (KISS!). Example:
t C inc? linear ∑ lin-up/log-down ∑
0 0 NA 0 0 0 0
1 62 y ( 1- 0)(62+ 0)/2= 31 31 apply linear 31 31
2 70 y ( 2- 1)(70+62)/2= 66 97 apply linear 97 97
4 57 n ( 4- 2)(57+70)/2=127 224 ( 4- 2)(70-57)/ln(70/57)=126.55518 223.55518
12 19 n (12- 4)(19+57)/2=304 528 (12- 4)(57-19)/ln(57/19)=276.71272 500.26791
24 4 n (24-12)( 4+19)/2=138 666 (24-12)(19- 4)/ln(19/ 4)=115.52201 615.78992
Perform the manual calculation to the same precision as the software’s output.
► AUC0-∞
- AUC0-∞=AUCt+Cz,obs/λz (based on the observed Cz) should give
666+4/0.13260=696.16555
(linear trapezoidal; extrapolated4.3331%
) and
615.78992+4/0.13260=645.95547
(lin-up/log-down; extrapolated4.6699%
).
- For the second variant we need the predicted Cz,pred=ℯln(C0)–λztz=ℯ
4.55928–0.13260×24
=3.96238
. Then
666+3.96238/0.13260=695.88220
(linear trapezoidal; extrapolated4.2941%
) and
615.78992+3.96238/0.13260=645.67212
(lin-up/log-down; extrapolated4.6281%
).
—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png)
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
![[image]](https://static.bebac.at/pics/Blue_and_yellow_ribbon_UA.png)
Helmut Schütz
![[image]](https://static.bebac.at/img/CC by.png)
The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Complete thread:
- reference dataset elba.romero 2013-07-02 05:08
- Public datasets Helmut 2013-07-02 15:44
- Public datasets yjlee168 2013-07-03 03:10
- Public datasets elba.romero 2013-07-03 05:19
- Public datasets yjlee168 2013-07-03 13:11
- Challenging data setsHelmut 2013-07-03 16:32
- Fantastic idea? ⇒ Challenging data sets yjlee168 2013-07-04 08:06
- Fantastic idea! Helmut 2013-07-04 13:18
- Fantastic idea? ⇒ Challenging data sets yjlee168 2013-07-04 08:06
- Challenging data setsHelmut 2013-07-03 16:32
- Public datasets yjlee168 2013-07-03 13:11
- NIST; cross-validation Helmut 2013-07-03 14:28
- NIST; cross-validation yjlee168 2013-07-03 23:47
- NIST; cross-validation Helmut 2013-07-04 00:45
- NIST; cross-validation yjlee168 2013-07-03 23:47
- Public datasets elba.romero 2013-07-03 05:19
- Public datasets elba.romero 2013-07-03 05:10
- Public datasets yjlee168 2013-07-03 03:10
- Eight Reference Datasets (2×2×2 ☑) Helmut 2014-10-06 02:24
- What??? ElMaestro 2014-10-06 13:20
- What??? Helmut 2014-10-06 17:08
- What??? ElMaestro 2014-10-06 17:31
- What the heck‽ Helmut 2014-10-06 19:05
- What??? ElMaestro 2014-10-06 17:31
- What??? Helmut 2014-10-06 17:08
- link of electronic supplement material? yjlee168 2014-10-07 09:53
- link of electronic supplement material? nobody 2014-10-07 10:59
- Thanks! yjlee168 2014-10-07 11:35
- Thanks! nobody 2014-10-07 11:44
- Direct links & hint Helmut 2014-10-07 13:54
- Thanks! nobody 2014-10-07 11:44
- Thanks! yjlee168 2014-10-07 11:35
- Replicated designs are relevant, too ElMaestro 2014-10-07 12:36
- Replicated designs are relevant, too yjlee168 2014-10-07 13:00
- RSABE: SAS and PHX/WNL only? Helmut 2014-10-07 14:47
- for the future? yjlee168 2014-10-08 10:05
- Star Trek: The Next Generation Helmut 2014-10-09 00:26
- Star Trek: The Next Generation d_labes 2014-10-09 08:09
- Star Trek: The Next Generation nobody 2014-10-09 08:23
- Star Trek: The Next Generation yjlee168 2014-10-09 11:06
- Some comments ElMaestro 2014-10-09 11:52
- Some comments yjlee168 2014-10-09 12:18
- Proc MIXED - start value d_labes 2014-10-10 08:35
- Proc MIXED - start value ElMaestro 2014-10-10 11:44
- Proc MIXED - start value yjlee168 2014-10-11 21:09
- Proc MIXED - start value ElMaestro 2014-10-11 21:18
- Proc MIXED - start value d_labes 2014-10-10 08:35
- Some comments yjlee168 2014-10-09 12:18
- Optimizer settings Helmut 2014-10-09 14:08
- Optimizer settings nobody 2014-10-09 15:32
- Optimizer settings yjlee168 2014-10-09 18:14
- FDA’s model… Helmut 2014-10-09 18:34
- Some comments ElMaestro 2014-10-09 11:52
- Star Trek: The Next Generation Helmut 2014-10-09 00:26
- for the future? yjlee168 2014-10-08 10:05
- RSABE: SAS and PHX/WNL only? Helmut 2014-10-07 14:47
- Replicated designs are relevant, too yjlee168 2014-10-07 13:00
- link of electronic supplement material? nobody 2014-10-07 10:59
- Just received a letter… Helmut 2014-11-03 14:54
- 11 Reference Datasets (2-group parallel ☑) Helmut 2014-12-09 15:51
- What??? ElMaestro 2014-10-06 13:20
- Public datasets Helmut 2013-07-02 15:44