d_labes
★★★

Berlin, Germany,
2012-07-10 13:02
(3386 d 15:02 ago)

Posting: # 8914
Views: 8,038
 

 randomizeBE - randomness control [R for BE/BA]

Dear All!

In the R-package randomizeBE you can control the randomness of the list by a Wald-Wolfowitz runs test as Helmut has suggested in this post :cool:.
This suggestion was aimed to avoid some 'ordered' lists beside the fact that such lists are of course obtainable by dicing, also with low probability.

Internally randomizeBE uses the sequence numbers for the runs test. The runs test is originally formulated for dichotomous input only.
It is generalized to other cases using the dichotomizing of the values via <=median and > median. See f.i. german wikipedia. The english version suggests to omit values = median. IMHO this is not appropriate here where we are dealing with sequences.

Here some observations that the randomness control does not work that good in case of more than 2 sequences.

Example: 3 sequences numbered 1, 2, 3 (substitute a character representation whatever fits your needs. F.i. latin square "ABC", "BCA" and "CAB").
The 'alternating' random lists analogous to the 2x2 case give the following results:
    list                  runs p.value
 1 2 3 1 2 3 1 2 3 1 2 3   0.2502
 1 3 2 1 3 2 1 3 2 1 3 2   0.2502
 2 1 3 2 1 3 2 1 3 2 1 3   0.0438
 2 3 1 2 3 1 2 3 1 2 3 1   0.0438
 3 1 2 3 1 2 3 1 2 3 1 2   0.2502
 3 2 1 3 2 1 3 2 1 3 2 1   0.2502

Majority of them considered as compatible with null hypothesis: sequence is random.

The ordered list on the other hand is correctly considered as 'non-random':
    list                  runs p.value
 1 1 1 1 2 2 2 2 3 3 3 3   0.0028


BTW: You needn't dicing with function RL4() until you get one of the above lists to see the runs p-value. Use the function runs.pvalue() and a numeric representation of the sequences.
Example:
rl <- rep(c(1,2,3),4)
runs.pvalue(rl)


BTW2: The runs test is implemented via normal approximation like the implementations in the R-packages tseries or lawstat. The drawback is that it by design will give only approximate p-values for such small lists as discussed here. If these approximate values are conservative or liberal I don't know.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-10 14:20
(3386 d 13:44 ago)

@ d_labes
Posting: # 8915
Views: 6,164
 

 randomizeBE - randomness control

Dear Detlew,

interesting; I will chew on it time allowing. ;-)

» BTW2: The runs test is implemented via normal approximation like the implementations in the R-packages tseries or lawstat. The drawback is that it by design will give only approximate p-values for such small lists as discussed here. If these approximate values are conservative or liberal I don't know.

In my aged program I used exact values for N ≤200 according to

Swed FS and C Eisenhart
Tables for testing randomness of grouping in a sequence of alternatives
Ann Math Statist 14, 66–87 (1943)

These tables are reproduced in the “Wissenschaftliche Tabellen Geigy 3” p165, 8th Ed 1983. If you you don’t have them let me know and I will send you a scan.

Part of the tables at Google books.

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-07-10 16:58
(3386 d 11:06 ago)

@ Helmut
Posting: # 8917
Views: 6,225
 

 runs test - critical values

Dear Helmut,

» Tables for testing randomness of grouping in a sequence of alternatives
» Ann Math Statist 14, 66–87 (1943) These tables are reproduced in the “Wissenschaftliche Tabellen Geigy 3” p165, 8th Ed 1983. If you you don’t have them let me know and I will send you a scan.

Thanks for your kind offer.

But working with critical values is against the spirit of the implementation.
Here the user can choose an alpha value for the randomness control.

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2012-07-12 10:34
(3384 d 17:30 ago)

@ Helmut
Posting: # 8938
Views: 6,238
 

 randomizeBE - randomness control

Dear Helmut, dear All,

meanwhile I have tried to implement the exact runs distribution.
If I had not made a a big mistake the results are as following for the above presented 'alternating' lists:
(Also a little bug corrected if value==median):crying:
                         asymptotic asymptotic    exact
    list                   normal    normal cc.  runs distri.
 1 2 3 1 2 3 1 2 3 1 2 3   0.2502     0.4209      0.4242
 1 3 2 1 3 2 1 3 2 1 3 2   0.2502     0.4209      0.4242
 2 1 3 2 1 3 2 1 3 2 1 3   0.0658     0.1350      0.1414
 2 3 1 2 3 1 2 3 1 2 3 1   0.2502     0.4209      0.4242
 3 1 2 3 1 2 3 1 2 3 1 2   0.0658     0.1350      0.1414
 3 2 1 3 2 1 3 2 1 3 2 1   0.2502     0.4209      0.4242

cc. denotes a continuity correction to the asymptotic normal distributed statistic like SPSS uses.

For these lists it had to be concluded that the use of the exact distribution worsens the situation. It is more unlikely that the Null "list is a random sequence" is rejected. The use of cc. approximates the exact distribution well but also leads to a conservative test.

The fact that the randomness control doesn't work in all cases seems known.
Just to cite from:
Ying Wang
ECE 461 PROJECT REPORT, MAY 2003
"Nonparametric Tests for Randomness"
online resource

"... From the above analysis, most of the tests are vulnerable to a certain set of sequences, which are deterministic but accepted as random processes."

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2021-07-15 16:07
(94 d 11:57 ago)

@ d_labes
Posting: # 22469
Views: 957
 

 randomness control

Dear Detlew,

excavating an old thread… Recently I saw a deficiency letter of a European assessor.

The Applicant is asked to provide block length used in randomization as clear pattern cannot be deduced from randomization scheme.


Single site, one group. The randomization was:

Subj Seq
  1   TR
  2   TR
  3   RT
  4   TR
  5   RT
  6   TR
  7   RT
  8   RT
  9   RT
 10   TR
 11   RT
 12   TR
 13   TR
 14   RT
 15   RT
 16   TR
 17   TR
 18   TR
 19   RT
 20   RT
 21   TR
 22   TR
 23   RT
 24   RT

What was the question?

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Ohlbe
★★★

France,
2021-07-15 17:59
(94 d 10:05 ago)

@ Helmut
Posting: # 22470
Views: 963
 

 randomness control

Dear Helmut,

» The Applicant is asked to provide block length used in randomization as clear pattern cannot be deduced from randomization scheme

Well, can't the assessor count up to 8 ?

SCNR :-D

Regards
Ohlbe
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2021-07-15 18:32
(94 d 09:32 ago)

@ Ohlbe
Posting: # 22471
Views: 940
 

 randomness control

Dear Ohlbe,

» » The Applicant is asked to provide block length used in randomization as clear pattern cannot be deduced from randomization scheme
»
» Well, can't the assessor count up to 8 ?

Given from the remainder of the deficiency letter, I wouldn’t place a bet on it. :-D

Dif-tor heh smusma 🖖
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2021-07-18 11:15
(91 d 16:49 ago)

@ Helmut
Posting: # 22477
Views: 881
 

 randomness control

Dear Helmut,

as numerous times noticed: Assessors are a strange bunch of subjects :lol3:

Regards,

Detlew
Activity
 Admin contact
21,731 posts in 4,544 threads, 1,543 registered users;
online 4 (0 registered, 4 guests [including 4 identified bots]).
Forum time: Monday 04:05 CEST (Europe/Vienna)

Law is mind without reason.    Aristotle

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5