Ravi
★    

India,
2012-06-15 15:39
(4304 d 17:43 ago)

Posting: # 8744
Views: 13,779
 

 USFDA requirement for randomization [Regulatives / Guidelines]

Hi All,

I have a query regarding USFDA requirement for randomization.

Do we need to submit the SAS code and seed used for randomization schedule generation to FDA as a part of our submission.

What if this schedule is being generated by some other software (Like GRand). What kind of document we need to submit to USFDA?

Regards,
Ravi Shankar Pandey

Thanks & Regards
Ravi Pandey
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-15 19:39
(4304 d 13:43 ago)

@ Ravi
Posting: # 8745
Views: 12,245
 

 randomization: validation

Dear Ravi!

I don’t think that you have to submit the code, but like all software it should be validated. I did it once for my own randomization in HP-UX Pascal based on the Wald–Wolfowitz runs test.
For SAS see here; set-up in other software is easy as well. For [image] have a look at package tseries runs.test().

A simple randomization for a 2×2 cross-over with balanced sequences and random frequency of runs:

library(tseries)
#set.seed(15062012)              # uncomment this line for testing
size <- 24
sig  <- 0                        # break condition
while (sig < 0.05) {             # … until nonsignificant runs
  x <- 1                         # break condition
  while (sum(x) != 0) {          # … until balanced
    x <- sign(rnorm(size))       # random: -1 or +1
  }
  runs <- length(rle(x)$lengths) # No. of runs
  x[x==1]  <- "RT"               # nicer ‘looks’
  x[x==-1] <- "TR"
  x   <- as.factor(x)
  rt  <- runs.test(x)            # perform test
  sig <- rt$p.value              # p-value
  }
x                                # display randomization
rt                               # show test results


Should give (with the testing seed):
[1] RT RT RT TR TR TR TR RT RT TR TR TR RT RT TR TR RT TR TR RT RT RT TR RT
Levels: RT TR

        Runs Test

data:  x
Standard Normal = -0.8348, p-value = 0.4038
alternative hypothesis: two.sided

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-06-16 15:10
(4303 d 18:12 ago)

@ Helmut
Posting: # 8753
Views: 11,823
 

 randomization: runs test? validation?

Dear Helmut!

Q1: What is the rationale behind using the runs test :confused:.
What did a randomisation based on this achieve extra compared to a randomization using lots, a hat and a NLYW dancing around it while drawing from the lots?

Q2: What can we do in validating a randomisation software or hardware as described above?
Remember your nice own post of a scene from Dilbert's life.
[image]

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 16:00
(4303 d 17:22 ago)

@ d_labes
Posting: # 8755
Views: 11,883
 

 randomization: runs test? validation?

Dear Detlew!

❝ Q1: What is the rationale behind using the runs test :confused:.

❝ What did a randomisation based on this achieve extra compared to a randomization using lots, a hat and a NLYW dancing around it while drawing from the lots?


A1: A deficiency letter 20+ years ago. The randomization (by chance) was:
RT TR RT TR RT TR RT TR RT TR RT TR (12 runs)
We have “learned” that alternating sequences were not considered random by the authority (as would be the other extreme):
RT RT RT RT RT RT TR TR TR TR TR TR (2 runs)
Both schemes give the same significant p of 0.002465…

A2: Well…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-06-16 18:11
(4303 d 15:11 ago)

@ Helmut
Posting: # 8760
Views: 11,750
 

 probability nearly zero?

Dear Helmut!

❝ A1: A deficiency letter 20+ years ago. The randomization (by chance) was:

RT TR RT TR RT TR RT TR RT TR RT TR (12 runs)


Wow! What is the probability to obtain this by chance?

BTW: My randomisation approach was always to randomize by blocks (block size 4 or if not possible by 2). Never obtained such an 'extreme' randomlist.
Random freaks may use varying block sizes chosen at random.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 19:08
(4303 d 14:14 ago)

@ d_labes
Posting: # 8761
Views: 11,720
 

 probability nearly zero!

Dear Detlew!

❝ ❝ The randomization (by chance) was:

❝ ❝ RT TR RT TR RT TR RT TR RT TR RT TR (12 runs)


❝ Wow! What is the probability to obtain this by chance?


~1:500 (including the reverse order); try it yourself:
set.seed(16062012)
size       <- 12
sims       <- 1e7
y          <- rep(c(-1,1), size/2)
z          <- rep(c(1,-1), size/2)
alternate  <- 0
for (i in 1:sims) {
  x <- 1
  while (sum(x) != 0) x <- sign(rnorm(size))
  if (all(x == y) | all(x == z)) alternate <- alternate+1
}
cat(paste("‘alternating’ runs:", format(100*alternate/sims, digits=15), "%\n"))


Nine nine nine nine nine nine” wouldn’t pass the algo, since not balanced. :-D

❝ BTW: My randomisation approach was always to randomize by blocks (block size 4 or if not possible by 2). Never obtained such an 'extreme' randomlist.


Chance for block size 4 ~0.9% and for block size 2 ~3%. ;-)

❝ Random freaks may use varying block sizes chosen at random.


Too stupid to code that…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-07-06 11:54
(4283 d 21:28 ago)

@ Helmut
Posting: # 8899
Views: 11,181
 

 R-package randomizeBE

Dear All!

❝ ❝ Random freaks may use varying block sizes chosen at random.


❝ Too stupid to code that…


We proudly present on CRAN now :cool::

A little (futile) randomizer for BE studies
R-package randomizeBE


Features:
  • Also free (think 'free' not like 'free beer' :party: but 'free in speech')
    like this one of ElMaestro's alter ego and open source
  • not restricted to 2x2 crossover
  • block randomization
  • randomly chosen block sizes from a list of block sizes
  • control of randomness via Wald-Wolfowitz runs test (may be switched off)

Here some simulation results concerning probability of obtaining 'alternating' random lists in 2x2 crossover (randomness control switched off):
1 000 000 simulations, alpha=0.05 for the runs test

                     runs test
subjects blocksize  %significant  %alternating
----------------------------------------------
12         0 (12)       2.62          0.206
           6            3.54          0.515
           4            5.57          0.928
           2 4          9.47          1.576    randomly chosen block sizes
----------------------------------------------
24         0 (24)       5.95         <0.0001
          12            6.28          0.0001
           4           19.33          0.0039
           4 6         14.90          0.0023   randomly chosen block sizes
----------------------------------------------

Lessons learned:
  • If sample size not too low and block size also the probability to 'randomize' an alternating list is vanishing small.
  • The randomness control via runs test with significance level alpha=0.05 is much to stringent (much more lists were considered non random than our alternates).
  • The question to control/not control the randomness of the random list should not concern us much.
    IMHO we are throwing a spanner in the works of chance if we do it too serious.
BTW: Helmut how did you act on the deficiency letter 20+ years ago? Could you convince them that dicing 6 times in series 6 pips (or six times 1 pips) although has low probability is possible?


Edit: Package linked. [Helmut]

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-06 14:51
(4283 d 18:31 ago)

@ d_labes
Posting: # 8901
Views: 11,197
 

 R-package randomizeBE

Dear Detlew!

R-package randomizeBE


THX, wonderful!

❝ BTW: Helmut how did you act on the deficiency letter 20+ years ago?


I implemented the WW-test and generated a new randomization if p<0.05.

❝ Could you convince them that dicing 6 times in series 6 pips (or six times 1 pips) although has low probability is possible?


No way. At that time CROs / clinical sites were randomly [sic] chosen by the Austrian Ministry of Health for GCP (facility) inspections. My rep("RT", "TR", 6) in one study was a ‘minor finding’ – in the next inspection they were very happy about my changed code. I wonder how the MoH’s random generator worked. ;-)

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-06-16 15:31
(4303 d 17:51 ago)

@ Helmut
Posting: # 8754
Views: 11,773
 

 randomization: validation

Hi HS,

I don't think I would use a runs test prior to releasing a randomisation scheme. This screws the principle of randomness; even random sequences will have a significant runs test occasionally and still be 'random' whatever that words means.

ElMaestro's approach:
  1. Populate an array will RT and TR in equal numbers. E.g. index 1-12 are RT and 13-24 are TR if we have 24 subjects.
  2. Loop through all entries 1-24 and swap each exactly once with a random index between 1 and 24. The random index must come from a uniform distribution. A NLYW dancing around a hat with 24 coloured and numbered balls may be used; but be advised that NLYW's may display unpredictable behaviour and variations in time and space as regards willingness to dance around hats and draw balls.
  3. Release scheme without testing for randomness.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 16:42
(4303 d 16:40 ago)

@ ElMaestro
Posting: # 8756
Views: 11,708
 

 randomization: runs test

Hi ElMaestro!

❝ I don't think I would use a runs test prior to releasing a randomisation scheme. This screws the principle of randomness; even random sequences will have a significant runs test occasionally and still be 'random' whatever that words means.


OK, but see my A1 above.

❝ ElMaestro's approach: […]



Nice; especially your remarks about NLYWs. Concerning your #3:
require(tseries)
set.seed(16062012)
size      <- 24
sims      <- 1e6
nonrandom <- 0
for (i in 1:sims) {
  x <- 1
  while (sum(x) != 0) x <- sign(rnorm(size))
  x[x ==  1] <- "RT"
  x[x == -1] <- "TR"
  x   <- as.factor(x)
  rt  <- runs.test(x)
  sig <- rt$p.value
  if (sig < 0.05) nonrandom <- nonrandom + 1
}
cat(paste("‘nonrandom’ runs: ", format(100*nonrandom/sims, digits = 4), "%\n"))


Gave 5.895% on your matey’s machine. Don’t know whether 20+ years after the deficiency letter mentioned above inspectors would still react similarly.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-06-16 16:48
(4303 d 16:34 ago)

@ Helmut
Posting: # 8757
Views: 11,677
 

 randomization: runs test

Hi HS,

a quickie:

Scheme222=function(NSeq1, NSeq2, Seed, Sillyfactor)
{
   set.seed(Seed)
   A=c(rep("RT", NSeq1), rep("TR",NSeq2))

   for (j in 1:Sillyfactor)
   {
   swaps = sample(1:length(A), length(A), replace=T)
   for (i in 1:length(A))
   {
     tmp = A[i]
     A[i]=A[swaps[i]]
     A[swaps[i]]=tmp
   } ## i loop
   } ## j loop
   print(A)
}

Scheme222(12,12, 87654321, 123)


I have no idea why regulators strictly would not consider an alternating sequence random; I think they must have assumed the sequence had not been through a number-cruncher or NLYW?

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 17:10
(4303 d 16:12 ago)

@ ElMaestro
Posting: # 8758
Views: 11,958
 

 randomization: runs test

Ahoy!

❝ a quickie:


Nice.

❝ I have no idea why regulators strictly would not consider an alternating sequence random; I think they must have assumed the sequence had not been through a number-cruncher or NLYW?


Referring to my code in Pascal maybe they thought about this guy; neither a number-cruncher* nor a W at all.



Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-06-16 21:12
(4303 d 12:10 ago)

@ Helmut
Posting: # 8762
Views: 11,668
 

 very cool coding style

Hi HS,

while (sum(x) != 0) x <- sign(rnorm(size))


a little off topic, but it took me some time to get my head around this code line of yours. Now that I (think I) get it, I must say this is an incredibly elegant way of achieveing a lot with minimum coding. It completely permutes equal numbers of TR and RT (or -1's and 1's). Very, very elegant style!!! Me likey! :cool::cool::cool:

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-17 15:22
(4302 d 18:00 ago)

@ ElMaestro
Posting: # 8763
Views: 11,567
 

 Better a good steal than a bad reinvention

Hi ElMaestro!

Thanks for the flowers, but I’m (still…) a lousy R-coder. I boiled it down to a one-liner because the commented version was already given there. The “sign(rnorm(foo))” comes from an example in package tseries.
For a single run your quickie is fine, but for simulations I would opt for mine (~140 times faster). ;-)

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-06-17 21:58
(4302 d 11:24 ago)

@ Helmut
Posting: # 8764
Views: 11,581
 

 Better a good steal than a bad reinvention

Hi HS,

❝ For a single run your quickie is fine, but for simulations I would opt for mine (~140 times faster). ;-)


You are reading me wrongly. I would not set the silly factor to 123, but to 1, which is adequate to ensure complete randomisation.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 15:26
(4301 d 17:56 ago)

@ ElMaestro
Posting: # 8770
Views: 11,450
 

 Sorry!

Hi ElMaestro!

❝ You are reading me wrongly. I would not set the silly factor to 123, but to 1, which is adequate to ensure complete randomisation.


Sorry, didn’t realize that ‘silly’ has to be taken literally. Why did you include this loop at all? To slow down the algo?

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-06-18 17:19
(4301 d 16:03 ago)

@ Helmut
Posting: # 8772
Views: 11,502
 

 Sorry!

Hi HS,

❝ Sorry, didn’t realize that ‘silly’ has to be taken literally. Why did you include this loop at all? To slow down the algo?


I had this discussion before with a client. The thing is, after a single pass through all the subjects, the sequence designation is as random as it wil be after 123 passes etc.
But I think this is difficult to somehow grasp out there. Key to realising it is that for each swap, there is exactly 50% chance that the subject will remain his/her firstly designated sequence, and 50% chance that it will be the other. After the full pass through all the subjects they are therefore fully randomised. But due to my lack of communication skills I failed to get that message across properly. So, it ended with a completely meaningless discussion like:
EM: "Would you like me to randomise the sequence XYZ times?"
Client: "Yes, please. They need to be fully randomised."
… and then I just smiled to my client and banged my head into the wall a couple times and took a handful of Schützomycin. It helped a little. They let me out of the loony bin after just 5 months.

Pass or fail!
ElMaestro
d_labes
★★★

Berlin, Germany,
2012-06-18 17:38
(4301 d 15:44 ago)

@ ElMaestro
Posting: # 8774
Views: 11,913
 

 Dancing with the dice

Hi Mon Captain, dear Helmut,

❝ ❝ while (sum(x) != 0) x <- sign(rnorm(size))


❝ ... It completely permutes equal numbers of TR and RT (or -1's and 1's).


Not really. It rolls the 'binary dice' 24 times again and again until the vector of +1's and -1's has the same number of both entries (is 'balanced').

Thus while being programmed along an "incredibly elegant way of achieving a lot with minimum coding" not so very effective concerning computational time if one considers what is intended (have seen more than 20 loop passes in a short experimentation).

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2012-06-18 18:30
(4301 d 14:52 ago)

@ d_labes
Posting: # 8775
Views: 11,499
 

 Silly's randomization 2x2 xover

Dear both!

what about this one-liner (ok, ok its a 3-liner :cool:):
set.seed(123987)
nsubj <- 24
(c(rep("RT", trunc(nsubj/2)), rep("TR", nsubj-trunc(nsubj/2)))[sample(1:nsubj)])

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 19:14
(4301 d 14:08 ago)

@ d_labes
Posting: # 8777
Views: 11,549
 

 Silly's randomization 2x2 xover

Dear Simulants!

❝ what about this one-liner…


Hey, ~20% faster than mine; great code. :-D Nice method to get balance. The time limiting factor seems to be the resampling.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 19:42
(4301 d 13:40 ago)

@ d_labes
Posting: # 8778
Views: 11,461
 

 Clumsy code

Dear Simulants!

❝ […] 'binary dice'…


If you flatten a die – isn’t that what we call a coin in Austria? :-D

❝ … 24 times again and again until the vector of +1's and -1's has the same number of both entries (is 'balanced').


Exactly.

❝ Thus while being programmed along an "incredibly elegant way of achieving a lot with minimum coding" not so very effective concerning computational time if one considers what is intended (have seen more than 20 loop passes in a short experimentation).


Sorry, couldn’t do better. Calls:
Min.  1st Qu.  Median  Mean  3rd Qu. Max.
 1      2        4     6.208   8      79


[image]

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2012-06-18 20:01
(4301 d 13:21 ago)

@ Helmut
Posting: # 8779
Views: 11,737
 

 Clumsy code

Gentlemen,

how bout
sample(c(rep("AB",12), rep("BA",12)))

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 20:56
(4301 d 12:26 ago)

@ ElMaestro
Posting: # 8780
Views: 11,573
 

 Speed kills

Simulants!

❝ how bout

sample(c(rep("AB",12), rep("BA",12)))


Aha! Almost a dead race with Detlew’s code; I’m only in the amateur’s league. ‘Nonrandom’ runs 5~6%.
  1. Your honorable’s one-liner: 115.68”
  2. Detlew’s: 116.18”
  3. Your 1st attempt, silly loop removed: 127.71”
  4. Mine: 148.72”

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
jag009
★★★

NJ,
2012-09-17 23:46
(4210 d 09:36 ago)

@ Helmut
Posting: # 9231
Views: 10,121
 

 Speed kills

Hi all,

If one wants to plan a randomization scheme for a 4 way study, how does one go about assigning the treatments order in each of the 4 sequence (yes, use 4)? A 4 treatment study will have 24 possible sequence so the selection of which 4 sequences to use should be randomly selected, correct? So I should have a SAS routine to select the sequences first and then another to generate the study randomization scheme?

Thanks
John
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,636 registered users;
85 visitors (0 registered, 85 guests [including 6 identified bots]).
Forum time: 08:22 CET (Europe/Vienna)

With four parameters I can fit an elephant,
and with five I can make him wiggle his trunk.    John von Neumann

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5