Ravi
★    

India,
2012-06-15 13:39

Posting: # 8744
Views: 11,893
 

 USFDA requirement for randomization [Regulatives / Guidelines]

Hi All,

I have a query regarding USFDA requirement for randomization.

Do we need to submit the SAS code and seed used for randomization schedule generation to FDA as a part of our submission.

What if this schedule is being generated by some other software (Like GRand). What kind of document we need to submit to USFDA?

Regards,
Ravi Shankar Pandey

Thanks & Regards
Ravi Pandey
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-15 17:39

@ Ravi
Posting: # 8745
Views: 10,679
 

 randomization: validation

Dear Ravi!

I don’t think that you have to submit the code, but like all software it should be validated. I did it once for my own randomization in HP-UX Pascal based on the Wald–Wolfowitz runs test.
For SAS see here; set-up in other software is easy as well. For R have a look at package tseries runs.test().

A simple randomization for a 2×2 cross-over with balanced sequences and random frequency of runs:

library(tseries)
#set.seed(15062012)              # uncomment this line for testing
size <- 24
sig  <- 0                        # break condition
while (sig < 0.05) {             # … until nonsignificant runs
  x <- 1                         # break condition
  while (sum(x) != 0) {          # … until balanced
    x <- sign(rnorm(size))       # random: -1 or +1
  }
  runs <- length(rle(x)$lengths) # No. of runs
  x[x==1]  <- "RT"               # nicer ‘looks’
  x[x==-1] <- "TR"
  x   <- as.factor(x)
  rt  <- runs.test(x)            # perform test
  sig <- rt$p.value              # p-value
  }
x                                # display randomization
rt                               # show test results


Should give (with the testing seed):
[1] RT RT RT TR TR TR TR RT RT TR TR TR RT RT TR TR RT TR TR RT RT RT TR RT
Levels: RT TR

        Runs Test

data:  x
Standard Normal = -0.8348, p-value = 0.4038
alternative hypothesis: two.sided

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-06-16 13:10

@ Helmut
Posting: # 8753
Views: 10,233
 

 randomization: runs test? validation?

Dear Helmut!

Q1: What is the rationale behind using the runs test :confused:.
What did a randomisation based on this achieve extra compared to a randomization using lots, a hat and a NLYW dancing around it while drawing from the lots?

Q2: What can we do in validating a randomisation software or hardware as described above?
Remember your nice own post of a scene from Dilbert's life.
[image]

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 14:00

@ d_labes
Posting: # 8755
Views: 10,372
 

 randomization: runs test? validation?

Dear Detlew!

» Q1: What is the rationale behind using the runs test :confused:.
» What did a randomisation based on this achieve extra compared to a randomization using lots, a hat and a NLYW dancing around it while drawing from the lots?

A1: A deficiency letter 20+ years ago. The randomization (by chance) was:
RT TR RT TR RT TR RT TR RT TR RT TR (12 runs)
We have “learned” that alternating sequences were not considered random by the authority (as would be the other extreme):
RT RT RT RT RT RT TR TR TR TR TR TR (2 runs)
Both schemes give the same significant p of 0.002465…

A2: Well…

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-06-16 16:11
(edited by d_labes on 2012-06-16 16:22)

@ Helmut
Posting: # 8760
Views: 10,126
 

 probability nearly zero?

Dear Helmut!

» A1: A deficiency letter 20+ years ago. The randomization (by chance) was:
» RT TR RT TR RT TR RT TR RT TR RT TR (12 runs)

Wow! What is the probability to obtain this by chance?

BTW: My randomisation approach was always to randomize by blocks (block size 4 or if not possible by 2). Never obtained such an 'extreme' randomlist.
Random freaks may use varying block sizes chosen at random.

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 17:08

@ d_labes
Posting: # 8761
Views: 10,165
 

 probability nearly zero!

Dear Detlew!

» » The randomization (by chance) was:
» » RT TR RT TR RT TR RT TR RT TR RT TR (12 runs)
»
» Wow! What is the probability to obtain this by chance?

~1:500 (including the reverse order); try it yourself:
set.seed(16062012)
size       <- 12
sims       <- 1e7
y          <- rep(c(-1,1), size/2)
z          <- rep(c(1,-1), size/2)
alternate  <- 0
for (i in 1:sims) {
  x <- 1
  while (sum(x) != 0) x <- sign(rnorm(size))
  if (all(x == y) | all(x == z)) alternate <- alternate+1
}
cat(paste("‘alternating’ runs:", format(100*alternate/sims, digits=15), "%\n"))


Nine nine nine nine nine nine” wouldn’t pass the algo, since not balanced. :-D

» BTW: My randomisation approach was always to randomize by blocks (block size 4 or if not possible by 2). Never obtained such an 'extreme' randomlist.

Chance for block size 4 ~0.9% and for block size 2 ~3%. ;-)

» Random freaks may use varying block sizes chosen at random.

Too stupid to code that…

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
d_labes
★★★

Berlin, Germany,
2012-07-06 09:54

@ Helmut
Posting: # 8899
Views: 9,632
 

 R-package randomizeBE

Dear All!

» » Random freaks may use varying block sizes chosen at random.
»
» Too stupid to code that…

We proudly present on CRAN now :cool::

A little (futile) randomizer for BE studies
R-package randomizeBE


Features:
  • Also free (think 'free' not like 'free beer' :party: but 'free in speech')
    like this one of ElMaestro's alter ego and open source
  • not restricted to 2x2 crossover
  • block randomization
  • randomly chosen block sizes from a list of block sizes
  • control of randomness via Wald-Wolfowitz runs test (may be switched off)

Here some simulation results concerning probability of obtaining 'alternating' random lists in 2x2 crossover (randomness control switched off):
1 000 000 simulations, alpha=0.05 for the runs test

                     runs test
subjects blocksize  %significant  %alternating
----------------------------------------------
12         0 (12)       2.62          0.206
           6            3.54          0.515
           4            5.57          0.928
           2 4          9.47          1.576    randomly chosen block sizes
----------------------------------------------
24         0 (24)       5.95         <0.0001
          12            6.28          0.0001
           4           19.33          0.0039
           4 6         14.90          0.0023   randomly chosen block sizes
----------------------------------------------

Lessons learned:
  • If sample size not too low and block size also the probability to 'randomize' an alternating list is vanishing small.
  • The randomness control via runs test with significance level alpha=0.05 is much to stringent (much more lists were considered non random than our alternates).
  • The question to control/not control the randomness of the random list should not concern us much.
    IMHO we are throwing a spanner in the works of chance if we do it too serious.
BTW: Helmut how did you act on the deficiency letter 20+ years ago? Could you convince them that dicing 6 times in series 6 pips (or six times 1 pips) although has low probability is possible?


Edit: Package linked. [Helmut]

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-07-06 12:51

@ d_labes
Posting: # 8901
Views: 9,493
 

 R-package randomizeBE

Dear Detlew!

» R-package randomizeBE

THX, wonderful!

» BTW: Helmut how did you act on the deficiency letter 20+ years ago?

I implemented the WW-test and generated a new randomization if p<0.05.

» Could you convince them that dicing 6 times in series 6 pips (or six times 1 pips) although has low probability is possible?

No way. At that time CROs / clinical sites were randomly [sic] chosen by the Austrian Ministry of Health for GCP (facility) inspections. My rep("RT", "TR", 6) in one study was a ‘minor finding’ – in the next inspection they were very happy about my changed code. I wonder how the MoH’s random generator worked. ;-)

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Belgium?,
2012-06-16 13:31

@ Helmut
Posting: # 8754
Views: 10,210
 

 randomization: validation

Hi HS,

I don't think I would use a runs test prior to releasing a randomisation scheme. This screws the principle of randomness; even random sequences will have a significant runs test occasionally and still be 'random' whatever that words means.

ElMaestro's approach:
  1. Populate an array will RT and TR in equal numbers. E.g. index 1-12 are RT and 13-24 are TR if we have 24 subjects.
  2. Loop through all entries 1-24 and swap each exactly once with a random index between 1 and 24. The random index must come from a uniform distribution. A NLYW dancing around a hat with 24 coloured and numbered balls may be used; but be advised that NLYW's may display unpredictable behaviour and variations in time and space as regards willingness to dance around hats and draw balls.
  3. Release scheme without testing for randomness.

I could be wrong, but...
Best regards,
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 14:42

@ ElMaestro
Posting: # 8756
Views: 10,179
 

 randomization: runs test

Hi ElMaestro!

» I don't think I would use a runs test prior to releasing a randomisation scheme. This screws the principle of randomness; even random sequences will have a significant runs test occasionally and still be 'random' whatever that words means.

OK, but see my A1 above.

» ElMaestro's approach: […]

Nice; especially your remarks about NLYWs. Concerning your #3:
require(tseries)
set.seed(16062012)
size      <- 24
sims      <- 1e6
nonrandom <- 0
for (i in 1:sims) {
  x <- 1
  while (sum(x) != 0) x <- sign(rnorm(size))
  x[x ==  1] <- "RT"
  x[x == -1] <- "TR"
  x   <- as.factor(x)
  rt  <- runs.test(x)
  sig <- rt$p.value
  if (sig < 0.05) nonrandom <- nonrandom + 1
}
cat(paste("‘nonrandom’ runs: ", format(100*nonrandom/sims, digits = 4), "%\n"))


Gave 5.895% on your matey’s machine. Don’t know whether 20+ years after the deficiency letter mentioned above regulators would still react similar.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Belgium?,
2012-06-16 14:48

@ Helmut
Posting: # 8757
Views: 10,189
 

 randomization: runs test

Hi HS,

a quickie:

Scheme222=function(NSeq1, NSeq2, Seed, Sillyfactor)
{
   set.seed(Seed)
   A=c(rep("RT", NSeq1), rep("TR",NSeq2))

   for (j in 1:Sillyfactor)
   {
   swaps = sample(1:length(A), length(A), replace=T)
   for (i in 1:length(A))
   {
     tmp = A[i]
     A[i]=A[swaps[i]]
     A[swaps[i]]=tmp
   } ## i loop
   } ## j loop
   print(A)
}

Scheme222(12,12, 87654321, 123)


I have no idea why regulators strictly would not consider an alternating sequence random; I think they must have assumed the sequence had not been through a number-cruncher or NLYW?

I could be wrong, but...
Best regards,
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-16 15:10

@ ElMaestro
Posting: # 8758
Views: 10,181
 

 randomization: runs test

Ahoy!

» a quickie:

Nice.

» I have no idea why regulators strictly would not consider an alternating sequence random; I think they must have assumed the sequence had not been through a number-cruncher or NLYW?

Referring to my code in Pascal maybe they thought about this guy; neither a number-cruncher* nor a W at all.



Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Belgium?,
2012-06-16 19:12

@ Helmut
Posting: # 8762
Views: 10,095
 

 very cool coding style

Hi HS,

» while (sum(x) != 0) x <- sign(rnorm(size))

a little off topic, but it took me some time to get my head around this code line of yours. Now that I (think I) get it, I must say this is an incredibly elegant way of achieveing a lot with minimum coding. It completely permutes equal numbers of TR and RT (or -1's and 1's). Very, very elegant style!!! Me likey! :cool::cool::cool:

I could be wrong, but...
Best regards,
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-17 13:22

@ ElMaestro
Posting: # 8763
Views: 10,071
 

 Better a good steal than a bad reinvention

Hi ElMaestro!

Thanks for the flowers, but I’m (still…) a lousy R-coder. I boiled it down to a one-liner because the commented version was already given there. The “sign(rnorm(foo))” comes from an example in package tseries.
For a single run your quickie is fine, but for simulations I would opt for mine (~140 times faster). ;-)

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Belgium?,
2012-06-17 19:58

@ Helmut
Posting: # 8764
Views: 10,079
 

 Better a good steal than a bad reinvention

Hi HS,

» For a single run your quickie is fine, but for simulations I would opt for mine (~140 times faster). ;-)

You are reading me wrongly. I would not set the silly factor to 123, but to 1, which is adequate to ensure complete randomisation.

I could be wrong, but...
Best regards,
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 13:26

@ ElMaestro
Posting: # 8770
Views: 9,993
 

 Sorry!

Hi ElMaestro!

» You are reading me wrongly. I would not set the silly factor to 123, but to 1, which is adequate to ensure complete randomisation.

Sorry, didn’t realize that ‘silly’ has to be taken literally. Why did you include this loop at all? To slow down the algo?

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Belgium?,
2012-06-18 15:19
(edited by ElMaestro on 2012-06-18 15:30)

@ Helmut
Posting: # 8772
Views: 10,021
 

 Sorry!

Hi HS,

» Sorry, didn’t realize that ‘silly’ has to be taken literally. Why did you include this loop at all? To slow down the algo?

I had this discussion before with a client. The thing is, after a single pass through all the subjects, the sequence designation is as random as it wil be after 123 passes etc.
But I think this is difficult to somehow grasp out there. Key to realising it is that for each swap, there is exactly 50% chance that the subject will remain his/her firstly designated sequence, and 50% chance that it will be the other. After the full pass through all the subjects they are therefore fully randomised. But due to my lack of communication skills I failed to get that message across properly. So, it ended with a completely meaningless discussion like:
EM: "Would you like me to randomise the sequence XYZ times?"
Client: "Yes, please. They need to be fully randomised."
...and then I just smiled to my client and banged my head into the wall a couple times and took a handful of Schützomycin. It helped a little. They let me out of the loony bin after just 5 months.

I could be wrong, but...
Best regards,
ElMaestro
d_labes
★★★

Berlin, Germany,
2012-06-18 15:38

@ ElMaestro
Posting: # 8774
Views: 10,365
 

 Dancing with the dice

Hi Mon Captain, dear Helmut,

» » while (sum(x) != 0) x <- sign(rnorm(size))
»
» ... It completely permutes equal numbers of TR and RT (or -1's and 1's).

Not really. It rolls the 'binary dice' 24 times again and again until the vector of +1's and -1's has the same number of both entries (is 'balanced').

Thus while being programmed along an "incredibly elegant way of achieving a lot with minimum coding" not so very effective concerning computational time if one considers what is intended (have seen more than 20 loop passes in a short experimentation).

Regards,

Detlew
d_labes
★★★

Berlin, Germany,
2012-06-18 16:30

@ d_labes
Posting: # 8775
Views: 9,981
 

 Silly's randomization 2x2 xover

Dear both!

what about this one-liner (ok, ok its a 3-liner :cool:):
set.seed(123987)
nsubj <- 24
(c(rep("RT", trunc(nsubj/2)), rep("TR", nsubj-trunc(nsubj/2)))[sample(1:nsubj)])

Regards,

Detlew
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 17:14

@ d_labes
Posting: # 8777
Views: 9,962
 

 Silly's randomization 2x2 xover

Dear Simulants!

» what about this one-liner…

Hey, ~20% faster than mine; great code. :-D Nice method to get balance. The time limiting factor seems to be the resampling.

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 17:42

@ d_labes
Posting: # 8778
Views: 9,974
 

 Clumsy code

Dear Simulants!

» […] 'binary dice'…

If you flatten a die – isn’t that what we call a coin in Austria? :-D

» … 24 times again and again until the vector of +1's and -1's has the same number of both entries (is 'balanced').

Exactly.

» Thus while being programmed along an "incredibly elegant way of achieving a lot with minimum coding" not so very effective concerning computational time if one considers what is intended (have seen more than 20 loop passes in a short experimentation).

Sorry, couldn’t do better. Calls:
Min.  1st Qu.  Median  Mean  3rd Qu. Max.
 1      2        4     6.208   8      79


[image]

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
ElMaestro
★★★

Belgium?,
2012-06-18 18:01

@ Helmut
Posting: # 8779
Views: 9,968
 

 Clumsy code

Gentlemen,

how bout
sample(c(rep("AB",12), rep("BA",12)))

I could be wrong, but...
Best regards,
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2012-06-18 18:56

@ ElMaestro
Posting: # 8780
Views: 9,964
 

 Speed kills

Simulants!

» how bout
» sample(c(rep("AB",12), rep("BA",12)))

Aha! Almost a dead race with Detlew’s code; I’m only in the amateur’s league. ‘Nonrandom’ runs 5~6%.
  1. Your honorable’s one-liner: 115.68”
  2. Detlew’s: 116.18”
  3. Your 1st attempt, silly loop removed: 127.71”
  4. Mine: 148.72”

Cheers,
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. ☼
Science Quotes
jag009
★★★

NJ,
2012-09-17 21:46

@ Helmut
Posting: # 9231
Views: 8,601
 

 Speed kills

Hi all,

If one wants to plan a randomization scheme for a 4 way study, how does one go about assigning the treatments order in each of the 4 sequence (yes, use 4)? A 4 treatment study will have 24 possible sequence so the selection of which 4 sequences to use should be randomly selected, correct? So I should have a SAS routine to select the sequences first and then another to generate the study randomization scheme?

Thanks
John
Activity
 Thread view
Bioequivalence and Bioavailability Forum |  Admin contact
19,871 posts in 4,212 threads, 1,364 registered users;
online 22 (1 registered, 21 guests [including 7 identified bots]).
Forum time (Europe/Vienna): 13:44 CEST

Multivariate analysis: A means of finding the answer
when you don’t know the question.    Stephen Senn

The BIOEQUIVALENCE / BIOAVAILABILITY FORUM is hosted by
BEBAC Ing. Helmut Schütz
HTML5