sarada06884
☆    

India,
2013-04-09 14:22
(4005 d 20:59 ago)

Posting: # 10381
Views: 14,632
 

 Incurred sample reanalysis [Regulatives / Guidelines]

Hi Helmut,

Incurred sample reanalysis has been suggested by USFDA but I would like to debate on this topic.

How are ISR samples different from subject samples? The answer would be they are in vivo samples and the QCs are in vitro samples. ISR samples are meant for demonstrating assay reproducibility. Based on this now my concern would be:
  1. When the whole validation has been done based on in vitro samples and when the results of these experiments have been accepted then why ISR again?
  2. When Long Term Matrix Stability of the drug and metabolites are proven taking invitro samples is proven and when the stability of such experiment is accepted then why ISR again?
  3. Even though the ISR samples are reanalysed they are analysed along with CCs and QCs which are invitro again!!!!
  4. What if ISR fails?? probably they have to ascertain the cause and go for the development of the method again?????
Please give me your views

Regards
Srinivas
Ohlbe
★★★

France,
2013-04-09 16:09
(4005 d 19:12 ago)

@ sarada06884
Posting: # 10382
Views: 14,020
 

 Incurred sample reanalysis

Dear Srinivas,

❝ Hi Helmut,


Not interested in the opinion of other members of the forum ?

❝ Incurred sample reanalysis has been suggested by USFDA


It is now also required for EMA submission.

❝ 1. When the whole validation has been done based on in vitro samples and when the results of these experiments have been accepted then why ISR again?


Because in vitro samples are not fully representative of the subject samples. They usually do not contain metabolites, and I've never seen overfilled QC samples (two of the possible causes of ISR failure).

❝ 2. When Long Term Matrix Stability of the drug and metabolites are proven taking invitro samples is proven and when the stability of such experiment is accepted then why ISR again?


The long-term stability is not the only issue anyway: back-conversion during sample processing can be of greater concern. And are you sure you know all metabolites ? EMA got trapped in their recommendations on clopidogrel.1

❝ 3. Even though the ISR samples are reanalysed they are analysed along with CCs and QCs which are invitro again!!!!


So what ? The idea with ISR is to test for the variability of the method when analysing subject samples. Some variability factors affect the subject samples more than the CCs and QCs, which are used as a reference.

❝ 4. What if ISR fails?? probably they have to ascertain the cause and go for the development of the method again?????


First try and find the reason for failure, and see if and how it can be corrected.


  1. Luigi Silvestro, Mihaela Cristina Gheorghe, Isabela Tarcomnicu, Silviu Savu, Simona Rizea Savu, Adriana Iordachescu and Constanta Dulea
    Development and validation of an HPLC–MS/MS method to determine clopidogrel in human plasma. Use of incurred samples to test back-conversion
    Journal of Chromatography B, 878 (2010) 3134–3142
    doi:10.1016/j.jchromb.2010.09.022
For more info on ISR have a look e.g. at:

M. Yadav, P. S. Shrivastav
Incurred sample reanalysis (ISR): a decisive tool in bioanalytical research
Bioanalysis (2011) 3(9), 1007–1024

A. Tan, S. Gagnon-Carignan, Sylvain Lachance et al.
Beyond successful ISR: case-by-case investigations for unmatched reassay results when ISR passed
Bioanalysis (2011) 3(9), 1031–1038

Regards
Ohlbe
The Outlaw Torn
★    

Europe,
2013-04-10 10:40
(4005 d 00:42 ago)

@ Ohlbe
Posting: # 10387
Views: 13,404
 

 Incurred sample reanalysis

Dear Ohlbe,

❝ The long-term stability is not the only issue anyway: back-conversion during sample processing can be of greater concern. And are you sure you know all metabolites ? EMA got trapped in their recommendations on clopidogrel1


I just happened to be cruising by this joint looking for anything new on ISR. Glad to see you tapping this subject. You answered a lot of my questions, except I'm not sure I understand what you mean by the EMA getting trapped. Do you mean that there are ways around justifying clopidogrel, or other drugs known for back-conversion, that EMA hadn't considered (the paper you quoted where the authors found a way of controlling back-conversion)? Is that it? If that's not the case, mind expanding on this while I sit back and have a drink?

Thanks,
Outlaw
Ohlbe
★★★

France,
2013-04-10 12:34
(4004 d 22:47 ago)

@ The Outlaw Torn
Posting: # 10393
Views: 13,471
 

 Clopidogrel

Dear Outlaw,

❝ [...] I'm not sure I understand what you mean by the EMA getting trapped. Do you mean that there are ways around justifying clopidogrel, or other drugs known for back-conversion, that EMA hadn't considered


Oh no... If the drug is know for back-conversion, I don't really see a way out.

❝ (the paper you quoted where the authors found a way of controlling back-conversion)?


Which means there is a need to study back-conversion anyway...

❝ If that's not the case, mind expanding on this


What I mean to say is that it may not be sufficient to test known metabolites for back-conversion, or to just say that the drug has no known metabolite which could be back-converted. What EMA recommended for clopidogrel was to demonstrate the lack of back-conversion of clopidogrel carboxyacid, the main metabolite, to clopidogrel. The simplest way to do it would be to spike blank plasma with clopidogrel carboxyacid, process it, and check that no clopidogrel is generated. It made sense: one could imagine that clopidogrel carboxyacid could be re-esterified with methanol, and that this could explain the back-conversion issues with clopidogrel which some people were talking about.

Problem: the metabolite which is back-converted is not the carboxyacid, but the acylglucuronide* (not described in the literature when EMA published their recommendation, as far as I know), by a transesterification mechanism. Silvestro et al. clearly demonstrate in their paper that what was recommended by EMA just doesn't work. Only ISR allowed them to first identify the issue, then develop a method with no back-conversion issue.


  • In addition to a number of posters, see another paper by Silvestro et al.
    Silvestro L, Gheorghe M, Iordachescu A, Ciuca V, Tudoroniu A, Rizea Savu S, Tarcomnicu I
    Development and validation of an HPLC-MS/MS method to quantify clopidogrel acyl glucuronide, clopidogrel acid metabolite, and clopidogrel in plasma samples avoiding analyte back-conversion.
    Anal Bioanal Chem. 2011 Aug;401(3):1023-34
    doi: 10.1007/s00216-011-5147-4

Regards
Ohlbe
The Outlaw Torn
★    

Europe,
2013-04-10 13:31
(4004 d 21:51 ago)

@ Ohlbe
Posting: # 10399
Views: 13,231
 

 Clopidogrel

Very thorough and answers my query quite clearly, Ohlbe. Merci beaucoups.

On a side note, does anyone have actual experience with the authorities concerning lack of ISR justifications? Whether they require it for products that have already been marketed for years now that are going through ruMRPs, etc? The only feedback we are getting for all ISR queries is a reference back to the Q&A document. Seems like they are hoping someone gets referred to CMDh so that whatever internal disagreement there is on ISR gets resolved by CHMP.
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-04-22 19:01
(3992 d 16:20 ago)

@ The Outlaw Torn
Posting: # 10459
Views: 14,273
 

 Lack of ISR / only 5% ISR in study with 656 samples

Hi Outlaw & all!

❝ On a side note, does anyone have actual experience with the authorities concerning lack of ISR justifications?


Last Friday (today is day 100): Line extension of an MR formulation (racemate), two studies (single & multiple dose), stable-isotope IS chiral GC/MS method (BE based on the active enan­tio­mer d-█████).
  • SD: analytical part finished April 2010, 558 samples, no ISR
  • MD: finished June 2011, 656 samples, 5% ISR, d-enantiomer deviation -2.75% (±6.80 [-15.58%, +11.13%]) and l-enantiomer +1.19% (±7.61 [-14.04%, +12.89%]).
    100% of both enantiomers’ ISR within ±20%.
So far accepted by RMS Germany1 and CMSs Austria, Denmark, Sweden, Spain.
But The Netherlands:

PSRPH:
Bioequivalence is considered not demonstrated in the single-dose study due to the absence of incurred sample reanalysis (ISR). The Applicant should discuss the vali­di­ty of the bio­­analytical method in the absence of ISR, taking into account the dis­cus­sion points in the PKWP Q&A document (EMA/618604/2008 Rev. 7).
Rationale:
The Applicant did not provide justification for the absence of ISR. According to the RMS, the absence of ISR is acceptable as the PK data is robust, back conversion of meta­bolites is not a significant issue for █████ and the study has been conducted in 2010. This is not agreed. The current guideline of bioanalytical method [sic] (EMEA/CHMP/EWP/192217/2009)2 came into effect on February 1, 2012. All app­li­ca­tions submitted after this date should comply to the guideline, irrespective of the time of study. The Applicant should discuss, in the absence of ISR, the validity of the bioanalytical method concerning the precision and accuracy of the study samples during processing and storage taking into account the discussion points in the PKWP Q&A document (EMA/618604/2008 Rev. 6).
Additional points for clarification:
[Study #2] Only 5% of the samples were used for ISR. The Guideline on bio­analyti­cal method validation requires3 10% of the samples. The Applicant should therefore provide a justification that the number of reanalysed samples used sufficiently re­flects the accuracy and precision of the analyte in all the samples during processing and storage.



  1. From the RMS’s PAR: Regarding the standard pharmacokinetic variables bioequivalence has undisputably [sic] been shown. The results are rather robust and statistical analysis indicate no significant difference.
    Moreover the shape of the plasma profiles are similar and the statistical evaluation of the advanced pharmacokinetic variables which reflect the biphasic drug liberation of the dosage form (e.g. AUC0-4, AUC4-t, Cmax(0-4) and Cmax(4-t)) are […] well within con­ven­tio­nal acceptance criteria.
    The applicant performed no Incurred Sample Reanalysis. This [is] considered accept­able due to the robust PK data, the convincing results of pre- and in-study validation, the fact that the study has been conducted in 2010 and that back conversion of metab­olites is not a significant issue for █████.
  2. Dated 21 July 2011 and published by EMA on 1 August 2011. A f**ing time machine would have been required for writing the analytical protocol back in April 2011. No per­cent­age at all was suggested in the 2009 draft.
  3. Requires?? WTF? I only read The extent of testing depends on the analyte and the study samples, and should be based upon in-depth understanding of the analytical method and analyte. However, as a guide, 10% of the samples should be reanalysed in case the number of samples is less than 1000 samples and 5% of the number of samples exceeding 1000 samples.” :cool:
    The method is robust (used in ~20 studies in many thousands of samples; not a single batch ever repeated). A back-conversion of the main metabolite in sample pro­cessing is not possible (selective chiral derivatization step). The CVintra of the API (in­de­pen­dent from the formulation – various IRs and MRs) is very – very! – low.
    With such a mean & SD it is improbable to get even a single repeat with a deviation >20% (6σ :blahblah:) and almost impossible to get more than ⅓ repeats outside…

    Do I really have to come up with a simulation (~10% repeats) showing the probability of ISR outside the acceptance range (>20% deviation in ⅓ of samples) based on the observed x -2.75% and SD 6.80%?
    data   <- c(-1.05,  -6.38, -5.30,  -5.55,  -2.99,  -2.24, +10.75, +0.90,
                -6.66,  -5.01, -2.74, -10.98,  +4.43,  -3.53,  -8.85, -7.38,
                -8.35, +11.13, -4.07, -15.58,  +2.05, -14.86,  -3.34, +2.66,
                -6.80,  +5.60, -2.48,  +5.61, -11.71, +10.61,  -4.00, -1.73)
                                # Observed deviations
    mean   <- mean(data)/100    # Mean deviation
    sd     <- sd(data)/100      # SD of deviations
    size   <- 656               # No. of samples in a study
    ifelse(size < 1000,         # EMA: 10% if <1000; +5% for >1000
      ISR.no <- ceiling(size*0.1),
      ISR.no <- 100 + ceiling((size-1000)*0.05))
    sims   <- 1e6               # No. of simulated studies
    failed <- 0                 # ISR failed criterion (≤1/3 of repeats >20% dev.)
    for (i in 1:sims) {
      x      <- 100*rnorm(n=ISR.no, mean=mean, sd=sd)
      y      <- length(x[abs(x)>20]) # No. of abs. deviations > 20%
      y      <- y/ISR.no        # Fraction of repeats with > 20% deviation
      if (y > 1/3) failed <- failed + 1
    }
    cat(formatC(sims, format="d", big.mark=","), "simulated studies with",
      size, "samples:\n",
      "Method: Pseudorandom (Mersenne-Twister) samples from normal distribution\n",
      sprintf("%i", ISR.no), "ISR repeats based on the observed mean of",
      sprintf("%.2f%%", 100*mean), "with a standard deviation of",
      sprintf("%.2f%%%s%i%s", 100*sd, " (n=", length(data),").\n"),
      "Failed criterion (deviations >20% in >1/3 of repeats) seen in",
      sprintf("%.2f%% %s", 100*failed, "of studies.\n"))


    Which – as expected – tells me:
    1,000,000 simulated studies with 656 samples:
     Method: Pseudorandom (Mersenne-Twister) samples from normal distribution
     66 ISR repeats based on the observed mean of -2.75% with a standard deviation
     of 6.80% (n=32).
     Failed criterion (deviations >20% in >1/3 of repeats) seen in 0.00% of
     studies.


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2013-04-22 20:56
(3992 d 14:26 ago)

@ Helmut
Posting: # 10461
Views: 13,103
 

 Lack of ISR / only 5% ISR in study with 656 samples

Hi Hötzi,

sad story. I'd love to see the faces of the CMD(h) members if or when they realise that they have yet another ISR case to deal with. The details are good: the ISR seemed to work well in June, but NL's fear is it didn't in April?!?

I might be inclined to play the regulatory chicken game and go ahead full throttle. If any situation would merit a deviation from the guideline's requirement then this development would be a good candidate, wouldn't it.
Anyways to mitigate any risk, if this was a compound like meth (very -very!- low CV intra) then you'd be looking at 12-16 subjects in a 222-BE trial. Would be a cheapo and could be done and dusted by any CRO way before the end of a clockstop.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-04-22 21:35
(3992 d 13:46 ago)

@ ElMaestro
Posting: # 10462
Views: 13,234
 

 Lack of ISR / only 5% ISR in study with 656 samples

Ahoy, my Capt’n, ahoy!

❝ sad story. I'd love to see the faces of the CMD(h) members if or when they realise that they have yet another ISR case to deal with. The details are good: the ISR seemed to work well in June, but NL's fear is it didn't in April?!?


That’s not a fear, but PSRPH… ISR ‘worked’ nicely in 5% of samples (in June), but 10% are “required” (published in August). Ha!

❝ […] and go ahead full throttle.


Do I look like James Dean?

❝ If any situation would merit a deviation from the guideline's requirement then this development would be a good candidate, wouldn't it.


I’m in the right mood for it. :-D

❝ Anyways to mitigate any risk, if this was a compound like meth (very -very!- low CV intra) …


Good guess. Not exactly meth (that was the story at the end of this post), but close. Remember the recent Two-Stage thread?

❝ … then you'd be looking at 12-16 subjects in a 222-BE trial. Would be a cheapo and could be done and dusted by any CRO way before the end of a clockstop.


Please no. BTW, at a meeting at the BfArM in 2010 they were surprised that we have done the MD study at all: “We expect that such a study will not be necessary any more according to the new MR GL.” Accumulation in the MD study was <1%…

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
ElMaestro
★★★

Denmark,
2013-04-22 22:03
(3992 d 13:19 ago)

@ Helmut
Posting: # 10463
Views: 13,136
 

 "Meth" - apologies

Hi again,

❝ Good guess. Not exactly meth (that was the story at the end of this post), but close. Remember the recent Two-Stage thread?


Jawohl; I apologise for that abbreviation - I actually meant meth█████ which I have seen have quite low intra-CVs in IR formulations. In the world of inhalation I also occasionally see meth used as acronym for methacholine. It is used to provoke artifical asthma attacks in volunteers. Wonderful stuff - crossover studies now and then have a lot of subjects who mysteriously don't feel like showing up for period 2 :-D


Edit: Your guess was exactly right. However, I blacked it out (not sure whether the sponsor wants to see it in public right now). Low CVs also for all MR formulations – unless you go for the first pAUC (not to speak about EMA’s invention pCmax,1). :angry:

Pass or fail!
ElMaestro
The Outlaw Torn
★    

Europe,
2013-04-23 10:27
(3992 d 00:55 ago)

@ ElMaestro
Posting: # 10468
Views: 13,001
 

 "Meth" - apologies

Thanks for the example, Helmut. Nothing like a concensus! This is the kind of thing that makes you want to bang your head on your desk! NL have been rather difficult lately for us as well (and Spain). And some of the eastern block countries to, for some reason. Ummmm.

El Maestro, you said "I'd love to see the faces of the CMD(h) members if or when they realise that they have yet another ISR case to deal with." Has anyone challenged ISR to the CMDh yet, as far as you know (I mean, has the committee had to make any decisions regarding ISR or the lack of it)?
ElMaestro
★★★

Denmark,
2013-04-23 16:29
(3991 d 18:53 ago)

@ The Outlaw Torn
Posting: # 10472
Views: 12,963
 

 "Meth" - apologies

Hi Outlaw,

❝ El Maestro, you said "I'd love to see the faces of the CMD(h) members if or when they realise that they have yet another ISR case to deal with." Has anyone challenged ISR to the CMDh yet, as far as you know (I mean, has the committee had to make any decisions regarding ISR or the lack of it)?


It is difficult to challenge the CMD(h) directly; in a DCP they will be on the receiving end of a PSRtPH if a national agency declares one. The CMD(h) doesn't really 'decide' much here, and the members are often relying on their experts since the core business of CMD(h) is regulatory rather than strictly scientific (but please don't ask me to draw the line between what's regulatory and what's science :pirate:).
They can talk and they can invite experts for specific issues in a procedure and they can more generally send over issues to the PK work group as needed.
Many companies have submitted study reports without ISR data and many times this has caused trouble at the CMD(h), and this in a sense one reason why the PK work group's statement re. ISR was born.
In a case like Helmut's I believe the CMD(h) members would with very decent effort try to reach consensus for acceptance due to e.g.
"ISR data obtained for the same analyte from other studies carried out in the same laboratory and with the same analytical method may be used as supportive data to justify the lack of ISR."

I hope this answers your question.

Pass or fail!
ElMaestro
The Outlaw Torn
★    

Europe,
2013-04-24 09:44
(3991 d 01:38 ago)

@ ElMaestro
Posting: # 10482
Views: 13,057
 

 "ISR" - apologies

❝ I hope this answers your question.


Yes, that answers a lot of my concerns. What I was wondering, more precisely, is the same old question I have on ISR, and that is whether anyone has been referred to CMDh for not providing ISR in a ruMRP. I'd like to hear the committee's reasoning for denying an ruMRP based on lack of ISR, when an ruMRP is based solely on a previously accepted dossier and now back up by several years of the actual product being taken by patients, yada, yada, yada. We are being told to refer back to the Q&A document for our answer, which says nothing or tells me that they expect some kind of justification for lack of ISR or else the ruMRP will be denied, which as I've mentioned before makes no sense. It's so difficult to get go advice, because each country is unsure what another country's position on a given issue will be (see the NL in Helmut's case; everything seemed fine with the other CMSs and DE). Very frustrating.
ElMaestro
★★★

Denmark,
2013-04-24 13:08
(3990 d 22:14 ago)

@ The Outlaw Torn
Posting: # 10485
Views: 12,822
 

 Repeat use

Hi Outlaw,

I would certainly go ahead with the repeat use MRP dossier.
If there is any concern it has probably been caught by the pharmacovigilance system. Hard to believe a repeat use MRP can be rejected, really. After all it is the same product - by nature equally efficient and safe as the previous/first MRP.

Pass or fail!
ElMaestro
Ohlbe
★★★

France,
2013-04-24 13:41
(3990 d 21:41 ago)

@ ElMaestro
Posting: # 10486
Views: 12,783
 

 Repeat use

Hi El Maestro,

❝ If there is any concern it has probably been caught by the pharmacovigilance system.


I doubt it, unless it is really a big, big problem... The substitution of the reference product by the generic is made by the pharmacist selling the drug to the patient. The prescriber usually doesn't even know which generic was given to his patient. If, by some sort of a miracle, he decides to make a pharmaco­vigilance declaration, to which product will he assign the AE ?

Sorry for being pessimistic, but I don't have much trust in the pharmaco­vigilance of generics :-(

Regards
Ohlbe
ElMaestro
★★★

Denmark,
2013-04-24 15:45
(3990 d 19:36 ago)

@ Ohlbe
Posting: # 10488
Views: 12,815
 

 Repeat use

Hi Ohlbe,

❝ I doubt it (...) If, by some sort of a miracle, he decides to make a pharmacovigilance declaration, to which product will he assign the AE?


❝ Sorry for being pessimistic, but I don't have much trust in the pharmaco­vigilance of generics :-(


I agree in the sense that the PV systems -and here I do not mean any applicant's own system but the EU-wide and possibly ROW-wide systems- do not pick up issues with great sensitivity and they rely on active reporting reporting. Only the serious stuff gets reported.
For a repeat use MRP the applicant will be able to refer to something like 1.000.000 patient years of treatment experience and garnish that with info about reported numbers of deaths or people who contracted syphilis while being treated with the drug etc etc. I think that counts for something although it is not formalised.

In a nutshell I would rather take a product that has 1.000.000 or whatever patient years in the luggage and no ISR rather than taking a product with acceptable ISR for which the total human exposure is a 2,2,2-BE trial in Hevo's totalling 1 day of exposure with one unit dose for the Test.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-09-20 23:50
(3841 d 11:32 ago)

@ The Outlaw Torn
Posting: # 11537
Views: 11,929
 

 "ISR" – resolved

Hi all,

17 pages of justification – essentially following the Q&A (simulations, later studies with ISR, literature data, back-conversion neither under physiological nor sample treatment conditions chemically possible, :blahblah:).

Day 145 (NL)
Clinical aspects
Potential Serious Risks to Public Health
Pharmacokinetics

Study #1: The conclusion of the preliminary assessment report stands true as both prerequisites are met for waiving ISR: At the time point of the studies the Guideline on analytical validation was not in operation and back conversion is not an issue (re-esterfication is not an issue under physiological or sample preparation conditions; interconversion can be excluded, plasma concen­tra­tions are consistent with literature data). Moreover the applicant provided sup­portive data e.g. ISR results of studies #2 and #3 using the same ana­lyti­cal method and results of remeasurements confirming the robustness and re­li­ability of the analytical method.

Study #2: ISR has been performed and the results comply with the actual re­com­mendations, however formal requirements for the sample size were not met. This is considered acceptable as the same considerations as for study #1 are applicable. Moreover the results demonstrated very high reproducibility as 100% of ISR in study #3 were within the acceptance criteria for deviation.

Overall Summary and Conclusion
Issues resolved. :-D


Edit: PAR from HMA as of 2014-01-27. DE (RMS), AT, DK, ES, FI, LU, NL, NO, PL, SE, UK (CMS). Both studies were performed in a Two-Stage Design (Potvin’s “Method C”) – already passing in the first stage (no α-ad­justment = 90% CIs). The estimated Ĉmin was assessed in the MD study.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
Dr_Dan
★★  

Germany,
2013-09-22 18:56
(3839 d 16:26 ago)

@ Helmut
Posting: # 11540
Views: 11,543
 

 "ISR" – resolved

Dear Helmut
Congratulations!:clap:

Kind regards and have a nice day
Dr_Dan
ElMaestro
★★★

Denmark,
2013-04-24 01:31
(3991 d 09:51 ago)

@ Helmut
Posting: # 10480
Views: 12,847
 

 Lack of ISR / only 5% ISR in study with 656 samples

Cool code Helmut.

Run your code, then do shapiro.test(data) and discuss if the high p-value proves that the data are Gaussian.
Next do this:
data   <- c(-1.05,  -6.38, -5.30,  -5.55,  -2.99,  -2.24, +10.75, +0.90,
            -6.66,  -5.01, -2.74, -10.98,  +4.43,  -3.53,  -8.85, -7.38,
            -8.35, +11.13, -4.07, -15.58,  +2.05, -14.86,  -3.34, +2.66,
            -6.80,  +5.60, -2.48,  +5.61, -11.71, +10.61,  -4.00, -1.73)
                            # Observed deviations
size   <- 656               # No. of samples in a study
ifelse(size < 1000,         # EMA: 10% if <1000; +5% for >1000
  ISR.no <- ceiling(size*0.1),
  ISR.no <- 100 + ceiling((size-1000)*0.05))
sims   <- 1e6               # No. of simulated studies
failed <- 0                 # ISR failed criterion (≤1/3 of repeats >20% dev.)
for (i in 1:sims) {
  xboot = sample(data, replace=T)
  y      <- length(xboot[abs(xboot)>20]) # No. of abs. deviations > 20%
  y      <- y/ISR.no        # Fraction of repeats with > 20% deviation
  if (y > 1/3) failed <- failed + 1
}
cat(sprintf("%i",sims),"simulated studies with", size, "samples:\n",
  "Method: Bootstrapping (Mersenne-Twister) of ISR data\n",
  "Failed criterion (deviations >20% in >1/3 of repeats) seen in",
  sprintf("%.2f%% %s", 100*failed, "of studies.\n"))


...and argue that even when you do not make any assumption of ISR distribution you are still very clearly very far from any sort of trouble.

Pass or fail!
ElMaestro
Helmut
★★★
avatar
Homepage
Vienna, Austria,
2013-04-24 04:08
(3991 d 07:14 ago)

@ ElMaestro
Posting: # 10481
Views: 12,980
 

 Some more stuff

Hi ElMaestro,

THX for your help over there. Now it looks even more nice (using the repeats’ % deviation):
data   <- c(-0.53, -3.19, -2.65, -2.77, -1.49, -1.12, +5.38, +0.45,
            -3.33, -2.51, -1.37, -5.49, +2.22, -1.77, -4.42, -3.69,
            -4.18, +5.57, -2.03, -7.79, +1.02, -7.43, -1.67, +1.33,
            -3.40, +2.80, -1.24, +2.80, -5.85, +5.30, -2.00, -0.87)
mean   <- mean(data)/100    # Mean deviation
sd     <- sd(data)/100      # SD of deviations


❝ Run your code, then do shapiro.test(data) and discuss if the high p-value proves that the data are Gaussian.


Yessir!

        Shapiro-Wilk normality test

data:  data
W = 0.9624, p-value = 0.3184


I was already amazed yesterday by:
xfit <- seq(-max(abs(data))*1.25, max(abs(data))*1.25, length=50)
yfit <- dnorm(xfit, mean=mean*100, sd=sd*100)
hist(data, prob=TRUE, col="grey", main="ISR: deviation from means",
  xlim=c(min(xfit), max(xfit)), ylim=c(0, max(yfit)*1.25),
  las=1, xlab="% deviation")
lines(xfit, yfit, lwd=2, col="blue")


Haven’t thought about a formal test for normality!

Of course I can politely ask for the probability of getting a deviation of -20% or lower:
pnorm(-20, mean=mean*100, sd=sd*100)
[1] 2.115518e-08

Didn’t I fantasize about 6σ already?

What about a wacky setup in the spirit of Anderson & Hauck to get the probability of a result outside of [-20%, +20%]?

❝ Next do this: […]



I would rather bootstrap for 10% of the study’s samples based on the 5% I have …

  xboot   <- sample(data, size=ISR.no, replace=T)

… since I want to justify that based on the x/SD from 32 samples I will pass with the Dutch 66 as well.

❝ ...and argue that even when you do not make any assumption of ISR distribution you are still very clearly very far from any sort of trouble.


Yep.

Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes
nobody
nothing

2014-10-23 12:06
(3443 d 23:15 ago)

@ ElMaestro
Posting: # 13775
Views: 9,395
 

 Lack of ISR / only 5% ISR in study with 656 samples

❝ Next do this:

for (i in 1:sims) {

  xboot = sample(data, replace=T)

  y      <- length(xboot[abs(xboot)>20]) # No. of abs. deviations > 20%

  y      <- y/ISR.no        # Fraction of repeats with > 20% deviation

  if (y > 1/3) failed <- failed + 1

}

cat(sprintf("%i",sims),"simulated studies with", size, "samples:\n",

  "Method: Bootstrapping (Mersenne-Twister) of ISR data\n",

  "Failed criterion (deviations >20% in >1/3 of repeats) seen in",

  sprintf("%.2f%% %s", 100*failed, "of studies.\n"))


Wanted to perform some kind of sensitivity analysis to see at which criterion studies start to fail. With 1/8 of samples deviating more than 20% I get 200% study failure, due to simply multiplying number of failed studies by 100 in the output, but should be divided by number of studies simulated, I guess. The code above works only as long as 0 trials fail, i guess ;-)

Οὐδείς


Edit: Full quote removed. Please delete everything from the text of the original poster which is not necessary in understanding your answer; see also this post. [Helmut]
Of course, you are correct. The line should be:
  sprintf("%.2f%% %s", 100*failed/sims, "of studies.\n"))

Kindest regards, nobody
UA Flag
Activity
 Admin contact
22,957 posts in 4,819 threads, 1,640 registered users;
71 visitors (0 registered, 71 guests [including 7 identified bots]).
Forum time: 10:22 CET (Europe/Vienna)

Nothing shows a lack of mathematical education more
than an overly precise calculation.    Carl Friedrich Gauß

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5