Bioequivalence and Bioavailability Forum

Lucas ★ Brazil, 2014-02-19 15:27 (4082 d 02:20 ago) Posting: # 12447 Views: 27,042	Qualitative analysis of the ANOVA residuals [General Statistics] Post reply
	Hello everybody. We've been asked to perform a qualitative analysis of the residuals of the ANOVA. Now, I don't know what that analysis is supposed to acomplish, so I don't know what is supposed to be done. Do you guys have any experience with that? Accept my apologies in advance if that is too much of beginner's question. Thank you.

Helmut
★★★

Vienna, Austria,
2014-02-19 16:22
(4082 d 01:25 ago)

@ Lucas
Posting: # 12450
Views: 25,285

Qualitative analysis of the ANOVA residuals

Post reply

Hi Lucas,

❝ We've been asked to perform a qualitative analysis of the residuals of the ANOVA.

ANVISA?

❝ Now, I don't know what that analysis is supposed to acomplish, so I don't know what is supposed to be done.

IMHO, assessing the studentized residuals makes sense. See also Canada’s Guidance, Section 2.3.4.

From the model’s raw residuals calculate the standardized residuals (aka internally studentized residuals).
From them calculate the studentized residual (aka deleted studentized residual, extrenally studentized residual).

Both are directly accessible in SAS and R – and with a little bit of struggle – in Phoenix/WinNonlin. I would suggest to come up with a box-plot (optionally overlayed with jittered values) and ±3 as an optical reference. Looks like this:

[image]

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Lucas
★

Brazil,
2014-02-19 16:58
(4082 d 00:49 ago)

@ Helmut
Posting: # 12453
Views: 25,019

Qualitative analysis of the ANOVA residuals

Post reply

Hi Helmut! Thanks for the fast response, as always.

❝ ANVISA?

No, the sponsor wants us to perform that. The study would be submitted at ANVISA, but I think they want a study that can be (if they ever need) submitted at other agencys. I just love how they ask us that kind of analysis without even explaining their objective with that. So I thought that you guys had experience with that being required in other countries. The sponsor is from England, if that helps.

❝ IMHO, assessing the studentized residuals makes sense. See also Canada’s Guidance, Section 2.3.4.

Doesn't seem to be a criteria for saying if the results are good or not, am I right?

❝ [...] and with a little bit of struggle – in Phoenix/WinNonlin.

Can u tell me how it can be done at Phoenix?

Thanks.

Helmut
★★★

Vienna, Austria,
2014-02-19 17:25
(4082 d 00:21 ago)

@ Lucas
Posting: # 12456
Views: 25,024

Qualitative analysis of the ANOVA residuals

Post reply

Hi Lucas,

❝ […] So I thought that you guys had experience with that being required in other countries.

Limited experience. Once I had to deal with studies on a HVD (pre-scaling age) where the sponsor – based on previous experiences – stated in the protocols that based on assessing the residuals (boxplots and Shapiro-Wilk test) in case of non-normality the study will be evaluated by a nonparametric method. The Dutch agency didn’t like that. In a hearing in The Hague the told me that “We in The Netherlands don’t believe in nonparametric statistics”. ;-)

❝ Doesn't seem to be a criteria for saying if the results are good or not, am I right?

In Canada you may exclude outliers if you have stated criteria in the protocol.

❝ ❝ [...] and with a little bit of struggle – in Phoenix/WinNonlin.

❝ Can u tell me how it can be done at Phoenix?

I’m currently struggling with some PopPK/PD-models which send my CPUs’ load to 80%. I don’t want to start concurrently another PHX-session because it would exceed my RAM and Phoenix would start swapping memory to disk. If I don’t come up with something in the next two weeks, remind me. Some hints in my whitepaper (Section 3.6) about EMA’s ABEL.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Lucas
★

Brazil,
2014-02-19 18:20
(4081 d 23:27 ago)

@ Helmut
Posting: # 12457
Views: 24,894

Qualitative analysis of the ANOVA residuals

Post reply

❝ I’m currently struggling with some PopPK/PD-models which send my CPUs’ load to 80%. I don’t want to start concurrently another PHX-session because it would exceed my RAM and Phoenix would start swapping memory to disk. If I don’t come up with something in the next two weeks, remind me. Some hints in my whitepaper (Section 3.6) about EMA’s ABEL.

Thanks for the hints. If I can not perform that by reading the whitepaper I'll come back to you later.

Best regards.

Lucas
★

Brazil,
2014-02-20 20:07
(4080 d 21:40 ago)

@ Helmut
Posting: # 12464
Views: 24,723

Qualitative analysis of the ANOVA residuals

Post reply

Hi Helmut and ElMaestro!

So, I got an answer from the sponsor about what would that qualitative analysis be. Both your's and ElMaestro's answers were right! This is how it would look like according to them:

[image]

Now my problem is how I would do that in Phoenix. I haven't got the time to read your whitepaper but I will soon, I only want to show you guys how this analysis would look like. Maybe it helps someone of the forum in the future.

Thank u.

Helmut
★★★

Vienna, Austria,
2014-02-20 21:11
(4080 d 20:36 ago)

@ Lucas
Posting: # 12465
Views: 25,802

First hints for Phoenix

Post reply

Hi Lucas,

❝ This is how it would look like according to them: […]

Interesting! I didn’t count, but ~200 subjects is not common in BE studies. It will not look that nice in the real world. The x-axis of the first plot is strange (negative means – I beg your pardon?). The distribution doesn’t look normal but skewed to the left* and the Q-Q plot looks extremely ugly. That’s actually a great example which EDA would suggest to use another transformation (e.g., ℯ^x or xⁿ, where n>1).
However, let’s see what you can get in PHX (I numbered the panels in your plot):

Plotting > XY-Plot (though for the abscissa I would either use the subject’s ID or the predicted model values).
Plotting > Histogram ☑ Show relative Frequency. In the current version (6.3) it is not possible to overlay a normal distribution, maybe in the next release?
Plotting > QQ Plot.
Upper part = trivial. Lower part = strange. In PHX the line in the Q-Q plot is fitted automatically – I don’t think it is possible to extract the parameters and AIC, BIC.
BTW, these values are only useful in comparing different models. I cannot imagine what might its purpose here. AICC is the AIC corrected for small sample sizes. It proves that the statistician is a “push-the-button-type” who does not know what he is doing. Of course you could fit the standard quantiles vs. stud. resid’s by PHX’ linear model.

If the sponsor insists on a similar layout, you have to leave PHX because different types of plots arranged in an array (aka trelis/lattice plots) are not supported. Either export relevant data from PHX and continue the analysis in R or generate the plots in PHX and paste them into a 4-cell table in your preferred text-processor.

AUC and C_max have been shown to follow a log-normal distribution – which is skewed to the right. However, in order to apply parametric methods the residuals (which are estimates of the residual error) must be normal – not the original values. The log-transformation is more a convention than justified. I have never seen values skewed to the left. Which kind of stuff is this?

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Helmut
★★★

Vienna, Austria,
2014-02-21 20:25
(4079 d 21:22 ago)

@ Lucas
Posting: # 12472
Views: 25,362

Phoenix: residuals

Post reply

Hi Lucas,

in the following a setup for PHX6.3+ (Clayton/Leslie’s data given in Chow/Liu’s table 6.9.1; validated against table 8.2.3); sequences coded as TR|RT. It seems complicated first, but works also for unbalanced sequences. This is not PHX’ standard model, but all effects fixed (like for EMA and in SAS Proc GLM). You have to code it only once and can use it as a template in other studies.

Setup the BE wizard for ln-transformed data.
Fixed effects: subject+period+treatment+sequence+subject(sequence),
delete Random1: subject(sequence)
Execute and navigate to Output Data > Residuals > Send To > Data > Data Wizard > Add Filter
- Exclude everything except Dependent, Subject, Predicted, Residual (i.e., keep them mapped to None)
- Built In > Add > Action: Include | Find: RT | Search Area Column: Sequence > OK
- Built In > Add > Action: Include | Find: 1 | Search Area Column: Period > OK
- Execute and rename to RT Filter
Final Results > Result > Send To > Data > Split Worksheet
- Map Sequence to Sort
- ☐ Create unique values worksheets (uncheck the box!)
- Execute and rename to RT Count
Navigate back to Bioequivalence > Output Data > Final Variance Parameters > Send To > Data > Join Worksheets
- Map Dependent to Sort, Estimate to Source Column, leave Parameter mapped to None
- Map Worksheet 2 > RT Filter.Result
- Map Dependent to Sort and all variables to Source Column
- Execute and rename to RT MS intra
Final Results > Result > Send To > Data > Join Worksheets
- Map Sequence to Sort and all other variables to Source Column
- Map Worksheet 2 > RT Count.Result
- Map Sequence to Sort and Count to Source Column
- Execute and rename to RT data
Final Results > Result > Send To > Data > Data Wizard > Add Transformation
- Transformation Type: Custom | Transformation: Custom Function | New Column Name: StudResIntra |
  Formula: Residual/sqrt(Estimate*(Count-1)/(2*Count))
- Add Filter > Exclude Residual, Estimate, Count
- Execute and rename to RT Stud Res intra
Repeat steps #2 – #6 for sequence TR. You should end up with TR Filter, TR Count, TR MS intra, TR data, TR Stud Res intra.
Navigate back to RT Stud Res intra > Final Results > Result > Send To > Data > Append Worksheets
- Map all variables to Source Column
- Map Worksheet 2 > TR Stud Res intra.Result
- Map all variables to Source Column
- Execute and rename to Stud Res intra

Using the standard plotting functions (see above), we get:

[image]

In Phoenix Pre-Release V1.4 (Build 6.4.0.511, 2013-12-20) still no overlays in box plots and histograms. :lookaround:

Maybe I will post some R-code later.

Eidt: I have to correct myself: In Phoenix (all versions) these are standardized (sometimes call internally studentized) residuals, not studentized (externally studentized) residuals.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

BRN ☆ Turkey, 2014-03-04 16:15 (4069 d 01:32 ago) @ Helmut Posting: # 12554 Views: 24,489	Phoenix: residuals Post reply
	Dear Helmut, I'm trying to run this process on WNL 6.3 but I'm getting confused in step 3 where I need to map sequence but there's no sequence since I exclude it in the previous step (2) because it's mentioned: ❝ `Exclude` everything except `Dependent`, `Subject`, `Predicted`, `Residual` (i.e., keep them mapped to `None`). Could you please kindly suggest? — Regards, BRN

Helmut
★★★

Vienna, Austria,
2014-03-04 17:52
(4068 d 23:55 ago)

@ BRN
Posting: # 12555
Views: 24,637

Phoenix: residuals

Post reply

Hi BRN,

❝ […] I'm getting confused in step 3 where I need to map sequence but there's no sequence since I exclude it in the previous step (2) because it's mentioned:

❝ ❝ Exclude everything except Dependent, Subject, Predicted, Residual (i.e., keep them mapped to None).

In Step 2 you have to keep Sequence mapped to None. I made a stupid error – sorry for the confusion caused.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Rosy ☆ Mexico, 2018-10-15 09:03 (2383 d 09:44 ago) @ Helmut Posting: # 19442 Views: 16,920	Phoenix: residuals Post reply
	Dear Helmut: I do studentized residual test with R for the outliers detection, but now my regulation prohibid to use R. Can i do with phoenix 7 or another statistical software like ncss. than u Edit: Full quote removed. Please delete everything from the text of the original poster which is not necessary in understanding your answer; see also this post #5! [Helmut]

Helmut
★★★

Vienna, Austria,
2018-10-15 13:43
(2383 d 05:04 ago)

@ Rosy
Posting: # 19443
Views: 16,811

Software validation

Post reply

Hi Rosy,

❝ I do studentized residual test with R for the outliers detection, but now my regulation prohibid to use R.

Bizarre. Consider presenting the document

R: Regulatory Compliance and Validation Issues
A Guidance Document for the Use of R in Regulated Clinical
Trial Environments

and to ask the agency why it believes (!) that commercial software (SAS, Phoenix, …) is “better” than open source. Any software has to be validated* – which is easy for base [image]

❝ Can i do with phoenix 7 or another statistical software like ncss.

In Phoenix you get only standardized residuals (aka internally studentized residuals). For the setup see above. In SAS both internally and externally studentized residuals are possible. Haven’t used NCSS for 15+ years – no idea.

* Examples where a commercial software screwed up:

Schütz H, Labes D, Fuglsang A. Reference Datasets for 2-Treatment, 2-Sequence, 2-Period Bioequivalence Studies. AAPS J. 2014;16(6):1292–7. doi:10.1208/s12248-014-9661-0. PMC Free Full text.
Fuglsang A, Schütz H, Labes D. Reference Datasets for Bioequivalence Trials in a Two-Group Parallel Design. AAPS J. 2015;17(2):400–4. doi:10.1208/s12248-014-9704-6. PMID 25488055. PMC Free Full text.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Rosy ☆ Mexico, 2018-10-18 02:38 (2380 d 16:08 ago) @ Helmut Posting: # 19462 Views: 16,602	Software validation Post reply
	Dear Helmut I know, what's wrong with my regulation, anyway... that's because I'm Mexican (if you remember Guillermo del Toro speach) LOL I will try to follow you directions on phoenix, but someone tell me that it's valid to make a linear regression between Ln_observed and predicted from "Residuals" output from Phoenix and obtain the Student residuals to find outliers, is that right?

Helmut
★★★

Vienna, Austria,
2018-10-18 19:14
(2379 d 23:33 ago)

@ Rosy
Posting: # 19470
Views: 16,528

Avoid raw residuals

Post reply

Hi Rosy,

❝ I know, what's wrong with my regulation, anyway... that's because I'm Mexican (if you remember Guillermo del Toro speach) LOL

This part?

❝ I will try to follow you directions on phoenix, …

Good luck.

❝ … but someone tell me that it's valid to make a linear regression between Ln_observed and predicted from "Residuals" output from Phoenix

What? That’s not interesting because it is rather difficult to “see” anything useful.*
You could plot the residuals (y) vs. Ln_predicted (x). Cave: In a 2×2 crossover for every subject you have two residuals with opposite signs. You need only one of them (commonly the first period is used). However, raw residuals are not very useful. Though you get an idea about trends (if they increase/decrease or show a strange shape the model is wrong) or homo-/heteroscedasticity (funnel shape) their magnitude is not telling. That’s why you should standardize/studentize them.

❝ … and obtain the Student residuals to find outliers, is that right?

Yep. Though I would not call them outliers (regulators don’t like it) but “aberrant values”. ;-)

That’s why in comparing two bioanalytical methods nobody plots method 1 / method 2 but uses the Bland-Altman plot.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Rosy
☆

Mexico,
2018-10-18 20:05
(2379 d 22:42 ago)

@ Helmut
Posting: # 19471
Views: 16,887

Avoid raw residuals

Post reply

Dear Helmut,

This video, i mean this became a meme for Mexicans, although everything it's black (wrong) you will see the light... Mexican Humor LOL

❝ Good luck.

Thank u, but i have trouble, i finish with 4 residuals for each patients (from final variance parameters) i dont know how analyze, because in R appears a table with intra and inter subject residuals for each subject, if i wanna make the same table which values i have to pick up.

❝ ...That’s why you should standardize/studentize them.

yep, i mean, i took "Ln_Observed" and "predicted" from Phoenix, then in ncss i run lineal regression and from that i do a regression diagnosis to obtain studentized residual, Do i have to do this with every sequence in every period (whole data) or just for the 1st period, of course the objetive it's to find aberrant values.... i know maybe this no make any sense but i have to comply my regulation

thank u for your patience, i'm new in bioequivalence but i hope to learn so much

Edit: youtube link is corrected. [Mittyri]

Helmut
★★★

Vienna, Austria,
2014-02-23 03:20
(4078 d 14:27 ago)

@ Lucas
Posting: # 12473
Views: 24,943

Phoenix → R

Post reply

Hi Lucas,

export the stuff to [image]

and some quick/dirty code:
In the Phoenix-project navigate to Stud Res intra > Output Data > Result right-click > Export

Filename: StudResIntra

Filetype: Text (Tab delimied) (*.txt)

Save

The default location is the desktop (which conveniently is also the default import site in R). Start R and paste this code into the console:

res <- read.table("StudResIntra.txt", header=TRUE, sep="\t")

attach(res)

ResLim <- c(-max(abs(StudResIntra))*1.1, max(abs(StudResIntra))*1.1)

PreLim <- c(min(abs(Predicted))*1.1, max(abs(Predicted))*1.1)

SubLim <- c(min(abs(Subject))*1.1, max(abs(Subject))*1.1)

TR <- subset(res, Sequence == "TR")

RT <- subset(res, Sequence == "RT")

RL <- function() {

  abline(h=c(0, -2, 2, -3, 3), lty=c(1, rep(2, 4)),

    col=c(rep("black", 2), rep("red", 2)))

}

lab <- "Studentized residual"

split.screen(c(3, 2))

screen(1)

plot(0, 0, type="n", xlim=PreLim, ylim=ResLim, xlab="Predicted", ylab=lab,

  main="Scatterplot 1", cex.main=0.9, las=1)

RL()

points(TR$Predicted, TR$StudResIntra, pch=23, col="red", bg="#ffdddd")

points(RT$Predicted, RT$StudResIntra, pch=22, col="darkgreen", bg="#ddffdd")

screen(2)

plot(0, 0, type="n", xlim=SubLim, ylim=ResLim, xlab="Subject", ylab=lab,

  main="Scatterplot 2", cex.main=0.9, las=1)

RL()

points(TR$Subject, TR$StudResIntra, pch=23, col="red", bg="#ffdddd")

points(RT$Subject, RT$StudResIntra, pch=22, col="darkgreen", bg="#ddffdd")

screen(3)

HisLim <- c(0, max(hist(StudResIntra, breaks="FD", plot=FALSE)$density)*1.1)

hist(StudResIntra, breaks="FD", freq=FALSE, xlim=ResLim, ylim=HisLim, xlab=lab,

  ylab="Frequency", main="Histogram", cex.main=0.9, las=1, border="blue",

  col="#ddddff")

curve(dnorm(x, mean=mean(StudResIntra), sd=sd(StudResIntra)), add=TRUE)

box()

screen(4)

qqnorm(StudResIntra, xlim=ResLim, ylim=ResLim, xlab="Theoretical quantiles",

  ylab=lab, main="Normal Q-Q plot", cex.main=0.9, pch=21, col="blue", bg="#ddddff",

  las=1, plot.it=TRUE)

qqline(StudResIntra, datax=F, distribution=qnorm)

screen(5)

boxplot(StudResIntra, xlim=c(0.5, 2.5), ylim=ResLim, ylab=lab, main="Boxplot",

  cex.main=0.9, las=1, at=1, border="blue", col="#ddddff")

RL()

points(x=jitter(rep(2, length(StudResIntra)), factor=4), StudResIntra,

  pch=21, col="blue", bg="#ddddff")

close.screen(all = TRUE)

detach(res)

Good luck! If you have a Connect license you even don’t have to leave PHX.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Lucas ★ Brazil, 2014-02-25 20:28 (4075 d 21:18 ago) @ Helmut Posting: # 12497 Views: 24,868	Phoenix → R Post reply
	Wow! Thanks for the help Helmut! I got tied up with other projects and only saw your responses now. Sorry about that. Thank god I got an e-mail from the sponsor a few hours ago saying that we no longer have to perform that analysis. But these hints will be very helpful for us. Thank you very much for your helpfulness! Best regards.

ElMaestro
★★★

Denmark,
2014-02-19 16:32
(4082 d 01:15 ago)

@ Lucas
Posting: # 12451
Views: 25,079

Qualitative analysis of the ANOVA residuals

Post reply

Hi,

it really depends on what your agency wants with that qualitative analysis of residuals.

I would do a histogram of residuals, possibly a histogram of within-subject differences.
Ideally that would amount to something that looks like a normal distribution, but realistically with a low number of subjects it may just look like the teeth of someone who pissed Mike Tyson seriously off. Looking for outliers (looking in this regards means looking, and not quantifying their potential presence) may be another aspect where this could come in handy.

—
Pass or fail!
ElMaestro

Helmut
★★★

Vienna, Austria,
2014-02-19 17:06
(4082 d 00:41 ago)

@ ElMaestro
Posting: # 12454
Views: 25,075

Qualitative analysis of the ANOVA residuals

Post reply

Hi ElMaestro,

❝ Ideally that would amount to something that looks like a normal distribution, but realistically with a low number of subjects it may just look like the teeth of someone who pissed Mike Tyson seriously off.

With the data from above:

[image]

48 subjects. Agree with “tooth gaps” in small studies. Below ~30 you will see funny patterns.

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна!
Helmut Schütz

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Lucas
★

Brazil,
2014-02-19 17:13
(4082 d 00:34 ago)

(edited on 2014-02-19 20:22)
@ ElMaestro
Posting: # 12455
Views: 25,116

Qualitative analysis of the ANOVA residuals

Post reply

Hello!

❝ [...] it may just look like the teeth of someone who pissed Mike Tyson seriously off.

That is an extraordinary way to describe it! :rotfl:

❝ Looking for outliers (looking in this regards means looking, and not quantifying their potential presence) may be another aspect where this could come in handy.

I don't know if I'm thinking too "statistically", but that wouldn't be very subjective? One can look and think that there are outliers and other can think that there aren't and both would not be wrong, since there is no criteria.
Anyway, your's is a good way to attend to that requirement from the sponsor, but IMHO a qualitative analysis for the residuals would not add any value to the study, from a regulatory perspective. Am I wrong?

ElMaestro
★★★

Denmark,
2014-02-19 21:40
(4081 d 20:07 ago)

@ Lucas
Posting: # 12458
Views: 24,921

Qualitative analysis of the ANOVA residuals

Post reply

Hi Lucas,

❝ I don't know if I'm thinking too "statistically", but that wouldn't be very subjective? One can look and think that there are outliers and other can think that there aren't and both would not be wrong, since there is no criteria.

You're right, there is some degree of subjectivity in my proposal. I will argue that this is the case for many if not most qualitative assessments, like for instance a CoA for a injectable where you have a spec like "A colorless or slightly yellowish liquid".

❝ Anyway, your's is a good way to attend to that requirement from the sponsor, but IMHO a qualitative analysis for the residuals would not add any value to the study, from a regulatory perspective. Am I wrong?

You're right, at least per principle. Regulatory decisions are as far as I know not based on residuals or their distributions in any territory. However, regulators may ask for anything they like ("Please submit all chromatograms for QCs and calibrators injected during validation, maintenance records from 2009-2011 for freezer ZX/Y353, and the phone number of the hot brunette from I.P. who attended the last pre-submission meeting" etc), and they may ultimately use that additional information against you as they please.
No idea why a sponsor would ask you to do this, other than if some newly hired numbercruncher wants to impress her/his boss by asking something that sounds complicated. If Helmut's figure above applies to you data then just submit that, tell the sponsor you think this looks reasonably normal (as in Gaussian) without the presence of definitively aberrant values (you will deliberately not use the term outliers).

—
Pass or fail!
ElMaestro

Qualitative analysis of the ANOVA residuals [General Sta­tis­tics]

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA resid­uals

Qualitative analysis of the ANOVA resid­u­als

First hints for Phoenix

Phoenix: resid­u­als

Phoenix: resid­u­als

Phoenix: resid­u­als

Phoenix: resid­u­als

Software validation

Software validation

Avoid raw residuals

Avoid raw residuals

Phoenix → R

Phoenix → R

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA resid­u­als

Qualitative analysis of the ANOVA residuals [General Statistics]

Qualitative analysis of the ANOVA residuals

Qualitative analysis of the ANOVA residuals

Phoenix: residuals

Phoenix: residuals

Phoenix: residuals

Phoenix: residuals

Qualitative analysis of the ANOVA residuals