Negative determinant [🇷 for BE/BA]

posted by ElMaestro  – Denmark, 2020-07-14 05:22 (1555 d 01:57 ago) – Posting: # 21691
Views: 19,226
(edited on 2020-07-14 12:43)

Hi Hötzi,

❝ In Obj.F12() the line

❝ B <- -0.5*log(det(t(X) %*% solve(CovM) %*% X))

❝ is the reason for the warnings because t(X) is not a square matrix. Therefore B is NaN.


I looked at this one by testing is.nan(B) and printing the components if/when it is. It happens right at the start for this data.

The problem is not that X changes dimensions, X is the design matrix and this is generally never a square matrix, just full column rank with as many rows as observations. X does not change at all during the optimisation.

All is in fact hunky dory when it happens, the variance estimates being:
[1] 0.06841615 0.03576077 0.01240137 0.05893007 (on my machine)
- that's nothing special, just a wee change from initial.

The issue here is that the determinant suddenly becomes apparently negative. This is mysterious to me at this point, I am not much into matrices. I take it as a case of NM taking some unfortunate decision, which is then corrected afterwards.

However this does give food for thought: if the parameter vector that gives a negative determinant is chosen as a starting value then NM will stop immediately and throw an error - you can try it with the vector above.
Therefore, it may be desirable to ascertain positiveness of that determinant (and the others?) when starting the algo, and to slightly modify individual starter values in case of such suboptimally chosen starter values until Obj.F12 actually returns something meaningful.

I believe the issue can not be remedied by fiddling with X.

I will be happy to hear a hardcore matrix algebrist's opinion here.

Update 8 hours later:
I think CovM becomes noninvertible at select levels of a variance component, given the others, when/if a certain value of the variance component means that one or more columns (rows) add up to one or more other columns. At exactly that point CovM will drop in its rank, and it becomes non-invertible; beyond the value that creates this phenomenon CovM is invertiable but the determinant may switch sign. It is a phenomenon with a singularity in the determinant easily explained in the numbers, but I cannot interpret the higher perspective. I am grateful that NM in the implementation of optim apparently safeguards against this phenomenon as long as the first call to the objective is evaluable.:-) Whodathunkit?
—
Pass or fail!
ElMaestro

Complete thread:

UA Flag
Activity
 Admin contact
23,258 posts in 4,886 threads, 1,661 registered users;
51 visitors (0 registered, 51 guests [including 8 identified bots]).
Forum time: 07:19 CEST (Europe/Vienna)

I’m not a pessimist,
I’m just a well informed optimist.    José Saramago

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5