Negative determinant [🇷 for BE/BA]

posted by ElMaestro  – Denmark, 2020-07-14 05:22 (1368 d 09:13 ago) – Posting: # 21691
Views: 16,934

(edited by ElMaestro on 2020-07-14 12:43)

Hi Hötzi,

❝ In Obj.F12() the line

B <- -0.5*log(det(t(X) %*% solve(CovM) %*% X))

❝ is the reason for the warnings because t(X) is not a square matrix. Therefore B is NaN.

I looked at this one by testing is.nan(B) and printing the components if/when it is. It happens right at the start for this data.

The problem is not that X changes dimensions, X is the design matrix and this is generally never a square matrix, just full column rank with as many rows as observations. X does not change at all during the optimisation.

All is in fact hunky dory when it happens, the variance estimates being:
[1] 0.06841615 0.03576077 0.01240137 0.05893007 (on my machine)
- that's nothing special, just a wee change from initial.

The issue here is that the determinant suddenly becomes apparently negative. This is mysterious to me at this point, I am not much into matrices. I take it as a case of NM taking some unfortunate decision, which is then corrected afterwards.

However this does give food for thought: if the parameter vector that gives a negative determinant is chosen as a starting value then NM will stop immediately and throw an error - you can try it with the vector above.
Therefore, it may be desirable to ascertain positiveness of that determinant (and the others?) when starting the algo, and to slightly modify individual starter values in case of such suboptimally chosen starter values until Obj.F12 actually returns something meaningful.

I believe the issue can not be remedied by fiddling with X.

I will be happy to hear a hardcore matrix algebrist's opinion here.

Update 8 hours later:
I think CovM becomes noninvertible at select levels of a variance component, given the others, when/if a certain value of the variance component means that one or more columns (rows) add up to one or more other columns. At exactly that point CovM will drop in its rank, and it becomes non-invertible; beyond the value that creates this phenomenon CovM is invertiable but the determinant may switch sign. It is a phenomenon with a singularity in the determinant easily explained in the numbers, but I cannot interpret the higher perspective. I am grateful that NM in the implementation of optim apparently safeguards against this phenomenon as long as the first call to the objective is evaluable.:-) Whodathunkit?

Pass or fail!

Complete thread:

UA Flag
 Admin contact
22,980 posts in 4,821 threads, 1,659 registered users;
31 visitors (0 registered, 31 guests [including 10 identified bots]).
Forum time: 14:35 CEST (Europe/Vienna)

Never, never, never use any software!
Never, never, never trust any computer!    Detlew Labes

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz