I want to find the rank of a very sparse, quite rectangular matrix mat, but I'm running out of RAM (I have 16 GB) if I try to use MatrixRank or RowReduce. The particular matrix I am considering is 44216 by 5958 and its Tally is
Tally@Flatten@Normal@mat
{{1, 89348}, {0, 263290898}, {-1, 58682}}
I have also tried to produce the Gram matrix
gram = Transpose[mat].mat
since it has the same rank, but this matrix is obviously much less sparse. I also run out of RAM trying to do MatrixRank[gram].
Also, should one take different approaches if the rank can be assumed to be small, or on the other hand, large (almost full rank)?
SingularValueList[]? – J. M.'s missing motivation Jul 07 '15 at 14:09PseudoInverse,NullSpace, andMatrixRankare based onSingularValueDecomposition." Doesn't that mean that I effectively have? :p I'll test it shortly. – Marius Ladegård Meyer Jul 07 '15 at 14:14MemoryConstrained[si = SingularValueList[gram];, 10000000000]is running now (usingmatdidn't work as expected), and it doesn't seem to eat memory as quickly asMatrixRank. I'll post again when it completes/aborts. – Marius Ladegård Meyer Jul 07 '15 at 14:27SingularValueList[N[mat]]. If need be, specify the second argument. – J. M.'s missing motivation Jul 07 '15 at 14:31SingularValueListworks with dense matrices (it prints a message saying this) andmatis roughly 8 times as large asgramI wasn't so surprised... But maybe I should have been? – Marius Ladegård Meyer Jul 07 '15 at 15:02SingularValueListonly returns nonzero SVs, do you mean to increment the second argument until no new SVs are found? – Marius Ladegård Meyer Jul 08 '15 at 05:07Tolerancesetting. – J. M.'s missing motivation Jul 08 '15 at 05:44