11

The answer to this question will most probably be a definite no. Nevertheless, I would like to know whether it is possible to represent reals in the memory on less than the standard 32 or 64 bits (depending on machine). This limits my exploration to $\sim 2^{14} \times2^{14}$ matrices, though with half or quarter precision I could venture a bit further. The matrix itself cannot be broken down to smaller pieces for further compuation, and it cannot be reworked to a simpler one, but high precision is of no importance. The question is not about refactoring matrices but about number representation and memory handling by Mathematica.

Warning, the following computation may eat through your memory limits.

n = 14;
all = (2^n)*(2^n);
m = RandomReal[{0, 1}, {2^n, 2^n}];
{all, (ByteCount@m/1024./1024) "MB", ((ByteCount@m/all*8.) // N) "bit"}
ClearAll[m];

{268435456, 1024. "MB", 64. "bit"}

I am using 64-bit Win7, with 2 GB RAM, and (expectedly) my system almost chokes on this code. And then of course I hadn't calculated anything with the matrix yet...

István Zachar
  • 47,032
  • 20
  • 143
  • 291
  • 2
    High precision may be of more importance than it appears, since for large matrices, at least in some cases, one may get large accumulated errors in the computations - this of course depends on the matrix and on the operations you want to do with it. – Leonid Shifrin Apr 04 '13 at 09:47
  • @Leonid Yes, I am aware of that, but still I wonder if it is possible to represent my data on less then the standard amount of memory. Or should I switch to C (or similar) if I want half precision... – István Zachar Apr 04 '13 at 09:49
  • You could give SetPrecision a try. But for a single float, this increases the memory usage from 24byte to 72byte. – Stefan Apr 04 '13 at 09:51
  • @Stefan Any change to SetPrecision practicly applies a tenfold increase in bytes... I should have said "lower" instead of "change" at the first place :) – István Zachar Apr 04 '13 at 10:07
  • @andre It seems to me that Integers are also represented on 64 bits on my machine. – István Zachar Apr 04 '13 at 10:50
  • I'm not convinced that a switch to C will help, unless it is an implementation that supports e.g. 16-bit floats. Do these even exist in IEEE-compliant C? I don't know. – Daniel Lichtblau Apr 04 '13 at 13:35
  • 2
    SetPrecision[...,something small] will cause machine numbers, which might live in a packed array, to become low precision, unpackable, bignums. Big loss in terms of memory due to the unpacking. – Daniel Lichtblau Apr 04 '13 at 13:36
  • What are you doing with these matrices? Is it possible that the operations might be broken down, using disk memory to store the full matrix and bringing in pieces at a time? Do a web search for "Tall and skinny QR factorization" to get an idea of what might be possible. – Daniel Lichtblau Apr 04 '13 at 13:38
  • @Daniel Thanks for the input! My question was intentionally left general to focus on number representations instead of other tricks of handling huge data. I think looking for solutions of "how to treat enormous matrices in memory?" would require a different question, at least I don't dare to modify my Q in this direction. In a nutshell however, my problem involves the calculation of the eigenvector corresponding to the largest eigenvalue. But again, this is just one particular problem, and I think this Q now has an answer (even if it is negative). – István Zachar Apr 04 '13 at 13:56
  • 2
    Try the following using out-of-core memory. (1) Power method to find the eigenvalue. Involves matrix.vector iterations, and that can be broken into pieces. (2) Divide-and-conquer null space on matrix-eibval*identity. See this post. Weirdly enough, after never referencing I have now sent it to two people today. Also (3) remember what @Leonid Shifrin said: if you go to a low precision representation you will be asking for numerical troubles. – Daniel Lichtblau Apr 04 '13 at 14:51
  • According to the docs, "Supported types are Integer, Real, and Complex. " so I'd go for "no". – acl Apr 04 '13 at 17:28

1 Answers1

4

(Based on the comments)

Matrices are best stored by Mathematica in packed arrayes. Packed arrays are limited to certain types. According to the docs‌​: "Supported types are Integer, Real, and Complex. "

Any change to SetPrecision practically applies a tenfold increase in bytes, because SetPrecision[..., small] will cause machine numbers, which might live in a packed array, to become low precision, unpackable, bignums. Big loss in terms of memory due to the unpacking.

Just for the record, IEEE 754-2008 does allow 16-bit floats (Half-precision floating-point format) so it is possible to lower bitwidth in e.g. C, however do take into account the possibility that with lower precision, accumulated error might be much higher in some computations.

István Zachar
  • 47,032
  • 20
  • 143
  • 291
  • It is a real shame, that mathematica does not allow that. The standard seems to be 64bit floats by now and that is way more than you need in most cases and hence costs a lot of unnecessary memory. – Wizard Jun 06 '15 at 22:37