0

I iterate through a process, each time generating a new number between 1 and a very large number (10^15 or larger). I am waiting for any number to repeat.

Clearly a 1D Boolean array (starting out all as false) is too large (for any memory that I have access to). However, I am hoping that a compressed version of the array might do the trick, if enough consecutive subsequences are generated. Please note that the array will likely not remain sparse.

This hinges on the ability to update a value (to true), without uncompressing.

Any ideas?

I thought of combinations of Compress, SparseArray, PackedArray, but nothing comes to mind.

Thanx for the thinx.

Aharon Naiman
  • 1,173
  • 6
  • 13
  • I would suggest to just try a simple SparseArray, see e.g. this question – Lukas Lang Nov 19 '22 at 19:33
  • What about using a HashTable? – Victor K. Nov 19 '22 at 20:38
  • Unless I'm missing something, I'm very skeptical of this approach. Even if you use a bit array, you would need UnitConvert[Quantity[10^15, "Bits"], "Terabytes"] = 125 TB of memory to store it. – Victor K. Nov 19 '22 at 20:45
  • Thank you for your answers. However, as I stated, since the array may likely not stay sparse, I wanted compression, which can take advantage of: from element m1 to n1, they are all false, from m2 to n2, they are all true, etc. – Aharon Naiman Nov 20 '22 at 13:20

0 Answers0