Isn't it possible to Pack a SparseArray?

Question

I have the following matrix Va:

Va = Table[
Which[i == j + 1, 2, True, 0], {l, 1, 2}, {m, 1, 2}, {s, 1,
 2}, {q, 1, 2}, {i, 1, 2}, {j, 1,2}];

There is no problem to pack it:

Va = Developer`ToPackedArray[Va];

Print@Developer`PackedArrayQ@Va;

True

But then there is the matrix Vb, which is the same as Va, but in the SparseArray form:

Vb = SparseArray[{l_, m_, s_, q_, i_, j_} :> 
Which[i == j + 1, 2, True, 0], {2, 2, 2, 2, 2, 2}];

And this one I can not Pack:

Vb = Developer`ToPackedArray[Vb];

Print@Developer`PackedArrayQ@Vb;

False

Does anybody know the way to pack this SparseArray Vb?

Have a look at Vb["Properties"] It's stuff like the "NonzeroValues" you want to be packed. (Can't really provide more detail, but I've found SparseArrays to be faster when the properties are packed arrays) — ssch, Dec 15 '13 at 17:13
To be a packed array, one first has to have an array (not a SparseArray). Try Developer`PackedArrayQ @ Normal@Vb. — Michael E2, Dec 15 '13 at 17:16
@ Michael E2 indeed, if you convert the SparseArray into an array, you can Pack it, but the reason why I use SparseArray is because I have memory issues, therefore I assume I will have memory problems when I convert my SparseArray into an array. — Mencia, Dec 15 '13 at 17:23
But it seems to be the only option since, apparently there is no way to Pack an SparseArray. — Mencia, Dec 15 '13 at 17:26
I believe @ssch is right. The relevant parts of the SparseArray are packed, if possible. There is some computational overhead with a SparseArray, but generally they are efficient. I'm not sure, but I think you have to be careful that some operations do not convert a sparse array to a full array. — Michael E2, Dec 15 '13 at 17:50
I agree with Michael. I think for packed we should read: "storing machine integers as opposed to larger objects (structs from C)". I know that a packed array is nothing but a thin data structure (struct) around an array of machine integers (in this case). That can be found out by using LibraryLink. I am not certain that a "normal mathematica integer" is a struct, maybe Michael's linksnooper skills can help there. Anyway, I would be very surprised if SparseArray called on a list of integers would produce a data structure in which structs were stored rather than machine integers. — Jacob Akkerboom, Dec 15 '13 at 19:15

score 11 · Accepted Answer · edited May 23 '17 at 12:35

This is a bit long for a comment, but I think it is useful information in the context of this question. No, SparseArrays cannot be packed, because a packed array and a sparse array are completely different and unrelated data structures. But, a sparse array can be constructed from packed arrays. Let's look at an example:

Vb = SparseArray[
 {l_, m_, s_, q_, i_, j_} :> Which[i == j + 1, 2, True, 0],
 {2, 2, 2, 2, 2, 2}
];

This is the array from the question. Now we consider its internal structure:

Vb /. Verbatim[SparseArray][args__] :> {args}
     /. _List?Developer`PackedArrayQ -> "packed"

(Here we are just stripping off the SparseArray head to produce a list of arguments, and then substituting "packed" in place of any packed arrays that appeared there.)

Some of its parts are packed, but not all:

(* -> {Automatic, "packed", 0,
       {1, {"packed", "packed"}, {2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2}}
      } *)

We can try to pack all the parts by converting it into an ordinary array, packing it, and then converting back into a sparse array. (I can't see any way to pack it from the start if we choose to define it using a conditional statement.)

Vb2 = SparseArray@Developer`ToPackedArray@Normal[Vb];
Vb2 /. Verbatim[SparseArray][args__] :> {args}
      /. _List?Developer`PackedArrayQ -> "packed"
(* -> {Automatic, "packed", 0, {1, {"packed", "packed"}, "packed"}} *)

Here we see that the list of non-zero values was packed by this process, although nothing else changed. It suggests that we can avoid having to unpack it fully, and just pack the relevant list by ourselves:

Vb3 = Vb /. Verbatim[SparseArray][args__] :> 
             SparseArray @@ MapAt[Developer`ToPackedArray, {args}, {4, 3}];
Vb3 /. Verbatim[SparseArray][args__] :> {args}
      /. _List?Developer`PackedArrayQ -> "packed"
(* -> {Automatic, "packed", 0, {1, {"packed", "packed"}, "packed"}} *)

Regarding the meaning of the parts of the SparseArray structure, I would suggest reviewing Oliver Ruebenkoenig's answer to this question, and Leonid's API for manipulating them as presented here and here.

Isn't it possible to Pack a SparseArray?

1 Answers1

Linked