Rank 3 Tensor Multiplication Memory issue

Question

I'm doing a complicated tensor multiplication experiment. Imaging there are two Rank 3 tensors generated as below:

ten0 = Table[g, {T+1}];
ten1 = Table[g, {2 T + 1}];

where g and dimension T are:

g := RandomComplex[10 + 5 I, {120, 120}];
T = 50;

I want to oragnize two tensors to a new {T+1,2T+1} tensor as:

tensorT = Table[ten0[[i]]*ten1[[j]], {i, T+1}, {j, 2 T + 1}];

And then multiply it to a matrix whose dimension is also {T+1,2T+1}:

mat = RandomComplex[10 + 5 I, {T+1, 2 T + 1}];

and calculate the sum:

Total[mat*tensorT, 2];

I got memory crash when I try to increase the dimension T to higher values. And the time efficiency is compared low.

 MemoryInUse[]/2^30.
 (*1.18938 GB*)

I was trying to research the issue and assuming MMA might be doing a "copied" passing thing in memory. see post here. So large matrix multiplications will generate this issue.

So I have tried to use Developer'ToPackedArray@ to those tensors but there is no difference.

And I managed to solve the memory issue by using DOUBLE Do loop instead:

mat0 = Table[0. + I 0., {i, 120}, {j, 120}];
Do[Do[mat0 = 
mat0 + (mat[[n + 1]][[m + T + 1]])*ten0[[n + 1]]*
 ten1[[m + T + 1]], {m, -T, T}], {n, 0, T}]
mat0

But as you might think, it's even slower which is not acceptable to me.

I have been stuck here for days. Could you please offer some advice?

This code worked for me. Mma eats about 5 Gb of memory, though WolframKernel.exe takes only about 1.3 Gb. — Andrew, Sep 08 '18 at 18:04
yes, for smaller T it will work. But I hope to increase it to 200 or more than that. — cj9435042, Sep 08 '18 at 18:07
Why not at once tensorT = Table[ten0[[i]]*ten1[[j]] RandomComplex[10 + 5 I], {i, T + 1}, {j, 2 T + 1}];? — Andrew, Sep 08 '18 at 18:08

score 1 · Accepted Answer · answered Sep 08 '18 at 18:48

The important idea is to avoid tensorT being built explicitly.

Buliding the tensors in a way that ansure that they are packed arrays (avoiding Table):

n = 120;
T = 50;
ten0 = RandomComplex[10 + 5 I, {T + 1, n, n}];
ten1 = RandomComplex[10 + 5 I, {2 T + 1, n, n}];
mat = RandomComplex[10 + 5 I, {T + 1, 2 T + 1}];

The actual computation:

AbsoluteTiming[MaxMemoryUsed[
  tensorT = Table[ten0[[i]]*ten1[[j]], {i, T + 1}, {j, 2 T + 1}];
  a = Total[mat*tensorT, 2];
  ]]
AbsoluteTiming[MaxMemoryUsed[
  b = Total[ten0 mat.ten1];
  ]]
Max[Abs[a - b]]

{2.90361, 2373581208}

{0.016172, 23501216}

7.00913*10^-10

Yes, this works, Thanks a lot! – cj9435042 Sep 08 '18 at 18:59 — cj9435042, Sep 08 '18 at 18:59

Rank 3 Tensor Multiplication Memory issue

1 Answers1