Optimize certain list over permutations, perhaps using recursions?

Question

I am trying to improve my code for computing products of monomial symmetric functions. It boils down to the following. Let lam and mu be two non-negative integer vectors of equal length, entries weakly increasing. I want to compute

Tally@
    Table[
        Sort[DeleteCases[lam + muPerm, 0], Greater]
    , {muPerm, Permutations@mu}]

as quickly as possible. Note that if the length of the vector is large, then the number of permutations is quite huge. However, the number of entries in the final result is much smaller (due to the sorting).

One can assume that lam is of the form (a1,...,ak,0,...,0) and mu is of the form (b1,...,bn,0,...,0) where the vector lengths is $k+n$, and all ai, and bi are greater than 0.

Can this code be improved, so that it is quicker (or perhaps utilizes memoization), and moreover, does not use as much memory?

Answer 1

Unfortunately, no, the code probably can't be improved much. Mathematica can be fast, but only when most of the work is spent inside its large, perfectly optimized built-in functions (such as Tally and Permutations in your case). Generally, the smaller the building blocks you use, the slower the program. Even Sort[list, Greater] is slower than plain Sort[list], because its fast internal comparisons are replaced with slow external evaluations of a user function.

That said, it's actually possible to make the code several times faster at the cost of even more memory consumption:

{Reverse[DeleteCases[#[[1]], 0]], #[[2]]} & /@ 
 Tally@Table[Sort[lam + muPerm], {muPerm, Permutations@mu}]

I also tried to optimize for memory with my own compiled permutation generator and Association instead of Tally to avoid storing everything at once. I failed miserably. While using only as much memory as needed, this version turned out to be ten times slower.

My advice to you is to code it in C or in Julia, which is as fast as C, as easy to use as Python, and has a Mathematica interface.

EDIT: no upvotes? OK, posting my slow yet memory effective code, based on Combinatorica's NextPermutation function, modified to support repeated entries and stop after the last permutation. Still about 20 times faster than the upvoted solution.

nextPermutation = Compile[{{l, _Integer, 1}},
   Module[{nl = l, n = Length[l], i, j},
    i = n - 1;
    While[i > 0 && nl[[i]] >= nl[[i + 1]], i--];
    If[i == 0, Return[{-1}]];
    j = n;
    While[nl[[j]] <= nl[[i]], j--];
    {nl[[i]], nl[[j]]} = {nl[[j]], nl[[i]]};
    Join[Take[nl, i], Reverse[Drop[nl, i]]]
    ]
   ];
perm = Sort[mu];
tally = <||>;
While[Min[perm] >= 0,
  item = Reverse@DeleteCases[Sort[lam + perm], 0];
  tally[item] = Lookup[tally, Key[item], 0] + 1;
  perm = nextPermutation[perm]
  ];

Answer 2

You don't need to generate all your permutations in one go. I found this nice solution from user 2012rcampion to generate permutations as you go and I think you could fit it into your code somehow:

permutation[l_List, n_Integer] := 
 Module[{i = n, remaining = l, tally = Tally[l], m = 1, len, 
   perm = {}}, 
  If[Multinomial @@ Last /@ tally >= n > 0, 
   While[Length[remaining] > 0, tally[[m, 2]]--;
    len = Multinomial @@ Last /@ tally;
    If[len >= i, AppendTo[perm, tally[[m, 1]]];
     remaining = DeleteCases[remaining, tally[[m, 1]], {1}, 1];
     tally = Tally[remaining];
     m = 1, i -= len;
     tally[[m, 2]]++;
     m++;]];
   perm, {}]]
(* the tally is in the association *)
assoc = <||>;
i = 1;
While[(muPerm = permutation[mu, i]) != {},
 item = Sort[DeleteCases[lam + muPerm, 0], Greater];
 assoc[item] = If[KeyExistsQ[assoc, item], assoc[item] + 1, 1];
 ++i;
]

Runtime performance might be a lot worse, but memory usage will be much lower I expect.

There's also this ResourceFunction here, though it does not skip the duplicate permutations of your data, and remembering these would occupy some more memory.