7

I'm trying to efficiently generate tuples of lists of objects that satisfy a given criterion. The similar questions that I have found on this website end up finding a specific workaround for the given problem.

An inefficient way of doing what I want is

Select[Tuples[lists] , criterion]

which is inefficient because MMA first needs to generate and store all the tuples. I'm thinking of using Outer, but I'm not sure how. I've tried something that in my head should work but in reality it does not:

Outer[If[#1 === a || #2 === 2, {##}] &, {a, b, c}, Range[2]]

(* {{{a, 1}, {a, 2}}, {Null, {b, 2}}, {Null, {c, 2}}} *)

What it wants to do is to leave the tuple in place if it satisfies the criterion or else remove it: I would like the output to be (in this case)

(* {{{a, 1}, {a, 2}}, {{b, 2}}, {{c, 2}}} *)

How do I do it?

EDIT: Thanks to the comments I have put to together this solution (which gives the same output of Select[Tuples[{list1,list2,...}], criterion@@##]):

f = Flatten[Outer[If[criterion@Flatten[{##}], Flatten[{##}], Nothing]&, ##, 1], 1] &

Fold[f, {list1, list2,list3,...}]

Which also allows one to apply the criterion on all the elements of a tuple.

Ziofil
  • 2,470
  • 16
  • 30
  • 1
    It may be much more efficient to do this recursively, and perhaps use a compiled function to do the actual selection of allowed tuples. But I cannot say for sure unless you share the actual list and condition you are interested in. – Marius Ladegård Meyer Feb 03 '16 at 21:13
  • Recursively as in Fold[Outer[criterion,##]& , list1, {list2,..}] or do you mean something else? Each of my lists is a list of associations, and my criterion is that all the lists of associations in each tuple must be "overlapping" on at least two associations with respect to the value of a given key. (example: considering the key "b", {<|"a"->1,"b"->2|>, <|"a"->1,"b"->3|>} overlaps with {<|"a"->1,"b"->2|>, <|"a"->1,"b"->4|>} only for one value (the value 2) and a tuple containing these two lists would be discarded.) – Ziofil Feb 03 '16 at 21:22
  • 3
    Your approach with Outer will work if you specify Nothing as the third argument of If – Simon Woods Feb 03 '16 at 21:24
  • 2
    Amazing, I did not know about Nothing! (And I love this paradoxical admission) – Ziofil Feb 03 '16 at 21:26
  • @Ziofil Nothing is a very new function. So, i'm not surprised you knew nothing about it. – rcollyer Feb 03 '16 at 21:51
  • 2
    In the past, it was called the "vanishing function" (by Mr. Wizard) and was implemented like this: ## &[]. I still use this because I still have V10.0. – march Feb 03 '16 at 22:14
  • 5
    @march ##&[] evaluates to Sequence[] which is broader in scope than Nothing. But, the biggest difference, AFAIK, is that they have different reactions to Hold. Sequence requires the SequenceHold attribute to hold it, Nothing is fine with HoldAll and its kin, e.g. If[a, first, Nothing] vs. If[a, first, Sequence[]]. Hence, the use of the function. – rcollyer Feb 04 '16 at 04:52
  • @rcollyer. I didn't realize they were actually different! I've use the ##&[] in some silly situations involving more complicated versions of something like Which[expr1, ## &[], expr2, else] vs Which[expr1, Sequence[], expr2, else], where clearly the first works and the second doesn't. Anyway, someday I'll have V10.(>1), and I can use Nothing. – march Feb 04 '16 at 05:11
  • @Ziofil, I meant something else. In your case, Outer still tries all combinations of elements in the two lists, which may or may not be wasteful, given the lists and/or the comparison function. But unless the associations in the lists have a sort of predeterminable structure, you probably need to compare all of them, so your solution will be fine =) – Marius Ladegård Meyer Feb 04 '16 at 05:12
  • 3
    @march just note that Nothing only works in List and Association, so in a lot of cases where ##&[] is needed, Nothing won't work. – rcollyer Feb 04 '16 at 15:55

1 Answers1

3

Solution

Here is a way of efficiently generating tuples of objects that satisfy a given criterion (criterion, which takes a list of objects and returns True or False). We consider the general case of creating tuples from several lists of objects.

This works best if the criterion can be applied before reaching the desired tuple length. (e.g. if I want to make tuples that contain only even numbers, I can discard a partial tuple as soon as it contains an odd number).

Start by defining a filtering function that deletes lists of objects that fail to satisfy the criterion.

filter = Module[{tuple = Flatten[{##}]}, If[criterion@tuple, tuple, Nothing]] &;

Then apply it to pairs of lists using Outer with the third argument set to 1 (so that Outer stops at the first level):

generator = Flatten[Outer[filter, ##, 1], 1] &;

Finally, fold the generator function on all the lists of objects, one by one:

Fold[generator,{list1,list2,...}]

This makes sure that we don't carry over to the next call of Outer lists that already fail to satisfy the criterion.

Performance

We will generate random matrices (e.g. 8 by 4) of numbers between 0 and 10 and we create tuples of even numbers out of the rows.

testList := RandomInteger[10, {8, 4}]
criterion = VectorQ[#, EvenQ] &;

Then we have

Mean@ParallelTable[First@AbsoluteTiming[tuples@testList;], {1000}];
(* 0.027 *)

On my modest machine I get an average of 0.03 seconds per tuple. Let's try a different way:

Mean@ParallelTable[First@AbsoluteTiming[Select[Tuples@testList, criterion];], {1000}];
(* 0.28 *)

Done inefficiently it takes about 10 times longer, but these numbers vary wildly for different problems.

Ziofil
  • 2,470
  • 16
  • 30
  • 4
    I think this is neater way Fold[Select[(Flatten /@ Tuples[{##}]), criterion] &, lists]. A little faster than your version – matheorem Aug 29 '16 at 04:40