39

It seems that this is a really basic question, and I feel that the answer should be obvious to me. However, I am not seeing. Can you please help me? Thanks.

Suppose I have a list of data list and a selector list sel. I would like to Map some function f onto elements in list that correspond to True in the selector list sel.

Thus for the input

list = {1, 10, 100};
sel = {True, False, True};

I would like to obtain the output

{f[1], 10, f[100]}

I can think of some complicated ways to accomplish this (e.g., using Table to step through both list and sel using an iterator i; or find the positions of True in sel using Position and then MapAt at those positions), but not simple ways. Do you have any advice?

rcollyer
  • 33,976
  • 7
  • 92
  • 191
Andrew
  • 10,569
  • 5
  • 51
  • 104
  • 7
    what's wrong with MapAt[f, list, Position[sel, True]] ? – ecoxlinux Aug 23 '12 at 22:58
  • 4
    I like this question as it asks about something that isn't necessarily straightforward, but there are copious ways of accomplishing it. So, +1. – rcollyer Aug 24 '12 at 00:51
  • 1
    @NasserM.Abbasi that is true, and that is why Mr.W's answer is so important: deciding on which one. Arguably, I weigh this against readability and maintainability. If it is not readable/maintainable, then it had better be very, very fast, or speed must be absolutely necessary. Otherwise, I go for something I can understand easily 6mos later. – rcollyer Aug 24 '12 at 01:54
  • 2
    @rcollyer: put another way, "I don't want to have to think so hard when reading code I've written many moons ago." – J. M.'s missing motivation Aug 24 '12 at 09:23
  • 2
    @J.M. of course, there are some code fragments of mine that I have difficulty understanding the next day despite the fact that they work. Modifying them is nightmarish. It is one of the principal reasons I've moved away from using postfix form, and instead tend to write small easily composable functions. Although, postfix creeps in now and again, but it isn't my primary vehicle for running successive transformations anymore. – rcollyer Aug 24 '12 at 14:19

12 Answers12

32

Updated with new functions and additional timings

Since this question inspired so many answers, I think there is a need to compare them.
I have included two of my own functions, freely borrowing from previous answers:

wizard1[] :=
 Inner[Compose, sel /. {True -> f, False -> Identity}, list, List]

wizard2[] :=
 Module[{x = list}, x[[#]] = f /@ x[[#]]; x] & @ 
  SparseArray[sel, Automatic, False]@"AdjacencyLists"

(wizard1 may not work as expected if list is a matrix; a workaround is shown in that post.)

Notes

These timings are conducted with Mathematica 7 on Windows 7 and may differ significantly from those conducted on other platforms and versions.

Specifically, I know this affects Leonid's method, as Pick has been improved between versions 7 and 8. His newer form with Developer`ToPackedArray@Boole is slower on my system, so I used the original.

Rojo's first function had to be modified or it fails on packed arrays, but I believe this affects other versions as well.

kguler's method list /. Dispatch[Thread[# -> f1 /@ #] &@Pick[list, sel]] does not produce the correct result if there are duplicates in list and was omitted from the timings.

Timings with symbolic (undefined) f

Here are timings for all functions, when f is undefined:

Mathematica graphics

$x$ is length of list; $y$ is average time in seconds.

We can see that all the methods appear to have the same time complexity with the exception of one, the line at the top on the right-hand side. This is MapAt[f, list, Position[sel, True]] at it makes quite clear "what's wrong with" this method. The warning on this page regarding MapAt rings true.

Timings for 10^5 in the chart by rank are:

$\begin{array}{rl} \text{wizard2} & 0.02248 \\ \text{ecoxlinux2} & 0.02996 \\ \text{wizard1} & 0.03184 \\ \text{leonid} & 0.03244 \\ \text{simon} & 0.03868 \\ \text{ruebenko} & 0.04116 \\ \text{artes3} & 0.0468 \\ \text{rojo3} & 0.04928 \\ \text{verbeia} & 0.05744 \\ \text{rm2} & 0.0656 \\ \text{rm1} & 0.0936 \\ \text{artes2} & 0.0936 \\ \text{artes1} & 0.0966 \\ \text{jm2} & 0.106 \\ \text{rojo2} & 0.1154 \\ \text{rojo4} & 0.1404 \\ \text{kguler4} & 0.1434 \\ \text{kguler2} & 0.1496 \\ \text{kguler1} & 0.1592 \\ \text{jm1} & 0.1654 \\ \text{rojo1} & 0.3432 \\ \text{ecoxlinux1} & 19.797 \end{array}$

Timings with a numeric compilable f

For an array of 10^6 Reals and with f = 1.618` + # & timings are:

$\begin{array}{rl} \text{wizard2} & 0.04864 \\ \text{leonid} & 0.2154 \\ \text{ecoxlinux2} & 0.452 \\ \text{ruebenko} & 0.53 \\ \text{artes3} & 0.577 \\ \text{simon} & 0.639 \\ \text{wizard1} & 0.702 \\ \text{rojo3} & 0.811 \\ \text{rm1} & 0.982 \\ \text{verbeia} & 1.014 \\ \text{artes2} & 1.06 \\ \text{artes1} & 1.123 \\ \text{rojo2} & 1.279 \\ \text{rm2} & 1.357 \\ \text{jm2} & 1.45 \\ \text{rojo4} & 1.747 \\ \text{kguler4} & 1.841 \\ \text{kguler2} & 1.934 \\ \text{kguler1} & 2.012 \\ \text{jm1} & 2.106 \\ \text{rojo1} & 3.37 \end{array}$

We're not done yet.

Leonid wrote his method specifically to allow for auto-compilation within Map, and my second method is directly based on his. We can take this a step further for a Listable function or one constructed of such functions as is f = 1.618` + # & by using f @ in place of f /@ as described here:

Module[{x = list}, x[[#]] = f @ x[[#]]; x] & @ 
  SparseArray[sel, Automatic, False]@"AdjacencyLists" // timeAvg

0.03496


Reference

The functions, as I named and used them, are:

ruebenko[] :=
 Block[{f},
  f[i_, True] := f[i];
  f[i_, False] := i;
  MapThread[f, {list, sel}]
 ]

artes1[] :=
 (If[#1[[2]], f[#1[[1]]], #1[[1]]] &) /@ Transpose[{list, sel}]

artes2[] :=
 If[Last@#, f@First@#, First@#] & /@ Transpose[{list, sel}]

artes3[] :=
 Inner[If[#2, f, Identity][#] &, list, sel, List]

ecoxlinux1[] :=
 MapAt[f, list, Position[sel, True]]

ecoxlinux2[] :=
 Transpose[{list, sel}] /. {{x_, True} :> f[x], {x_, _} :> x}

rm1[] :=
 Transpose[{list, sel}] /. {x_, y_} :> (f^Boole[y])[x] /. 1[x_] :> x

rm2[] :=
 Transpose[{list, sel}] /. {x_, y_} :> (y /. {True -> f, False -> Identity})[x]

rojo1[] :=
 With[{list = Developer`FromPackedArray@list},
  Normal[SparseArray[{i_ /; sel[[i]] :> f[list[[i]]], i_ :> list[[i]]}, Dimensions[list]]]
 ]

rojo2[] :=
 Total[{#~BitXor~1, #} &@Boole@sel {list, f /@ list}]

rojo3[] :=
 If[#1, f[#2], #2] & ~MapThread~ {sel, list}

rojo4[] :=
 #2 /. _ /; #1 :> f[#2] & ~MapThread~ {sel, list}

jm1[] :=
 MapIndexed[If[sel[[Sequence @@ #2]], f[#1], #1] &, list]

jm2[] :=
 MapIndexed[If[Extract[sel, #2], f[#1], #1] &, list]

verbeia[] :=
 If[#2, f[#1], #1] & @@@ Transpose[{list, sel}]

kguler1[] :=
 MapThread[(#2 f[#1] + (1 - #2) #1) &, {list, Boole[#] & /@ sel}]

kguler2[] :=
 (#2 f[#1] + (1 - #2) #1) & @@@ Thread[{list, Boole@sel}]

(*kguler3[]:=
list/.Dispatch@Thread[#->f/@#]&@Pick[list,sel]*)

kguler4[] :=
  Inner[(#2 f[#1] + (1 - #2) #1) &, list, Boole@sel, List]

simon[] :=
 Block[{g},
  g[True, x_] := f[x];
  g[False, x_] := x;
  SetAttributes[g, Listable];
  g[sel, list]
 ]

leonid[] :=
  With[{pos = Pick[Range@Length@list, sel]},
   Module[{list1 = list},
    list1[[pos]] = f /@ list1[[pos]];
    list1
   ]
  ]

wizard1[] :=
 Inner[Compose, sel /. {True -> f, False -> Identity}, list, List]

wizard2[] :=
 Module[{x = list}, x[[#]] = f /@ x[[#]]; x] & @ 
  SparseArray[sel, Automatic, False]@"AdjacencyLists"

Timing code:

SetAttributes[timeAvg, HoldFirst]
timeAvg[func_] := 
  Do[If[# > 0.3, Return[#/5^i]] & @@ Timing@Do[func, {5^i}], {i, 0, 15}]

funcs = {ruebenko, artes1, artes2, artes3,(*ecoxlinux1,*)ecoxlinux2, 
   rm1, rm2, rojo1, rojo2, rojo3, rojo4, jm1, jm2, verbeia, kguler1, 
   kguler2,(*kguler3,*)kguler4, simon, leonid, wizard1, wizard2};

ClearAll[f]
time1 = Table[
  list = RandomInteger[99, n];
  sel = RandomChoice[{True, False}, n];
  timeAvg@ fn[],
  {fn, funcs},
  {n, 10^Range@5}
 ] ~Monitor~ fn

f = 1.618 + # &;
time2long = Table[
    list = RandomReal[99, 1*^6];
    sel = RandomChoice[{True, False}, 1*^6];
    {fn, timeAvg@ fn[]},
    {fn, funcs}
   ] ~Monitor~ fn
user42582
  • 4,195
  • 1
  • 10
  • 31
Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
  • Thanks! I wonder if the speed difference between Map (/@) and Apply at level 1 (@@@) is general? – Verbeia Aug 24 '12 at 01:38
  • @Mr.Wizard Nice work ! You haven't mentioned artes3[]:=Inner[If[#2, f, Identity][#1] &, list, sel, List] which seems faster than artes2. – Artes Aug 24 '12 at 01:43
  • @Rojo I'm not sure if these timings are reliable enough but I found rojo3[] is the fastes of yours but artes3[] was even slightly faster. – Artes Aug 24 '12 at 02:07
  • @Artes (et al) as stated this is a first pass for timings and not conclusive. Further, this test was performed on Mathematica 7 and may not agree with later versions. I already see several errors in my method that I shall correct tomorrow. For now I believe the ranking is mostly right. – Mr.Wizard Aug 24 '12 at 02:11
  • @Mr.Wizard, this seems to be another one of those cases when something has a bug for PackedArrays. :S – Rojo Aug 24 '12 at 02:12
  • Nice. This does answer my question. – ecoxlinux Aug 24 '12 at 02:28
  • rojo1 is second to last if you unpack the RandomInteger first, and artes3 comes between rojo3 and verbeia – Rojo Aug 24 '12 at 02:54
  • That graph might as well not be there... – rm -rf Aug 24 '12 at 02:55
  • @R.M Why? I think it shows that the rankings are quite consistent. – Mr.Wizard Aug 24 '12 at 07:15
  • +1 for beating all with a solution that uses a function that's supposedly been superseded in 1991 – Rojo Aug 27 '12 at 18:46
  • How did you get to know of AdjacencyLists? – Rojo Aug 27 '12 at 18:51
  • +1, but in your last method, you should know explicitly that your function is Listable. This is why I discarded it (of course, I tried that as well, and actually my timings (V8) did not reveal any difference). – Leonid Shifrin Aug 27 '12 at 18:54
  • @Rojo Thanks! :-D I learned that from Oliver Ruebenkoenig on MathGroup. You can get the full list of valid "properties" with SparseArray[{1}]["Properties"] these allow you to access all the internal parts of a SparseArray object by name, which will presumably be more robust than by Part which might change in later versions. – Mr.Wizard Aug 27 '12 at 22:05
  • @Leonid yes, which is why I only mentioned it in a footnote on optimization. Interesting that perhaps in v8 it making that optimization automatically. – Mr.Wizard Aug 27 '12 at 22:06
20

Perhaps something like this:

list = {1, 10, 100};
sel = {True, False, True};
MapThread[f, {list, sel}]
(* {f[1, True], f[10, False], f[100, True]} *)

So, like:

f[i_, True] := f[i]
f[i_, False] := i

MapThread[f, {list, sel}]
(* {f[1], 10, f[100]} *)
16
If[ #[[2]], f[#[[1]]], #[[1]]] & /@ Transpose[{list, sel}]
{f[1], 10, f[100]}

this should be a bit faster :

If[ Last @ #, f @ First @ #, First @ #] & /@ Transpose[{list, sel}]

or using Inner :

Inner[ If[#2, f, Identity][#1] &, list, sel, List]
Artes
  • 57,212
  • 12
  • 157
  • 245
15

This version seems to be about twice faster than the fastest so far (generally, as much faster as small is a fraction of selected elements), and about an order of magintude faster when Listable functions are mapped on a numerical list - since it automatically utilizes Map auto-compilation in such cases:

ClearAll[conditionalMap];
conditionalMap[f_, lst_, condlst_] :=
  With[{pos = Pick[Range[Length[lst]], Developer`ToPackedArray@Boole[condlst], 1]},
    Module[{list1 = lst},
      list1[[pos]] = f /@ list1[[pos]];
      list1]];
Leonid Shifrin
  • 114,335
  • 15
  • 329
  • 420
  • You would think that that would result in going over the list multiple times and cause it to be much slower, but Pick combined with the vector form of Equals is such a powerful combination, and thoroughly underutilized. Unfortunately, Pick does not usually occur to me to use, +1. – rcollyer Aug 24 '12 at 14:24
  • @rcollyer Thanks for the upvote. Actually, this was first thing that came to my mind given effciency considerations, and the desire to utilize autocompilation of Map automatically. – Leonid Shifrin Aug 24 '12 at 14:31
  • While this is a nice answer, a comment on Pick is that is unpacks. –  Aug 24 '12 at 14:32
  • @ruebenko I thought it no longer did in version 8? – Mr.Wizard Aug 24 '12 at 14:36
  • Grrrrrrrrrrrrrrrrrrrreat – Rojo Aug 24 '12 at 14:39
  • @ruebenko Good point, please see an updated version (which does not unpack unless the mapped function itself is symbolic and demands it). – Leonid Shifrin Aug 24 '12 at 14:41
  • 2
    @Mr.Wizard, in a way there were two 'issues' one Leonid fixed and this one is not an issue any longer unpack Developer`PackedArrayQ@Pick[Range[5], RandomInteger[{0, 1}, 5], 1] - but this one still does Developer`PackedArrayQ@Pick[Range[5], RandomChoice[{True, False}, 5]] –  Aug 24 '12 at 14:50
  • @ruebenko I would say that the issue I did not fix is really a flaw of the original formulation of the problem, which would be better off by using 1 and 0 in place of True and False to select elements, if one wants to incorporate numeric functions / lists as well. – Leonid Shifrin Aug 24 '12 at 14:55
13

How about something unconventional?

Transpose[{list, sel}] /. {x_, y_} :> (f^Boole[y])[x] /. 1[x_] :> x
(* {f[1], 10, f[100]} *)

Again, another unconventional solution:

Transpose[{list, sel}] /. {x_, y_} :> (y /. {True -> f, False -> Identity})[x]
rm -rf
  • 88,781
  • 21
  • 293
  • 472
10

Here is another way:

Transpose[{list, sel}] /. {{x_, True} :> f[x], {x_, _} :> x}

Although I think that MapAt with Position seems the cleanest way.

ecoxlinux
  • 979
  • 4
  • 10
  • I don't think MapAt is cleaner than this. But I would use Transpose instead of Thread +1 – Rojo Aug 23 '12 at 23:23
  • @Rojo, Thanks. I guess it is in the eye of the beholder. Updated to use Transpose. – ecoxlinux Aug 23 '12 at 23:29
  • Yes, it's a matter of taste. I like it more because it is faster, an can be put as a small \[Transpose]. And because I like algebra stuff – Rojo Aug 23 '12 at 23:32
  • @Rojo, Oh, I agree with your Transpose comment. I was referring to MapAt versus this. :) – ecoxlinux Aug 23 '12 at 23:36
10

Just playing around

Normal@SparseArray[{i_ /; sel[[i]] :> f[list[[i]]], i_ :> list[[i]]}, Dimensions@list]

Another playful one

Total[{#~BitXor~1, #} &@Boole@sel {list, f /@ list}]

An almost similar solution to @Artes and @ruebenko's that I find neater could be

If[#1, f[#2], #2] & ~ MapThread ~ {sel, list}

The function on the lhs of MapThread could be written as #2 /. _ /; #1 :> f[#2] &,

or as

Function[sel, 
     If[sel, f, 
      RandomChoice[{Identity, 
        Evaluate, # &}]]][#1]@#2 &~MapThread~{sel, list}
Rojo
  • 42,601
  • 7
  • 96
  • 188
9

Another one, similar to ruebenko's but a bit faster on my machine:

simon[] := Block[{g},
  g[True, x_] := f[x];
  g[False, x_] := x;
  SetAttributes[g, Listable];
  g[sel, list]]
Simon Woods
  • 84,945
  • 8
  • 175
  • 324
8

I quite like the following myself:

list = {1, 10, 100}; sel = {True, False, True};

MapIndexed[If[sel[[Sequence @@ #2]], f[#1], #1] &, list]
    {f[1], 10, f[100]}

Here, we leverage the fact that MapIndexed[] conveniently produces the position of the objects its first argument is being mapped at. For the positions to be usable by Part[], one has to turn the List[] of positions into a Sequence[] that can be spliced into Part[]. Otherwise, here is an alternative:

MapIndexed[If[Extract[sel, #2], f[#1], #1] &, list]

As mentioned in the docs, "Extract[expr, {i, j, …}] is equivalent to Part[expr, i, j, …]."


To demonstrate the flexibility of this approach, here's how to adapt it if list and sel are matrices as opposed to one-dimensional lists:

list = {{2, 1, 1, -3}, {-8, -1, 0, 2}, {-4, 4, -8, -6}, {-8, -1, 9, -2}};
sel = SparseArray[{Band[{1, 1}] -> True, {4, _} -> True}, {4, 4}, False];

MapIndexed[If[Extract[sel, #2], f[#1], #1] &, list, {ArrayDepth[list]}]
    {{f[2], 1, 1, -3}, {-8, f[-1], 0, 2}, {-4, 4, f[-8], -6}, {f[-8], f[-1], f[9], f[-2]}}

The ArrayDepth[] ensures that f[] is only applied to the innermost elements of the given matrices; the same behavior will be seen if instead of matrices, you have rank-$n$ tensors.


Here's a generalization of the snippets given above, encapsulated as a routine blending both the qualities of Map[] and Pick[]:

MapPick[fun_, list_, sel_, patt_, lev_] := 
  MapIndexed[If[MatchQ[Extract[sel, #2], patt], fun[#1], #1] &, list, lev] /;
  Dimensions[list] === Dimensions[sel]

MapPick[fun_, list_, sel_, patt_: True] := MapPick[fun, list, sel, patt, {ArrayDepth[list]}]
J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
7

For those that don't like the look of multiple nested brackets arising from the use of Part (eg [[1]]), I would point out that Artes' answer is equivalent to using Apply at level 1, which has the shorthand syntax @@@

If[#2, f[#1], #1] & @@@ Transpose[{list, sel}]
Verbeia
  • 34,233
  • 9
  • 109
  • 224
  • 4
    and for those that don't like [] arising from a function call, we have If @@ {#2, f@#1, #1} & @@@ Transpose@{list, sel} :D :P (although, it doesn't respect If's attributes) – rm -rf Aug 24 '12 at 00:00
7
 MapThread[(#2 f1[#1] + (1 - #2) #1) &, {list, Boole@ sel}]

or

(#2 f1[#1] + (1 - #2) #1) & @@@ Thread[{list, Boole@sel}]

or

Inner[(#2 f1[#1] + (1 - #2) #1) &, list, Boole@sel, List]

or

 list /. Dispatch[Thread[# -> f1 /@ #] &@Pick[list, sel]]
kglr
  • 394,356
  • 18
  • 477
  • 896
4

This is a great syntactic sugar:

MapIf[function_, list_] := Map[
   With[{r = function[#]}, If[r =!= Null, r, #]]&, list];

MapIf[function_, list_, test_] := Map[
   If[test[#], function[#], #]&, list];

In[180]:= MapIf[f, # > 3 &, {1, 2, 3, 4, 5}]

Out[180]= {1, 2, 3, f[4], f[5]}
M.R.
  • 31,425
  • 8
  • 90
  • 281
  • It seems the 3-param definition of MapIf should flip the order of list_ and test_ , that is, MapIf[function_, test_, list_] := ... produces the expected result, whereas running as-is gives MapIf[f, # > 3 &, {1, 2, 3, 4, 5}] -> (If[{1, 2, 3, 4, 5}[#1], f[#1], #1] &)[#1 > 3] & – Rabbit Mar 15 '20 at 20:57