11

Im trying to escape doing double loops given the large number of items in the arrays.

I have two list with a format like this:

list1={{1,2,0},{1,3,0},{4,6,0},{2,3,0}} (*Third element of each item is 0*)

list2={{3,2,1},{1,3,1},{4,5,1}} (*Third element of each item is 1*)

List might not be of equal length.

If the first two elements of a list match I want to replace that item in list1 with the corresponding item in list2.

result={{1,2,0},{1,3,1},{4,6,0},{2,3,0}}

Since there seems to be confusion I'll provide another example:

test1={{0.5,0.5,0},{1,1,0},{1.5,1.5,0},{2.0,2.0,0}};
test2={{0.5,0.5,1},{2.0,2.0,1}};

The results should then be:

 result={{0.5,0.5,1},{1,1,0},{1.5,1.5,0},{2.0,2.0,1}};

Which includes every item in test1.

I am sorry if its confusing, this is my first post here.

I have this setup but it takes too much time:

densitydata = Reap[Do[If[{cross[[m, 1]],cross[[m, 2]]} == {fullzone0[[n,1]] = fullzone0[[n, 2]]},
Sow[cross[[m]]], Sow[fullzone0[n]]],
{n, 1,Dimensions[fullzone0][[1]]}, {m, 1,Dimensions[cross][[1]]}]][[2]][[1]];
kglr
  • 394,356
  • 18
  • 477
  • 896
Giovanni Baez
  • 967
  • 5
  • 12
  • Welcome! To make the most of Mma.SE start by taking the [tour] now. There are things to do after your question is answered. It's a good idea to stay vigilant for some time, better approaches may come later improving over previous replies. Experienced users may point alternatives, caveats or limitations. New users should test answers before voting and wait 24 hours before accepting the best one. – rhermans Jul 12 '17 at 17:05
  • Thanks for taking the [Tour]! Participation is essential for the site, please come back to do your part tomorrow. As you receive give back, vote and answer questions, keep the site useful, be kind, correct mistakes and share what you have learned. – rhermans Jul 12 '17 at 17:10
  • What should be returned for the lists {{1, 2, 0}} and {{3,2,1}, {1, 2, 1}}? – Carl Woll Jul 12 '17 at 17:53
  • @CarlWoll It should return every element in list1 with the elements that meet the criteria replaced. so it should be. result={{1,2,0},{1,3,1},{4,6,0},{2,3,0} – Giovanni Baez Jul 12 '17 at 18:00
  • In addition, what should be returned for the lists {{1, 2, 0}, {3, 2, 0}} and {{3, 2, 1}? – Carl Woll Jul 12 '17 at 18:00
  • I'm giving examples of list1 and list2 different from yours, and wondering what the expected output is. Specifically, are you comparing the first element of list1 with only the first element of list2, or with all elements of list2? – Carl Woll Jul 12 '17 at 18:01
  • @CarlWoll Oh, right. So if the lists are {{1, 2, 0}} and {{3,2,1}, {1, 2, 1}} then the output should be {{3,2,1}, {1, 2, 1}}. However, for my arrays, all elements present in list2 are in list1. However if for some reason this happens the output most always include any item with the third element equal to one. – Giovanni Baez Jul 12 '17 at 19:06
  • I expected {{1, 2, 0}} to become {{1, 2, 1}}, not {{3, 2, 1}, {1, 2, 1}}. Why did the length of list1 increase? Also, your example has a list2 where not elements in list2 are in list1. – Carl Woll Jul 12 '17 at 19:09
  • The end goal is to have a list with all possible points {x,y,1} that are in list2,list1 just marks all possibles x & y points. Hence why list1 is of the form {x,y,0}. If for some reason there is an x,y combination not present in list1 it should be added. – Giovanni Baez Jul 12 '17 at 19:27
  • Giovanni, (1) If the first two elements of a list match I want to replace that item in list1 with the corresponding item in list2 AND (2) If for some reason there is an x,y combination not present in list1 it should be added. You can't have both, no? – kglr Jul 12 '17 at 21:41
  • @GiovanniBaez i am lost now. Your question mentions something else and now you are mentioning that all elements absent in the first list should be added to it? is it even possible? – Ali Hashmi Jul 12 '17 at 22:09
  • The final result should be every element in list1, the one with third element 0, with the special cases, the ones appearing in list2, replaced. In my arrays the cases prompted by @CarlWolll will never show up. – Giovanni Baez Jul 12 '17 at 22:11
  • @AliHashmi I edited the question with another example, to try and show what Im searching for. Sorry if I confused you guys. – Giovanni Baez Jul 12 '17 at 22:20
  • @GiovanniBaez please see Carl's or my answer. hope it helps – Ali Hashmi Jul 12 '17 at 22:39

12 Answers12

11

Update after discussions in comments

I've revised my code to use Replace with a level spec, and to create rules only from list2, since the latter list can be much smaller than the first list. This provides a speed boost of about 50%.

update[l1_,l2_] := Module[{p, q, r=l1},
    p = Replace[
        l1[[All,;;2]],
        Dispatch @ Thread @ Rule[l2[[All,;;2]], l2[[All,3]]],
        {1}
    ];
    q = Replace[p[[All,0]], {List->0,_->1}, {1}];
    r[[Pick[Range[Length[l1]], q, 1], 3]] = Pick[p,q,1];
    r
]

For your initial example:

update[list1, list2]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

For your latest example:

test1={{0.5,0.5,0},{1,1,0},{1.5,1.5,0},{2.0,2.0,0}};
test2={{0.5,0.5,1},{2.0,2.0,1}};

update2[test1,test2]

{{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2., 2., 1}}

Timing

Since you mention working with large arrays, here's the speed of update on some made up data:

list1=Sort@RandomReal[1,{10^4,3}];
list2=RandomSample[list1,10^3];
list2[[All,3]]=RandomReal[1,10^3];

update[list1,list2];//AbsoluteTiming

{0.01681, Null}

list1=Sort@RandomReal[1,{10^6,3}];
list2=RandomSample[list1,10^5];
list2[[All,3]]=RandomReal[1,10^5];

update[list1,list2];//AbsoluteTiming

{1.99171, Null}

Old code

update2[l1_,l2_] := Module[{r=l1, l=Join[l2,l1]},
    r[[All,3]] = r[[All, ;;2]] /. Dispatch@Thread@Rule[l[[All, ;;2]],l[[All,3]]];
    r
]
Carl Woll
  • 130,679
  • 6
  • 243
  • 355
10
list1 = {{1, 2, 0}, {1, 3, 0}, {4, 6, 0}, {2, 3, 0}};
list2 = {{3, 2, 1}, {1, 3, 1}, {4, 5, 1}, {4, 3, 1}};

If[Most[#] === Most[#2], #2, #] & @@@ Transpose[{list1, list2}]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

C. E.
  • 70,533
  • 6
  • 140
  • 264
9
f[v1_, v2_] := If[Most[v1] == Most[v2], v2, v1];
MapThread[f, {list1, list2}]

which gives you {{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

Revised to incorporate your unequal length list condition

Map[(match = Cases[list2, Join[Most[#], {_}]]; 
     If[match == {}, #, match[[1]]]) &, list1]
Bill
  • 12,001
  • 12
  • 13
8
mask = Boole[# == {0, 0}] & /@ Unitize[list1[[;; , ;; 2]] - list2[[;; , ;; 2]]];

list1 mask + (1 - mask) list2

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

Update: For lists with possibly unequal lengths:

ClearAll[f1]
f1 = Module[{ml = Min[Length /@ {##}], mask, l1 = #, l2 = #2}, 
    mask = Boole[# == {0, 0}] & /@ Unitize[l1[[;; ml, ;; 2]] - l2[[;; ml, ;; 2]]];
    l1[[;; ml, -1]] = l1[[;; ml, -1]] mask + (1 - mask) l2[[;; ml, -1]]; l1] &;

Examples:

list1 = {{1, 2, 0}, {1, 3, 0}, {4, 6, 0}, {2, 3, 0}} ;
list2 = {{3, 2, 1}, {1, 3, 1}, {4, 5, 1}, {4, 3, 1}};
f1[list1, list2]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

SeedRandom[1]
list3 = {#, #2, 2} & @@@ RandomInteger[5, {3, 3}];

f1[Join[list1, list3], list2]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}, {4, 2, 2}, {0, 1, 2}, {0, 2, 2}}

f1[list1, Join[list2, list3]]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

test1 = {{0.5, 0.5, 0}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 0}};
test2 = {{0.5, 0.5, 1}, {2.0, 2.0, 1}};
f1[test1, test2]

{{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2., 2., 0}}

kglr
  • 394,356
  • 18
  • 477
  • 896
  • The first approach is really nice, I never thought of just using matrix algebra for list manipulation like this. – Giovanni Baez Jul 12 '17 at 17:07
  • f1[{{1, 2, 0}, {1, 3, 0}}, {{1, 3, 1}, {1, 2, 1}}] produces a strange result. I would expect either {{1, 2, 0}, {1, 3, 0}} or {{1, 2, 1}, {1, 3, 1}}, but not {{1, 2, 0}, {1, 3, 1}}. – Carl Woll Jul 12 '17 at 20:55
  • Thank you @Carl. Fixed it now. I had list1 and list2 leftover from previous versions changed them to l1 and l2. – kglr Jul 12 '17 at 21:16
  • @kglr This method seems to be the one better suited for very large arrays. If the numbers in the list are not integers would it present a problem in the execution of the code? – Giovanni Baez Jul 12 '17 at 21:47
  • @kglr The following test cases produce a negative value for the third element in the result. test1= {{0., 0.045105, 0.}, {0., 0.047361, 0.}, {0., 0.049616, 0.}, {0.998047, 1.67229, 0.}},test2={{0.208984, 0.120657, 1}, {0.765625, 1.59448, 1}} give {{0., 0.045105, -1.}, {0., 0.047361, -1.}, {0., 0.049616, 0.}, {0.998047, 1.67229, 0.}} – Giovanni Baez Jul 12 '17 at 21:52
  • Am I crazy or are list1 and list2 hard-coded in f1? – Mr.Wizard Jul 12 '17 at 23:33
  • Mr.Wizard, yes they were -- before my edit 2 hours ago:) – kglr Jul 12 '17 at 23:47
8

The question was changed after I wrote my first method making it invalid.

For the new question I propose simply using Associations.

test1 = {{0.5, 0.5, 0}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 0}};
test2 = {{0.5, 0.5, 1}, {2.0, 2.0, 1}};

fn2 =
  KeyValueMap[Append] @*
      (AssociationThread[#[[All, ;; -2]] -> #[[All, -1]]] &) @* Join;

fn2[test1, test2]

{{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2., 2., 1}}

This is faster than the presently Accepted answer:

list1 = Sort@RandomReal[1, {10^6, 3}];
list2 = RandomSample[list1, 10^5];
list2[[All, 3]] = RandomReal[1, 10^5];

r1 = update2[list1, list2]; // AbsoluteTiming
r2 = fn2[list1, list2];     // AbsoluteTiming

r1 === r2

{3.30789, Null}

{2.30772, Null}


True

Method for original question:

pos[a_, b_][p_, i_] :=
 p[[
   "AdjacencyLists" //
     SparseArray[Unitize[Subtract @@ {a, b}[[All, p, i]]], Automatic, 1]
  ]]

fn[a_, b_] :=
  Module[{x = a, pp},
    pp = Fold[pos[a, b], Range@Length@b, {1, 2}];
    x[[pp]] = b[[pp]];
    x
  ]

fn[list1, list2]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

If I have time I'll benchmark this and other answers later unless someone else undertakes that first.

Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
7

Assuming: "for my arrays, all elements present in list2 are in list1"

The code:

GatherBy[Join[list1, list2], Most][[All, -1]]

works if either DuplicateFreeQ[Drop[list1, None, -1]] is True or if it is False you want duplicates of list1 to be deleted.

Coolwater
  • 20,257
  • 3
  • 35
  • 64
6
list1 = {{1, 2, 0}, {1, 3, 0}, {4, 6, 0}, {2, 3, 0}, {3, 2, 0}};
list2 = {{3, 2, 1}, {1, 3, 1}, {4, 5, 1}};

replace[list1_, list2_] := 
Module[{temp, pos, val, l = list1}, 
temp = Outer[If[SameQ @@ Map[Most]@{##}, Last[#2]] &, list1, list2, 1];
val = Cases[temp, _?NumericQ, {2}];
pos = Map[First]@Position[temp, _?NumericQ, {2}];
l[[pos, -1]] = val;
l]

replace[list1,list2]

(* {{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}, {3, 2, 1}} *)

one more case:

test1 = {{0.5, 0.5, 0}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 0}};
test2 = {{0.5, 0.5, 1}, {2.0, 2.0, 1}};

replace[test1, test2]
(* {{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2., 2., 1}} *)
Ali Hashmi
  • 8,950
  • 4
  • 22
  • 42
  • Your method modifiers list1 (or test1). That means something like replace[{{1, 2, 0}}, {{1, 2, 2}}] won't work. Also, it always updates the result to 1. – Carl Woll Jul 12 '17 at 22:54
  • @CarlWoll now it does not modify list1 in place. Also now it does not update to 1 always. But your performance is much much superior to this answer – Ali Hashmi Jul 13 '17 at 00:02
5

Another one-liner:

list1[[All, ;; 2]] /. (Join[list2, list1] /. {x_, y_, z_} :>  ({x, y} -> {x, y, z}))

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

 

For the OP second example: 

test1[[All, ;; 2]] /. (Join[test2, test1] /. {x_, y_, z_} :>  ({x, y} -> {x, y, z}))

{{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2., 2., 1}}

 

(With ReplaceAll, the first rule that matches is applied to each part)

{1, 2, 3, 4, 1} /. {1 -> 10, 1 -> 100}

{10, 2, 3, 4, 10}

Edit

If the third element of list1 is always 0 and the third element of list2 is always 1 the following, I think, should also work:

(list1 /. (list2 /. {x_, y_, 1} :>  ({x, y, 0} -> {x, y, 1})))

and 

(test1 /. (test2 /. {x_, y_, 1} :>  ({x, y, 0} -> {x, y, 1})))
user1066
  • 17,923
  • 3
  • 31
  • 49
4
list1 = {{1, 2, 0}, {1, 3, 0}, {4, 6, 0}, {2, 3, 0}};
list2 = {{3, 2, 1}, {1, 3, 1}, {4, 5, 1}, {4, 3, 1}};

f[{{x__, _}, a : {x__, _}}] := a
f[{a_, _}] := a

f /@ Transpose[{list1, list2}]

{{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}}

eldo
  • 67,911
  • 5
  • 60
  • 168
1
test1 = {{0.5, 0.5, 0}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 0}};
test2 = {{0.5, 0.5, 1}, {2.0, 2.0, 1}};

KeyValueMap[Append] @ Map[Last] @ GroupBy[Most -> Last] @ Join[test1, test2]

{{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2., 2., 1}}

eldo
  • 67,911
  • 5
  • 60
  • 168
1

Using Lookup:

Clear["Global`*"];

list1 = {{1, 2, 0}, {1, 3, 0}, {4, 6, 0}, {2, 3, 0}};
list2 = {{3, 2, 1}, {1, 3, 1}, {4, 5, 1}};
r1 = {{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}};


test1 = {{0.5, 0.5, 0}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 0}};
test2 = {{0.5, 0.5, 1}, {2.0, 2.0, 1}};
r2 = {{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 1}};


f[our_List, repl_List] := Module[{lut = (Most@# -> #) & /@ repl},
  Sequence @@@ Lookup[lut, {Most@#}, #] & /@ our
  ]


f[list1, list2] == r1
f[test1, test2] == r2

True

True

Syed
  • 52,495
  • 4
  • 30
  • 85
0

First, you can use GroupBy to construct a rule:

l1 = {{1, 2, 0}, {1, 3, 0}, {4, 6, 0}, {2, 3, 0}};
l2 = {{3, 2, 1}, {1, 3, 1}, {4, 5, 1}};
r1 = {{1, 2, 0}, {1, 3, 1}, {4, 6, 0}, {2, 3, 0}};

rule = Values@GroupBy[Catenate@{l1, l2}, Most, 
       If[Length@# > 1, Position[l1, #[[1]]] -> #[[2]], Nothing] &];

Finally, using ReplacePart:

ReplacePart[l1, rule] === r1

(True)

An alternative to construct the above rule using GatherBy and Dispatch:

l1 = {{0.5, 0.5, 0}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 0}};
l2 = {{0.5, 0.5, 1}, {2.0, 2.0, 1}};
r2 = {{0.5, 0.5, 1}, {1, 1, 0}, {1.5, 1.5, 0}, {2.0, 2.0, 1}};

rule = Dispatch@(Position[l1, #1] -> #2 & @@@ 
       Select[GatherBy[Catenate@{l1, l2}, Most], Length@# > 1 &]);

Testing with ReplacePart:

ReplacePart[l1, rule] === r2

(True)
E. Chan-López
  • 23,117
  • 3
  • 21
  • 44