9

I've got following data I'm reading from a file:

data = {{{2013, 1, 11}, 4},
   {{2013, 1, 12}, 1},
   {{2013, 1, 14}, 1},
   {{2013, 1, 16}, 2},
   {{2013, 1, 18}, 1},
   {{2013, 1, 19}, -1, 16},
   {{2013, 1, 20}, 2},
   {{2013, 1, 21}, 2}
   };

(notice that list elements may have an extra item.) Now I want a similar datelist with each time the accumulated sum (ignoring possible third elements after a data item, like the 16 in the example). Procedurally I'd do something like this: 

tally = 0;
datelisttally = {};
For[i = 1, i <= Length[data],
  tally = tally + data[[i]][[2]]; 
  datelisttally = Append[datelisttally, {data[[i]][[1]], tally}]; i++];
DateListPlot[datelisttally, PlotRange -> {0, Automatic}, Joined -> True]

but I'd like to learn the best way to do this with functional programming, best being the fastest. (The file may hold several thousands of items.)

update
I've benchmarked the 3 current solutions, and these are the results (list length = 1 million elements, averaged over 50 runs):

  • Szabolcs's: 0.33 s
  • PlatoManiac's: 0.54 s
  • Leonid's: 3.33 s
Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
stevenvh
  • 6,866
  • 5
  • 41
  • 64

3 Answers3

12

One (not functional) possibility out of many:

dataacc = data;
dataacc[[All, 2]] = Accumulate[data[[All, 2]]]

This will keep the third elements. If you want those removed, you can start with dataacc = data[[All, 1 ;; 2]];

I didn't benchmark, but I expect this to be fast.

Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
  • 2
    Thanks. I didn't know about the Accumulate function. (which reminds me of what a fellow student of mine said: "programming is like magic. It's easy once you know the magic words.") – stevenvh Jan 14 '13 at 15:45
  • @stevenvh Before Accumulate, we used FoldList (like Leonid's answer). It's like Accumulate generalized for operations other than Plus. – Szabolcs Jan 14 '13 at 15:48
  • @stevenvh Accumulate is optimized on packed arrays, which is one of the good reasons to use it in place of FoldList for running sums. In this case, since the dataacc[[All, 2]] is not going to be packed at the start, it may improve the performance to pack it first, before the call to Accumulate. The resulting list is going to be unpacked during an assignment though, so I don't know how much speed improvement this may bring, if any. – Leonid Shifrin Jan 14 '13 at 15:52
  • @Leonid Perhaps the dates can be converted to AbsoluteTime format. Then we can keep everything packed. – Szabolcs Jan 14 '13 at 15:54
  • @Szabolcs The question is, how time-consuming would that be as compared to what Accumulate or FoldList have to do. Adding integers is supposedly a much faster operation than date conversion. – Leonid Shifrin Jan 14 '13 at 15:56
6

Here is an arguably functional-style version without in-place modifications:

FoldList[{First@#2, #2[[2]] + #1[[2]]} &, First@data, Rest@data]

It is likely to be slower than the solution proposed by Szabolcs.

Leonid Shifrin
  • 114,335
  • 15
  • 329
  • 420
5

Another one!

val = data[[All, 2]];
Transpose@{data[[All, 1]], Take[FoldList[Plus, 0, val], -Length@val]}
PlatoManiac
  • 14,723
  • 2
  • 42
  • 74
  • I've thought of the Transpose + Take too, but I'm not sure it will be the fastest. I'll benchmark the different answers later and report back. In any case, thanks. – stevenvh Jan 14 '13 at 15:48