Append a list as a column in a dataset

Question

I have a Dataset and a List, called dataset and intervals, and I would like to append the list as a new column of the dataset. I have tried:

ds = Dataset[{
   <|"char" -> 1, "freq" -> 0.1|>,
   <|"char" -> 2, "freq" -> 0.2|>,
   <|"char" -> 3, "freq" -> 0.3|>,
   <|"char" -> 0, "freq" -> 0.4|>
   }]

intervals = {{0, 0.4}, {0.4, 0.7}, {0.7, 0.9}, {0.9, 1.}}
ds[All, <|#, "intervals" -> intervals|> &]

but this adds the whole list to every row, whereas I would like each item of the list to be added to a different row.

Can this be achieved?

Answer 1

One way:

ds // Transpose // Append["intervals" -> intervals] // Transpose

In Mathematica it is usually easier to operate on rows, which is what this solution demonstrates. I would have done the same if I was working with lists too. However, as other answers show there are dataset specific solutions that might work better.

Answer 2

Another way:

ds[MapThread[Append[#1, "intervals" -> #2] &, {#, intervals}] &]

Answer 3

Suppose your column is a list of Association objects or a Dataset:

ds = Dataset[{
<|"char" -> 1, "freq" -> 0.1|>,
<|"char" -> 2, "freq" -> 0.2|>,
<|"char" -> 3, "freq" -> 0.3|>,
<|"char" -> 0, "freq" -> 0.4|>
}];

intervals = {{0, 0.4}, {0.4, 0.7}, {0.7, 0.9}, {0.9, 1.}};

assoc=<|"interval"->#|>& /@ intervals;
col = Dataset[assoc];

Then you can simply use Join to add a column:

Join[ds, assoc, 2]
Join[ds, col, 2]

Answer 4

Here is a way using MapIndexed:

ds[MapIndexed[<| #, "intervals" -> intervals[[#2[[1]]]]|> &]]

---

Streams

In some other software environments, streams are often used to solve problems like this. For lists, we can define a poor-man's version like this:

stream[list_List] := Module[{i = 1}, If[i > Length[list], Missing[], list[[i++]]] &]

Then we can write:

nextInterval = intervals // stream;

ds[All, <| #, "interval" -> nextInterval[] |> &]

Streams can greatly simplify merging operations. Let's say we wanted to add each interval as two columns instead of one:

nextLimit = intervals // Flatten // stream;

ds[All, <| #, "lower" -> nextLimit[], "upper" -> nextLimit[] |> &]

Or perhaps we wanted to add columns from multiple sources:

nextInterval = intervals // stream;
nextCode = "ROYG" // Characters // stream;
nextColor = {Red, Orange, Yellow, Green} // stream;

ds[All, <| #, "interval"->nextInterval[], "code"->nextCode[], "color"->nextColor[] |>&]

Coming Soon?

Since at least version 10.4 of Mathematica there has been an undocumented set of iterator functions. Perhaps they will become documented some day? Then we could officially write:

nextInterval = GeneralUtilities`ToIterator[intervals];

ds[All, <| #, "interval" -> Read[nextInterval] |> &]

Also in the Coming Soon? department, we have the Streaming package of Leonid Shifrin.

Answer 5

More easy way

ds[All, Append["intervals" -> # & /@ intervals]]