Error changing Dataset using Part

Question

Update

It was a bug in the documentation of V10.0, this functionality was not implemented yet, and V10.1 changed the documentation, removing it. It's a pity, because it's a very useful operations, common in other languages like R. I miss data.frame like notation in Mathematica.

---

In the new guide Computation With Structured Datasets we can find this part, on how to change a Dataset

But if we create a Dataset like:

ds=Dataset[{<|"a"->1,"b"->"x"|>,<|"a"->2,"b"->"y"|>,<|"a"->6,"b"->"z"|>}];

And then make:

ds[[1, 1]] = 2

Or, closer to my real case test:

ds[[All, "a"]] = Accumulate@Normal@ds[[All, "a"]]

We get an error:

"Part specification ds[[1,1]] is longer than depth of object"

"Part specification ds[[All,1]] is longer than depth of object. "

Is this a Bug?

Setting is not working on Dataset as stated by documentation.

This post on Wolfram Community

Answer 1

I'm the developer of Dataset.

Yes, this is a gross documentation oversight. We planned this functionality but had to push it back to a point release. Somehow no-one caught this piece of legacy documentation.

I've filed a bug on the documentation problem right now, it's easy to fix.

As for when L-value assignment will be available, I'm hoping 10.0.1 or 10.0.2, which are in the next month or two. It gets complicated, because you might well want to write things like:

dataset[ Select[#age > 30&] , "salary"] *= 2

That's certainly a powerful kind of operation, but also hard to implement. Even part-like assignments can get complicated when you are assigning multidimensional datasets to each other.

Thanks for trying the functionality, though!

Answer 2

I have implemented the underlying kernel functionality that is needed to make this possible. However it is not yet implemented on the Dataset side. I don't think this will happen in the immediate future owing to other priorities.

Here is a stop-gap that implements a simple version of mutable updating, this of course is not production-grade. I'm happy for anyone who wants to modify this answer to extend its functionality, add error handling, etc.

Unprotect[Dataset];
Language`SetMutationHandler[Dataset, DatasetMutationHandler];

SetAttributes[DatasetMutationHandler, HoldAllComplete];
DatasetMutationHandler[Set[sym_Symbol[[args___]], newvalue_]] := Block[{tmp},
    tmp = Normal[sym];
    tmp[[args]] = If[Dataset`ValidDatasetQ[newvalue], Normal[newvalue], newvalue];
    sym = Dataset[tmp];
];

You can use it as follows:

In[51]:= d = Dataset[{1, 2, 3}];
d[[2 ;; 3]] = 99;
Normal[d]

Out[53]= {1, 99, 99}

Answer 3

In lieu of Set, the Query syntax offers various ways to update selective elements of a dataset. For example, we can change the value of the field a in the first row like this:

ds[{1 -> (<| #, "a" -> 999|> &)}]

or like this:

ds[{1 -> Query[{"a" -> (999 &)}]}]

Multiple fields can be updated simultaneously:

ds[{1 -> (<| #, "a" -> 999, "b" -> "ZZZ" |> &)}]

We can update selective rows, in this case field "b" in rows with even a:

ds[All, If[EvenQ[#a], <| #, "b" -> "!!!!"|>, #] &]

The accumulation use case can be accomplished like this:

With[{a = ds[Accumulate, "a"]}
, ds @ MapIndexed[<| #, "a" -> a[[First@#2]] |> &]
]

or like this:

Module[{acc = 0}, ds[All, {"a" -> (acc += # &)}]]

Note that none of these operations involve destructively altering the dataset, so they should all read ds = ds[...] if desired. Presumably Set will eventually perform destructive updates in those restricted circumstances that Mathematica tolerates mutation.

Answer 4

Though I don't know what is the efficiency impact of it, a workaround could be converting the Dataset to Association by Normal, making the update on the Association, then converting it back to Dataset.

ds = Dataset[{<|"a" -> 1, "b" -> "x"|>, <|"a" -> 2, "b" -> "y"|>, <|"a" -> 6, "b" -> "z"|>}]

ds = Module[{temp = Normal[ds]},
            temp[[All, "a"]] = Accumulate[temp[[All, "a"]]];
            temp // Dataset]

Answer 5

It looks like that in Mathematica 13, there is still no easy way to modify values in a Dataset.