Unexpected behavior of Dataset

Question

This code form documention of Dataset in ref/Dataset->Applicastions->Indexed Tables(Associations of Associations):

planets = ExampleData[{"Dataset", "Planets"}];
moonMass = planets["Earth", "Moons", "Moon", "Mass"];
planets[All, "Moons", Select[#Mass > moonMass/2 &] /* Keys]

But I cannot understand this behavior all the time.I this that Keys should act on the last result to get a outcome same with Keys[planets[All,"Moons",Select[#Mass>moonMass/2&]]].I make example for simulate it.

dataset = Dataset[<|
   ld -> <|"a" -> 1, "b" -> "x", "c" -> {1}|>,
   le -> <|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>,
   lf -> <|"a" -> 3, "b" -> "z", "c" -> {3}|>,
   lg -> <|"a" -> 4, "b" -> "x", "c" -> {4, 5}|>,
   lh -> <|"a" -> 5, "b" -> "y", "c" -> {5, 6, 7}|>,
   li -> <|"a" -> 6, "b" -> "z", "c" -> {}|>|>]

dataset[Select[#a < 4 &] /* f, "c"]

f[<|ld -> {1}, le -> {2, 3}, lf -> {3}|>] As we can see,the f act on the last result.If we want to get a result same to the documentation,the code should be:

dataset[Select[#a < 3 &] /* (Keys /@ # &)]

Do anyone agree this this opinion?

You might be interested in the Long Answer section of (98193) which discusses the action of ascending and descending operators. — WReach, Nov 19 '16 at 15:48

score 4 · Accepted Answer · answered Nov 19 '16 at 01:32

For

Keys[planets[All,"Moons",Select[#Mass>moonMass/2&]]]

First planets[All,"Moons",Select[#Mass>moonMass/2&]] must be resolved before Keys can operate on this. The planets Query returns a Dataset whose first level are the planet names as keys. Therefore Keys returns the planet name keys.

For

dataset[Select[#a < 4 &] /* f, "c"]

You must first appreciate ascending and descending operators of Query (read the Details and Options section). Select and Key ("c" in this case) are descending operators. f is taken to be an ascending operator. Query does the following.

Descend

1.1 Select[#a < 3 &] - This returns dataset with the rows filtered at level 1

1.2 "c" - Get the Values of Key "c" at level 2 for these filtered rows
Ascend

2.1 No further action at level 2

2.2 f - At level 1 apply f to the results

For

dataset[Select[#a < 3 &] /* (Keys /@ # &)]

Select is descending and (Keys /@ # &) is ascending. Select filters the rows and then Keys is Maped onto the Values of each row. Here, the value of each row is the Association that defines the row. Therefore the column name keys are returned. I find it more intuitive to code dataset[Select[#a < 3 &], Keys] as it clearly shows what is happening at level 1 and level 2 of the Dataset.

Hope this helps.

score 0 · Answer 2 · answered Nov 19 '16 at 01:40

It seems like the dataset in the documentation has a different format than your example. They have an association as the value of the last key, you have a list as the value of the last key. If we modify your data as follows:

dataset = 
 Dataset[<|ld -> <|"a" -> 1, "b" -> "x", "c" -> <|"l" -> {1}|>|>, 
   le -> <|"a" -> 2, "b" -> "y", "c" -> <|"l" -> {2, 3}|>|>, 
   lf -> <|"a" -> 3, "b" -> "z", "c" -> <|"l" -> {3}|>|>, 
   lg -> <|"a" -> 4, "b" -> "x", "c" -> <|"l" -> {4, 5}|>|>, 
   lh -> <|"a" -> 5, "b" -> "y", "c" -> <|"l" -> {5, 6, 7}|>|>, 
   li -> <|"a" -> 6, "b" -> "z", "c" -> <|"l" -> {}|>|>|>]

then the following works

dataset[Select[#a < 4 &], "c", #l & /* f]

With your original dataset, the following code would produce the same result

f /@ dataset[Select[#a < 4 &], "c"]

Unexpected behavior of Dataset

2 Answers2