7

Given an example data set:

data = {"region" -> "AA", 
        "systems" -> {{"name" -> 1, 
                        "sub" -> {{"name" -> "sub11"}, {"name" -> "sub12"}}},
                      {"name" -> 2, 
                        "sub" -> {{"name" -> "sub21"}, {"name" -> "sub22"}}}}};

I would like to extract the various systems and subsystems attributes. I would prefer a sysntax similar to JsonPath ot XPath.

So for example, I would like to be able to write something similar to:

data."region" (* AA *)
data."systems"[2]."name" (* 2 *)
data."systems"[2]."sub"[2]."name" (* sub22 *)

Filtering is a nice bonus

 data."systems"[#."name" == 1 &]."sub"."name" (* {"sub11", "sub12"} *)

How could I make this or a similar syntax work?

Using ReplaceRepeated almost works (just the substitutions have to be written in reverse ie "name" /. "systems"[[2]] instead of the more logical "systems"[[2]] -> "name").

{"region", "name" /. "systems"[[2]], "sub"[[2]] /. "systems"[[2]]} //. data    
(* ==> {"AA", 2, {"name" -> "sub22"}} *)

But this fails, as "name" appears on two levels

{"region", "name" /. "systems"[[2]], 
  "name" /. "sub"[[2]] /. "systems"[[2]]} //. data

(* ==> {"AA", 2, 2}  but I would like {"AA", 2, sub22}   *)
Ajasja
  • 13,634
  • 2
  • 46
  • 104
  • 2
    I think that your option list does not qualify as a valid option list, at least in the sense you wish it to be. Options are fundamentally named arguments (and not positional). So, the name, or sequence of names at each level, should uniquely define a value. In your case, your first and second sub-system options are having the same set of names, so it is impossible to distinguish between them without taking their positions into account. If you are asking about generalized options which are also positional, you should clearly say so, but be warned that this is (IMO) a shaky ground. – Leonid Shifrin Jul 16 '12 at 09:23
  • I understand. I just think that in your case, options abstraction is wrong, and trying to shoehorn it into the one where you could use options-related functionality does not strike me as a good idea. Rules and options buy you position-independence, but only of your data can be accessed in a position-independent way. I would either reformulate the problem (e.g. split your data set into several each of which represents a valid option set), or use different data structures, which can distinguish elements based on their position in them. – Leonid Shifrin Jul 16 '12 at 09:32
  • @Szabolcs Thanks, I indeed did not. I can't seem to be able to read the draft from my WordPress blog account though, although I see the draft listed in the "Posts" tab. Is it the correct behavior (I assume that only admin can see read drafts)? – Leonid Shifrin Jul 16 '12 at 09:42
  • @LeonidShifrin Hmm perhaps I should just make this into a relational table and then use SQL. But I was hoping to avoid that... The elements I would like to access are well defined. I just find OptionValue[data, "systems"[[2]] -> "sub"[[2]] -> "name"] easier to read than OptionValue[ OptionValue[OptionValue[data, "systems"][[2]], "sub"][[2]], "name"]. I guess I would like some sort of Xpath approximation where it is also possible to get nodes either by position or name. – Ajasja Jul 16 '12 at 09:51
  • Aha, Including Xpath in my search yields something interesting http://mathematica.stackexchange.com/questions/3916/casesdata-colonkey-vs-casesdata-key-toward-xpath-xquery – Ajasja Jul 16 '12 at 09:58
  • You could probably implement your own option-like data structure to do this, and if you design it to be general enough and easy to use, it may be quite useful to all of us. Or perhaps, we can treat this question as a question of how to design such a data structure, in which case you may wish to reformulate it somewhat, emphasizing that options can not do this. – Leonid Shifrin Jul 16 '12 at 10:06
  • @LeonidShifrin Just one clarification (that I wanted to ask from the beginning): Isn't a list of rules already such a data structure (it just has to be parsed correctly?) – Ajasja Jul 16 '12 at 10:11
  • I would not use a list of rules, just because people are used to the idea that such a list of rules can be applied, and then the order of rules is either irrelevant, or determines the way rules are applied. In either case, I think we are used to the idea that the absolute position of a given rule in a list of rules is irrelevant, and using it here would likely lead to a confusion. – Leonid Shifrin Jul 16 '12 at 10:20
  • Thanks @LeonidShifrin, will think about your comments (For now I'll hack something ugly, since I have to finish a report today). I'll also look into using a Java implementation of JsonPath from mma – Ajasja Jul 16 '12 at 11:00
  • I thought about suggesting Java. This would be a particularly valid choice if your values are of some basic types (strings, integers, etc) rather than general Mathematica expressions. For the latter case, using Java may involve some transformations of the values so that they can be stored on the Java side and reconstructed identically on the Mathematica side. The only fully bulletproof way I know is to use Compress-Uncompress, and store values in Java as compressed strings, but this induces quite an overhead (I tried in the context of set data structure). – Leonid Shifrin Jul 16 '12 at 11:05

2 Answers2

3

Would something like this be suitable?

sys_~s~sub_ := sub /. sys
sys_~s~sub_[n_] := (sub /. sys)[[n]]

data~s~"region"
data~s~"systems"[2]~s~"name"
data~s~"systems"[2]~s~"sub"[2]
data~s~"systems"[2]~s~"sub"[2]~s~"name"

"AA"

2

{"name" -> "sub22"}

"sub22"

Simon Woods
  • 84,945
  • 8
  • 175
  • 324
  • A great idea, thanks! (I replaced ~s~ with \[CircleMinus], but that is just a matter of preference) – Ajasja Jul 16 '12 at 13:51
  • It would also be nice to be able to specify All and Span with tree-structured key-value pair data (eg XML); I asked a similar question: http://mathematica.stackexchange.com/questions/3325/path-queries-for-tree-structured-data – alancalvitti Jul 16 '12 at 17:27
  • @Ajasja, good idea. I forget that Mathematica has all those infix operators. – Simon Woods Jul 16 '12 at 21:01
  • @alancalvitti, I'm not sure what All and Span mean in the context of data structures and XML, it's not really my thing. I assume you are aware of Mathematica's Symbolic XML functionality? – Simon Woods Jul 16 '12 at 21:06
  • @SimonWoods, I use symbolic XML; attributes are stored as key->value pairs. All is easy to explain, it would return all branches from a given node in the data tree. Span would only work on hybrid tree/relational data structures, eg where data at the lower levels is relational matrix, but to access and aggregate it XPath type indexing could be nice syntactic sugar. Routinely in data analysis, I use such hybrid data structures. – alancalvitti Jul 16 '12 at 21:15
3

With a small modification to @SimonWoods answer it's possible to filter the nodes as well:

Clear[CircleMinus]
sys_\[CircleMinus]sub_ := sub /. sys
sys_\[CircleMinus]sub_[n_] := (sub /. sys)[[n]]
sys_\[CircleMinus]sub_[f_Function] := Module[{s},
   s = Select[(sub /. sys), f];
   (*remove redundant {}*)
   s /. {x_List} :> x];

Now, one can do things like select only system nodes that have a particular value of the "name" attribute:

data\[CircleMinus]"systems"[#\[CircleMinus]"name" >=1 &]\[CircleMinus]"sub"[2]\[CircleMinus]"name"
data\[CircleMinus]"systems"[#\[CircleMinus]"name" ==1 &]\[CircleMinus]"sub"[1]\[CircleMinus]"name"
data\[CircleMinus]"systems"[#\[CircleMinus]"name" ==1 &]\[CircleMinus]"sub"\[CircleMinus]"name"

(This really does look better in the notebook)

Example of code format

Test that the previous syntax still works:

data\[CircleMinus]"region"
data\[CircleMinus]"systems"[2]\[CircleMinus]"name"
data\[CircleMinus]"systems"[2]\[CircleMinus]"sub"[2]

Note that this is still note well tested. For example I'm not sure why I need If [Length@s == 1 && Head[s[[1]] ] === List, s[[1]], s]. But I got some redundant {{}} and this is hacky way to remove them.

With some more effort it's also possible to override Dot

Attributes[withDotPath] = HoldFirst;
withDotPath[code_] := Internal`InheritedBlock[{Dot}, Unprotect[Dot];
   Dot[sys_, sub_] := sub /. sys;
   Dot[arg1_, arg2_, arg3_] := Dot[Dot[arg1, arg2], arg3];
   Dot[arg1_, arg2_, args__] := Dot[Dot[arg1, arg2], args];
   Dot[sys_, sub_[n_]] := (sub /. sys)[[n]];
   Dot[sys_, sub_[f_Function]] := Module[{s},
     s = Select[(sub /. sys), f]; (*remove redundant {}*)
     s /. {x_List} :> x];
   code
   ];

Then one can use the following beautiful syntax:

withDotPath[Column@{
   data."region",
   data."systems"[2]."name",
   data."systems"[2]."sub"[2],
   data."systems"[#."name" >= 1 &]."sub"[2]."name",
   data."systems"[#."name" == 1 &]."sub"[1]."name",
   data."systems"[#."name" == 1 &]."sub"."name"
   }]

Any comments or further improvements are most welcome.

Ajasja
  • 13,634
  • 2
  • 46
  • 104
  • Nice work! I think the {{}} is because Select always returns a list of results, even if there is only one of them. A neater (IMO) way to strip the excess brackets is s/.{x_List}:>x. – Simon Woods Jul 17 '12 at 11:04
  • @SimonWoods Thanks! I included your suggestion, but kept the code a bit more verbose. The shortest way would be. Dot[sys_, sub_[f_Function]] := Select[(sub /. sys), f] /. {x_List} :> x; – Ajasja Jul 17 '12 at 11:19