Import nested JSON file

Question

For a project where I receive JSON messages from a web server I was trying to convert these messages into a format that can be used easily within Mathematica. Because JSON also consists of key-value pairs I thought it would be logical to convert it to an Association. For a JSON file that is not nested this is easy:

Association[Import["example.json","JSON"]]

...but for more complex JSON files this does not work.

Question: Is it possible to convert a nested JSON file to a nested Association? Or is there a better way to handle JSON data in Mathematica?

Example JSON file from json.org.

I think the linked topic answers your question since imported json is a nested list/rule expression, right? — Kuba, May 26 '15 at 11:57
I think you are right. I did not realize the imported JSON file actually is a nested list of rules. Since I am probably not the only one not realizing that, this question might help others doing so. Is it an idea if you post the link to the related question as an answer? — Aart Goossens, May 26 '15 at 12:25
Topic marked as a duplicate will stay as a link to the main topic. — Kuba, May 26 '15 at 12:27
IMHO, the best way to do it is 79393. Needs["GeneralUtilities`"]; ToAssociations[list] — Murta, May 26 '15 at 13:24
is there a chance to make this not a duplicate anymore? I think this question differs from the "duplicate" as it asks specifically about JSON format, and I just added a note about RawJSON import to my answer which only addresses the import JSON case and would not make sense as an answer to the "duplicate"... — Albert Retey, May 06 '16 at 17:37

Albert Retey · Answer 1 · 2016-05-06T17:33:27.873

If you know what the JSON you receive looks like your own approach might work well enough. On the other hand JSON might not necessarily contain just nested "Objects" but also arrays, in which case it will cause errors or invalid unevaluated Assocation-expressions if one assumes that every list resulting from the imported JSON is a list of rules (or even has depth 2). So I'd recommend to use something more robust, which in this case is even shorter:

ImportString[json, "JSON"] //. lst : {__Rule} :> Association[lst]

it will only convert those parts which in fact are list of rules, while it lets alone other arrays. This will also work for a JSON string like:

{"object":{"length":5,"width":2},"string":"something","list":["a","b","c"],"array":[[1,2],[3,4]]}

Using pattern matching for such tasks is very common, powerful and useful approach when using Mathematica, so it is usually a good approach for a first shot. The only drawback is that such code might not be very efficient for really huge datasets. It probably is also worth noting that this only does work because pattern matching does work within associations: the first match is the outermost list of rules, if associations would be opaque to the pattern matching, the inner list of rules would then not be seen by ReplaceRepeated.

EDIT: current versions (>= 10.2) do also offer to import as "RawJSON" which does what the OP wants directly. As the documentation states: "RawJSON identifies JSON objects with Wolfram Language associations." It is not clear to me if and how importing as "RawJSON" will have any other effects than returning associations instead of lists of rules. Of course I would guess that that is the most efficient way to get associations...

Thanks Albert Retey. You are right about arrays causing errors, I did not think about that. Your answer also looks nicer! If I would update my solution to work with arrays, would that make sense performance-wise or is your code still faster with larger files? — Aart Goossens, May 26 '15 at 12:32
Honestly I don't know if you will/can be faster with a more "low level" approach with large files. As applying replacements rules with the pattern matcher is at the very core of Mathematica it is implemented in a quite efficient way, but it is of course an expensive operation, so you might be able to outperform it. I'd go with the most simple approach (pattern matching) and only try to improve when I had an example where it clearly seems necessary. That would then make up a nice extra question, IMO. — Albert Retey, May 26 '15 at 12:56

score 1 · Answer 2 · answered May 26 '15 at 11:52

Because I actually had this problem but eventually solved it I thought I'd question and answer it myself. Feel free to submit a better solution!

First, import the data using:

json = Import["example.json","JSON"]

The function that does all the magic is:

JSONToAssociation[list_] :=
 Association[
  Table[
   If[
    ListQ[i[[2]]], i[[1]] -> JSONToAssociation[i[[2]]], i],
   {i, list}
   ]
  ]

Let me try to explain what this function does: The function traverses the imported JSON file using Table and checks for sublists. If a sublist is found, the function calls itself (recursive) and than repeats the same check untill no more sublists are found. It than converts all (sub)lists to Associations.

Apply JSONToAssociation on the data:

jsonAssociation = JSONToAssociation[json]

You can then access your data for example like:

jsonAssociation["widget","text","name"]
Output: "text1"

This solution performs well on relatively small and non-nested files, but I think there might be performance issues when using (very) large nested JSON files.

Import nested JSON file

2 Answers2

Linked