3

My Problem

Some people gave me a JSON file, which is exported from a camera. He asked me to parse the JSON file, and extract the value to each key. However, when I import the JSON file with:

Import[pathToTestJSON, "JSON"]

Mathematica warns: Import::jsonexpendofinput: Unexpected character found while looking for the end of input.

Potential Solution

I searched the mathematica.Stackexchange.com, and found this post: Openning JSON file, (NDJSON, new line delimited) — big file where @Jason B. explained:

Import[myfile.ndjson,"JSON"] doesn't work because the file as a whole is not valid JSON. But each line is valid JSON

But his solution is not for my situation, because my JSON record is not line-by-line, instead, its layout is a block. The following is a sample:

{
    "ChannelInfo" : null,
    "DevNo" : "DS-2CD2T86FWDV2-I8S20191216AACH",
    "MecNo" : "100",
    "MsgType" : 1000,
    "Timestamp" : 1606910698042
}

My Puzzle

(1) To take advantage of Openning JSON file, (NDJSON, new line delimited) — big file, how can I extract the JSON record block, instead of a line, and then parse the JSON with Import[jsonRecordInBlock, "JSON"]?

(2) If we do not copy the solution from Openning JSON file, (NDJSON, new line delimited) — big file, what is the solution?

Attachment

JSON file for test

PureLine
  • 1,310
  • 7
  • 20
  • 3
    Your JSON file is invalid according to the JSON specification. You need to Import it as text and surround the string with braces { ... } then use ImportString[text, "JSON"] . You'd also need to add commas after each object delimiter. – flinty Dec 08 '20 at 13:14

1 Answers1

5
(* Import as text first *)
text = Import["test.json", "Text"];

(* Split out the lines *)
lines = StringSplit[text, "\n"];

(* Replace lines containing ONLY } with }, 
 * then stitch the lines back together with newlines and remove the final comma.
 * Finally, surround all objects with a JSON list [...], not braces. *)
newjson = "[\n" <> StringDrop[StringJoin[lines /. "}" -> "},", "\n"], -2] <> "]\n"

(* Import the fixed string and decode as JSON *)
result = ImportString[newjson, "JSON"]
flinty
  • 25,147
  • 2
  • 20
  • 86
  • By the way, this assumes all your top-level objects in the json are delimited by START OF LINE then } then NEWLINE. If you have any stray whitespace after a } then you may need to trim it. It will not work if it looks like START OF LINE whitespaces+ } as it becomes indistinguishable from the other nested delimiters. – flinty Dec 08 '20 at 14:07
  • Your comment above should be part of the answer. – Anton Antonov Dec 08 '20 at 14:52
  • Thanks for your solution. Anyway, I revised part of your code for the output readability as following: mynewjson = "[\n" <> StringDrop[StringRiffle[lines /. "}" -> "},", "\n"], -1] <> "]\n"; – PureLine Dec 09 '20 at 06:45