0

So I hit a minor snag in testing the users part of an API connection to SE. It seems Mathematica can't always decode the GZIP?

For instance this:

ImportString[
 FromCharacterCode[
  URLFetch["https://api.stackexchange.com/2.2/users?site=mathematica.\
stackexchange.com&pagesize=100&page=5",
   "ContentData"
   ]
  ],
 "GZIP"
 ]

Throws:

Import::infer: Cannot infer format of file InputStream[Name: String, Unique ID: 3].

On the other hand this:

Import["https://api.stackexchange.com/2.2/users?site=mathematica.\
stackexchange.com&pagesize=100&page=5",
 "Text"
 ]

Seems to work totally fine.

How can I import the string the same way Import does?

Update:

Another wrinkle. If I import the bytes:

cdata = URLFetch[
   "https://api.stackexchange.com/2.2/users?site=mathematica.\
stackexchange.com&pagesize=100&page=5",
   "ContentData"
   ];

Then dump them to a file:

$setempfile = CreateFile[];
BinaryWrite[$setempfile, cdata];

I can import that file as bytes, but it's different from what I wrote:

In[196]:= cdata == Import[$setempfile, "Byte"]

Out[196]= False

And what is now read-in can be decoded properly:

In[197]:= StringTake[FromCharacterCode@Import[$setempfile, "Byte"], 50]

Out[197]= "{\"items\":[{\"badge_counts\":{\"bronze\":21,\"silver\":14"

So I'm doubly confused, as 

Import[$setempfile, "gzip"]

Still fails.

b3m2a1
  • 46,870
  • 3
  • 92
  • 239
  • You have to respect the header, partially :), that is to take content type and ignore encoding: "Content-Type", "application/json; charset=utf-8", encoding when enforced gives failures, don't know why. So, is this what you want? 145769, {"GZIP", "RawJSON"}? – Kuba Jun 14 '17 at 06:15
  • @Kuba seems to work. I'd assumed {"GZIP", "RawJSON"} was just reading in as "GZIP" then applying the "RawJSON". I guess not. {"GZIP", "Text"} is what I actually need for various reasons, so I'll be going with that. Thanks. – b3m2a1 Jun 14 '17 at 06:19
  • By default it would be but manually you have to do that yourself. p.s. do you understand why content-type encoding does not matter? – Kuba Jun 14 '17 at 06:20
  • @Kuba I do not. I did see that you were having troubles with an explicit CharacterEncoding in your FromCharacterCode. I'm guessing that has to be applied at the time we decode the "Text" after converting the "GZIP". E.g. ImportString[FromCharacterCode@cdata,{"GZIP", "Text"}, CharacterEncoding -> "UTF8"] – b3m2a1 Jun 14 '17 at 06:25

0 Answers0