How to calculate SHA256 of a number, not string

Question

Mathematica always interprets the input of the Hash function as a string. I would expect this:

IntegerString[
 Hash[FromDigits["00010966776006953D5567439E5E39F86A0D273BEE", 16], 
  "SHA256"], 16, 64]

to give me:

"445c7a8007a93d8733188288bb320a8fe2debd2ae1b47f0f50bc10bae845c094"

as verified here: Online Hash Calculator

But instead, I get:

"c168f056a6d0238c0df71da640a0892db3f4f2dcac2477810ce7839c55dcf4a0"

I tried to solve the problem by converting the number to an array of bytes and then to a string from these bytes:

IntegerString[
 Hash[FromCharacterCode@
   IntegerDigits[
    FromDigits["00010966776006953D5567439E5E39F86A0D273BEE", 16], 
    256], "SHA256"], 16, 64]

This works in some cases, but in this particular example, it gives me:

"2d001285e2fe3db967101c75e17e0fc409d88b811e6e926a2bb3088c9ee309a2"

How can I force Mathematica to calculate the hash using the input as a number, not a string? Thanks.

Welcome! I suggest the following:

As you receive help, try to give it too, by answering questions in your area of expertise.

Take the tour and check the faqs!

When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign! — , Aug 07 '16 at 15:45
Think about using BinaryWrite to write the 21 bytes to a file, then running FileHash — george2079, Aug 07 '16 at 19:19
@george2079 BinaryWrite and FileHash do work, but ByteArray does not work because it adds another Head to the expression. — JungHwan Min, Aug 07 '16 at 22:08

JungHwan Min · Answer 1 · 2016-08-08T16:02:25.953

Observation

It seems that Mathematica's Hash interprets the input as a number.

For instance:

In[1]:= IntegerString[Hash[0, "SHA256"], 16, 64]

Out[1]= "3d2657e44444e31b63ca19355e31677556fdc59d5e9e9382140fa9953816a7af"

In[2]:= IntegerString[Hash["0", "SHA256"], 16, 64]

Out[2]= "5feceb66ffc86f38d952786c6d696c79c2dbc239dd4e91b46729d73a27fb57e9"

If we compare these hashes with the output of the string hash function of the online hash calculator:

5feceb66ffc86f38d952786c6d696c79c2dbc239dd4e91b46729d73a27fb57e9

However, if we put 0 in the binary hash, we get:

6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d

Why the difference then?

I believe the difference occurs because Mathematica likes to store values using 2^n bytes.

For instance, your number of interest is 42/2 = 21 bytes:

In[3]:= StringLength["00010966776006953D5567439E5E39F86A0D273BEE"]

Out[3]= 42

But, Mathematica stores it in 64 bytes:

In[4]:= FromDigits["00010966776006953D5567439E5E39F86A0D273BEE", 16]

Out[4]= 5918623552017386584225127869298065229098728430

In[5]:= ByteCount@5918623552017386584225127869298065229098728430

Out[5]= 64

This causes the difference.

How do I get the correct hash?

Taking @george2079 's advice, you could use BinaryWrite and FileHash (unfortunately, Hashing a ByteArray includes the head ByteArray itself, so you don't get the correct hash).

That is:

In[6]:= temp = $TemporaryDirectory <> "temp";
 BinaryWrite[temp, 
  FromDigits[#, 16] & /@ 
   StringPartition["00010966776006953D5567439E5E39F86A0D273BEE", 2]];
 Close[temp];
 IntegerString[FileHash[temp, "SHA256"], 16, 64]

Out[6]:= "445c7a8007a93d8733188288bb320a8fe2debd2ae1b47f0f50bc10bae845c094"

If you have Mathematica version prior to 10.1, instead of StringPartition, use:

StringJoin @@@ Partition[
  Characters["00010966776006953D5567439E5E39F86A0D273BEE"], 2]

Ok, but then the first result should be 6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d as in: http://www.fileformat.info/tool/hash.htm?hex=0 — Jan Hubik, Aug 07 '16 at 15:36
My number of interest is 21 bytes -- it is in hexadecimal base. Is there any way to make the Mathematica result match the one from the online hash calculator? — Jan Hubik, Aug 07 '16 at 16:39
@JanHubik My usual trick for this (not sure if correct) is to take Hash[FromCharacterCode[IntegerDigits[number, 256]], "SHA256"]. — LegionMammal978, Aug 07 '16 at 19:16
@LegionMammal978 This is what I use, but it does not work in the example I provided — Jan Hubik, Aug 07 '16 at 22:02
@JHM The version with BinaryWrite works! Thank you very much! Is it possible to do it all in RAM without actually writing the file? Apart from writing to HDD being weird, I would like to make it faster. Documentation of FileHash says something about streams... — Jan Hubik, Aug 07 '16 at 22:36

score 3 · Answer 2 · answered Aug 08 '16 at 17:19

I found the issue. In the end of my question, I proposed converting the number to a list of bytes and then to a string from these bytes using:

FromCharacterCode@
   IntegerDigits[
    FromDigits["00010966776006953D5567439E5E39F86A0D273BEE", 16], 
    256]

But my number had trailing zeros on the left, so the conversion to integer created a number which was not 21 bytes and the resulting string was wrong. It also explains why it sometimes worked. This is the correct way to do the conversion of hexadecimal number to a string suitable for hashing in Mathematica:

sha256[string_] := IntegerString[
  Hash[
   FromCharacterCode[
    FromDigits[#, 16] & /@ StringPartition[string, 2]
    ], "SHA256"
   ], 16, 64]

where string_ is a hexadecimal number. It finally gives the correct answer to my problem and it is more than three times faster than the BinaryWrite solution.

In[1]:= sha256["00010966776006953D5567439E5E39F86A0D273BEE"]

Out[1]= "445c7a8007a93d8733188288bb320a8fe2debd2ae1b47f0f50bc10bae845c094"

score 2 · Accepted Answer · answered Mar 11 '18 at 04:07

Hash has been updated in version 11.3.0 and offers a somewhat easier way to do this

hex = "00010966776006953D5567439E5E39F86A0D273BEE";

bytes = IntegerDigits[FromDigits[hex, 16], 256, StringLength[hex]/2];

Hash[ByteArray[bytes], "SHA256", "HexString"]

(* "445c7a8007a93d8733188288bb320a8fe2debd2ae1b47f0f50bc10bae845c094" *)

How to calculate SHA256 of a number, not string

3 Answers3