Converting two 8-bit integers into a single 16-bit integer

Question

I have a situation where I have a pair of integers {a, b} where a and b are in bytes from a binary file. I want to combine these two 8-bit representations to a single 16-bit one. I am struggling to see how to do this.

Looking on the web suggests performing a BitShiftLeft[a,8] by 8 bits on the most significant byte (a) and then doing a bit-wise operation to 'inject' b. I am finding it tricky to get my head around what I need to do in Mathematica, so any help would be much appreciated!

As suggested by people in the comments, here is a simple example: {1,0} as given by BinaryReadList of my data file, is represented as two 8 bit values as {00000001,00000000}. Therefore I would expect the following result of converting to a single 16bit value {0000000100000000} or 256 in decimal.

Note my original binary file is several hundreds of megabytes.

Answer 1

Because MMA handles floats with arbitrary precision, combining bytes into Real32 or Real64 can get messy. However, with integers things can be much more straightforward with ByteArray:

ba = {1, 0};
ImportByteArray[ByteArray[ba], "Integer16", ByteOrdering -> 1]
{256}

You'll need to change the byte ordering/endianness (1 or -1) to suit your needs.

If you are importing a large binary file, look at BinaryRead or BinaryReadList and use the same arguments as above to import successive Integer16s.

Answer 2

Since there are multiple solutions to the problem, I wanted to compare their relative performance because the source data is hundreds of megabytes. @Richard does not state whether the resulting 16-bit value is signed or unsigned. I have assumed unsigned, as did @creidhne and @josh in their answers and @Daniel Lichtblau in a comment. I also assumed the data file was effectively stored as BigEndian. I adapted @creidhne's example as follows.

Setup by creating a timing function and a 2 MB test file of 1,000,000 byte pairs. For the timing function I chose RepeatedTiming with a 5 second evaluation period. After many trial runs with different periods, I determined that 5 seconds was sufficient to provide repeatedly consistent results.

ClearAll[myRepeatedTiming]
SetAttributes[myRepeatedTiming, HoldFirst];
myRepeatedTiming[exp_] := RepeatedTiming[exp, 5]
file = FileNameJoin[{$TemporaryDirectory, "8,16-bit-compare"}];
BinaryWrite[file, RandomInteger[{0, 65535}, 1000000], 
  "UnsignedInteger16"];
Close[file];

Then time each of four different methods on processing the test data file:

1. From the answer by @creidhne: Read list directly as 16-bit unsigned integers.
2. From the comment by @Daniel Lichtblau: Read list as 8-bit unsigned integers. Process in pairs, multiplying high byte by 256 and adding low byte.
3. From the answer by @josh: Read list as 8-bit unsigned integers. Process in pairs, converting each byte to a list of binary digits, joining the lists—high byte first—and converting to integer.
4. As suggested in original post by @Richard: Read list as 8-bit unsigned integers. Process in pairs, left shifting high byte by 8 bits and adding low byte.

I then ran timings for each and displayed the results as average run time for the method and its ratio to the timing of the fastest. The display also compares results from the four methods to show they are identical.

{time16, result16} =
  myRepeatedTiming[
   BinaryReadList[file, "UnsignedInteger16", ByteOrdering -> 1]];
{timeMultiply, resultMultiply} =
  myRepeatedTiming[
   BlockMap[256*First@# + Last@# &, BinaryReadList[file], 2]];
{timeBits, resultBits} =
  myRepeatedTiming[
   BlockMap[
    FromDigits[
      Join[IntegerDigits[First@#, 2], IntegerDigits[Last@#, 2, 8]], 
      2] &, BinaryReadList[file], 2]];
{timeShift, resultShift} =
  myRepeatedTiming[
   BlockMap[BitShiftLeft[First@#, 8] + Last@# &, BinaryReadList[file],
     2]];
Grid[{
  {, Item["Time\n(sec)", Alignment -> Center], 
   Item[" Time\nratios", Alignment -> Center]}
  , {"16-bit:", time16, 1}
  , {"8-bit multiply:", timeMultiply, timeMultiply/time16}
  , {"IntegerDigits:", timeBits, timeBits/time16}
  , {"8-bit shift:", timeShift, timeShift/time16}
  , {}
  , {"All equal?", 
   result16 == resultShift == resultMultiply == resultBits}
  }
 , Alignment -> {{Right, Left, "."}}
 , Spacings -> 2
 ]
DeleteFile[file];

Timing results were:

                    Time      Time
                    (sec)    ratios
16-bit:           0.0045457    1
8-bit multiply:   0.201478    44.3227
IntegerDigits:    0.608403   133.841
8-bit shift:      0.611216   134.46
All equal?        True

It is unsurprising that the direct 16-bit integer file read was the fastest, because it requires no post-processing. I was however quite surprised to see that the 8-bit shift was so much slower than the 8-bit multiply. I was equally surprised that the conversion to binary digits and back was no slower than the 8-bit shift.

Answer 3

To use BinaryReadList, let's assume the 8-bit values in your file are in this order: 00000001, 00000000, that is, the most significant value is first in the file.

For testing, make a list of 16-bit integer test values, and convert 16-bit values to pairs of 8-bit integers. Make a file of the 8-bit integer values.

SeedRandom[1];
test = Join[{256}, RandomInteger[{0,65535},8] ,{65535}]
bigEndianBytes = QuotientRemainder[#, 256] &@ test(* most significant 8-bit integer first *)
file = FileNameJoin[{$TemporaryDirectory, "test"}];
test (16-bit integers)
(* {256, 53568, 28680, 24829, 16840, 24558, 7953, 13002, 7776, 65535} *)

Write a test file. Use BinaryReadList to read the file as unsigned 16-bit integers.

BinaryWrite[file, Flatten@bigEndianBytes];
Close[file];
BinaryReadList[file, "UnsignedInteger16", ByteOrdering -> 1]
(* {256, 53568, 28680, 24829, 16840, 24558, 7953, 13002, 7776, 65535} *)

If the file is ordered as least-significant 8-bit value first, then change ByteOrdering -> -1.

littleEndianBytes = Reverse/@bigEndianBytes;(* least significant 8-bit integer first *)
BinaryWrite[file, Flatten@littleEndianBytes];
Close[file];
BinaryReadList[file, "UnsignedInteger16", ByteOrdering -> -1]
(* {256, 53568, 28680, 24829, 16840, 24558, 7953, 13002, 7776, 65535} *)

Remove the temporary file.

DeleteFile[file]

Answer 4

Here's an idea using num1 as the highest order bits. Try running it and see if that's what you want:

num1 = 47;
num2 = 120;
id1 = IntegerDigits[num1, 2]
id2 = IntegerDigits[num2, 2]
newBinary = Join[id1, id2]
newNum = FromDigits[Join[id1, id2], 2]
IntegerDigits[newNum, 2]