How to convert a RawArray of bytes to ByteArray?

Question

A ByteArray can be converted to a RawArray:

ba = ByteArray[{1, 2, 3, 4}]
(* ByteArray[{1, 2, 3, 4}] *)

ra = RawArray["Byte", ba]
(* RawArray["UnsignedInteger8",{1, 2, 3, 4}] *)

Normal[ra]
(* {1, 2, 3, 4} *)

Presumably, this is done without unpacking the ByteArray contents to 64-bit integers first (although I have no proof of this).

Is there a similar space-efficient way to convert a rank-1 byte-type RawArray to a ByteArray?

The CURLLink package reveals that "ByteArray" is a valid type specification for LibraryFunctionLoad. If anyone is willing to investigate this, it may lead to an answer. — Szabolcs, Nov 11 '17 at 13:31

ilian · Accepted Answer · 2017-11-15T17:01:01.617

At present there is no way to do this from top level, since there is no interface to the internal function that converts a rank-1 RawArray of bytes into a ByteArray.

However, as already pointed out, it's possible to take advantage of LibraryLink which will do a conversion given an MRawArray on the C side and a "ByteArray" return type, for example

Needs["CCompilerDriver`"]

src = "
  #include \"WolframLibrary.h\"
  #include \"WolframRawArrayLibrary.h\"

  DLLEXPORT mint WolframLibrary_getVersion() {
    return WolframLibraryVersion;
  }

  DLLEXPORT int WolframLibrary_initialize( WolframLibraryData libData) {
    return 0;
  }

  DLLEXPORT void WolframLibrary_uninitialize( WolframLibraryData libData) {
    return;
  }

  DLLEXPORT int mrawarray_to_bytearray( WolframLibraryData libData, 
    mint Argc, MArgument *Args, MArgument Res) {

    WolframRawArrayLibrary_Functions rawFuns = libData->rawarrayLibraryFunctions;
    MRawArray ra;
    mint rank;
    rawarray_t type;

    if (Argc != 1) return LIBRARY_FUNCTION_ERROR;
    ra = MArgument_getMRawArray(Args[0]);

    type = rawFuns->MRawArray_getType(ra);
    rank = rawFuns->MRawArray_getRank(ra);

    if (rank == 1 && type == MRawArray_Type_Ubit8) {
      MArgument_setMRawArray(Res, ra);
      return LIBRARY_NO_ERROR;
    } 
    return LIBRARY_FUNCTION_ERROR;

  }";

lib = CreateLibrary[src, "toByteArray"];

toByteArray = LibraryFunctionLoad[lib, 
  "mrawarray_to_bytearray", {"RawArray"}, "ByteArray"];

toByteArray[RawArray["Byte", {1, 2, 3, 4}]]

(* ByteArray["AQIDBA=="] *)

Thank you! I meant to try this, but haven't gotten to it yet, and I really didn't expect it to be as simple as returning an MRawArray as type "ByteArray". — Szabolcs, Nov 15 '17 at 16:47
I was trying to find out which is the earliest version in which this works, so I tried it in 10.4 (first version with RawArray support in LibraryLink). In that version, it only starts working after ByteArray has been used at least once. I.e. if I evaluate ByteArray[{0}] before I try toByteArray, everything is fine. If I don't the first evaluation of toByteArray[RawArray["Byte", {1, 2, 3, 4}]] returns ByteArray[""] (from the second onwards, everything is fine). — Szabolcs, Nov 15 '17 at 16:57
Well, it's undocumented, so I wasn't expecting much :-) I also noticed that trying to pass a ByteArray argument using "Constant" passing crashes the kernel. — Szabolcs, Nov 15 '17 at 17:29

score 7 · Answer 2 · answered Apr 17 '19 at 01:53

The new in M12 function NumericArray is essentially the documented version of the undocumented RawArray. The conversions between ByteArray and NumericArray are straightforward:

ba = ByteArray[{1,2,3,4}];
ba //InputForm

ByteArray["AQIDBA=="]

Conversion to a NumericArray:

na = NumericArray[ba, "Byte"];
na //InputForm

NumericArray[{1, 2, 3, 4}, "UnsignedInteger8"]

Conversion back to a ByteArray:

ba === ByteArray @ na

True

Note that your RawArray syntax is still supported, but it now constructs a NumericArray object instead:

RawArray["Byte", ba] //InputForm

NumericArray[{1, 2, 3, 4}, "UnsignedInteger8"]

How to convert a RawArray of bytes to ByteArray?

2 Answers2

Linked