MTensor from existing memory?

Question

I am wondering whether it's possible to return a tensor from my C++ code using LibraryLink without copying the memory (because it's large). So, instead of

libData->MTensor_new(..., &T0);
for(i...) libData->MTensor_setXXX(T0, ...);
MArgument_setMTensor(res, T0);

I would like to create the tensor from a block of memory,

libData->MTensor_new(..., buf, &T0);
// no copying needed
MArgument_setMTensor(res, T0);

Is this possible, and if yes, how?

PS: This would require Mathematica to release the memory eventually. In other binding frameworks, such as pybind11, this is achieved py passing a corresponding deleter function.

PPS: This would also require some information about internal layout of tensors in Mathematica. Again, no problem in the Python setting (with numpy) as one can specify strides. It also seems unlikely that Mathematica's internal tensor layout will change given the amount of collateral work that would cause.

PPPS: There is a related question from 2013. Perhaps something changed since then?

Answer 1

I do not believe that this is possible.

What you can do is create a new MTensor with the desired amount of storage, and use that pre-allocated storage for the data structures of other libraries (if those other libraries allow this).

I have done something similar with the Armadillo library, which has the capability to wrap existing storage with its matrix type. A slight problem when using Armadillo with Mathematica is that it uses column-major storage, while Mathematica uses row-major storage. Thus when wrapping an MTensor's data with Armadillo, you will get the transpose of the matrix. I included some examples of this in LTemplate.