How to convert data types in a LibraryLink wrapper

Question

I have a dynamic library which I want to load into Mathematica using LibraryLink. I don't have access to the source code of the library, so I need a C wrapper to do the job. In the C wrapper, I would recieve arguments from Mathematica using the MArgument_get* functions and call my library function to do the computation. Then, the results would be sent back to Mathematica using the MArgument_set* functions.

Since the data type I receive from Mathematica is a Wolfram Library type such as mint, double, MTensor, etc., and the data types my external library function expects to receive are standard C types like int or float, my question is: how do we convert between these two different data types in the wrapper so that Mathematica and my external library functions can understand each other?

Update:

Here is an very simple example, following example here.

I have defined a external function which add two vectors, in fortran:

!addvec.f90

subroutine addvec(a,b)
implicit none
integer,parameter::N=3   
integer a(N),b(N)
a(:)=a(:)+b(:)
return
end subroutine addvec

The C wrapper is

//wrapper.cc

#include "WolframLibrary.h"
#include "WolframCompileLibrary.h"

DLLEXPORT mint WolframLibrary_getVersion(){
  return WolframLibraryVersion;
}
DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData){
  return 0;
}

extern "C" {
  void addvec_(int a[], int b[]);
} 

EXTERN_C DLLEXPORT int addvec(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res){

  MTensor ta;
  MTensor tb;

  ta=MArgument_getMTensor(Args[0]);
  tb=MArgument_getMTensor(Args[1]);

  addvec_((int *)((*ta).data),(int *)((*tb).data));

  MArgument_setMTensor(Res,ta);
  return LIBRARY_NO_ERROR;  
}

Mathematica code

Needs["CCompilerDriver`"]
CreateLibrary[{"wrapper.cc", "addvec.o"}, "myadd", "Debug" -> True, "TargetDirectory" -> "."]
addvec = LibraryFunctionLoad["./myadd", "addvec", {{Integer, 1}, {Integer, 1}}, {Integer, 1}]
addvec[{1, 2, 3}, {4, 5, 6}]

The Mathematica Kernel will crash soon after execute the third line.

Update 2

In Mathematica version 9 the above code seems partially work:

addvec[{1, 2, 3}, {4, 5, 6}]
(*{5, 7, 3}*)
addvec[{1, 2, 3}, {4, 5, 6}]
(*{6, 8, 4}*)

The third number in the list is calculated wrong.

The third number is calculated incorrectly because your integers in Fortran are probably only 4 bytes. Change them to 8 byte integers. — bdforbes, Dec 29 '14 at 04:15

user0501 · Answer 1 · 2013-04-06T06:06:42.433

There are a lot of def functions in the header file "WolframCompileLibrary.h", which make the type conversion very easy and straight forward. I'm using Mathematica version 8.

For example, the following functions can be used to get data from the MTensor variables, they will return pointers to the basic m types such as mint, mreal, or mcomplex from the MTensor variables. After get the basic m types, you can directly use them as the C standard types, however corresponding size must be carefully considered.

MTensor_getDimensionsMacro(mt)          //return mint* dimension
MTensor_getIntegerDataMacro(mt)         //return mint* data
MTensor_getRealDataMacro(mt)            //return mreal* data
MTensor_getComplexDataMacro(mt)         //return mcomplex* data

Here is a simple example modified from the question:

External Subroutine

!addvec.f90

subroutine addVec(a,b,n)
implicit none
integer n
integer,dimension(n)::a,b

a(:)=a(:)+b(:)

return
end subroutine addVec

C Wrapper

//MMA.cc

#include "WolframLibrary.h"
#include "WolframCompileLibrary.h" 

DLLEXPORT mint WolframLibrary_getVersion(){
  return WolframLibraryVersion;
}
DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData){
  return 0;
}
extern "C" {
  void addvec_(mint a[], mint b[], mint* n);
} 


EXTERN_C DLLEXPORT int addvec(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res){

  MTensor ta;
  MTensor tb;

  ta=MArgument_getMTensor(Args[0]);
  tb=MArgument_getMTensor(Args[1]); 
  addvec_(MTensor_getIntegerDataMacro(ta),MTensor_getIntegerDataMacro(tb),MTensor_getDimensionsMacro(ta));

  MArgument_setMTensor(Res,ta);
  return LIBRARY_NO_ERROR;
}

Mathematica

Needs["CCompilerDriver`"]
CreateLibrary[{"MMA.cc", "addvec.o"}, "myadd", "Debug" -> True, "TargetDirectory" -> "."]
addVec = LibraryFunctionLoad["./myadd", "addvec", {{Integer, 1}, {Integer, 1}}, {Integer, 1}]

addVec[{1, 2, 3}, {5, 6, 7}]
addVec[{1, 2, 3, 2}, {4, 2, 3, 5}]
(*{6, 8, 10}*)
(*{5, 4, 6, 7}*)

score 4 · Accepted Answer · edited Dec 05 '13 at 18:03

4

The data types defined for LibraryLink are just simple typedefs for standard data types.

--> see WolframLibrary.h

typedef int mint; /* 32-bit architecture */
typedef long long mint; /* 64-bit architecture */

typedef double mreal;

The MArgument_setter and MArgument_getter are just simple #defines for accessing the union MArgument:

typedef union {
    mbool *boolean;
    mint *integer;
    mreal *real;
    mcomplex *cmplex;
    MTensor *tensor;
    char **utf8string;
} MArgument;

for instance MArgument_getInteger:

#define MArgument_getInteger(marg) (*((marg).integer))

You can use the LibraryLink data types as you would use the standard ones.

Since, if you call a function which expects an integer and you assign an mint as an argument it's just a typedef for that standard type (this is therefore not introducing a new type, it's just another name for exactly the same type).

The MTensor data type is somewhat more complex, but if you look at it is again just a typedef for some data, its properties and its length.

--> see WolframCompileLibrary.h

struct M_TENSOR_STRUCT
{
    void *data;
    TensorProperty properties;
    mint flattened_length;
};

TensorProperty is again a structure with definitions for the precision, its dimension etc. Luckily you don't have to deal with this structure directly because there are again #defines in order to access the MTensor structure data type.

Now let's deal with the mreal data-type.

As I've mentioned above that mreal is just a simple typedef for the standard data-type double, converting from double to float is somewhat a mystery. Why is that ok?

float and double don't store decimal places. They store binary places: float is (assuming IEEE 754) 24 significant bits and double is 53 significant bits.

So. Converting from double to float will give you the closest possible float.

The point why this is ok is, floating point numbers are assumed to be 'double' in C unless they have a suffix (F).

Hope that helps.

edited Dec 05 '13 at 18:03

Jacob Akkerboom

12,215
45
79

answered Apr 05 '13 at 08:13

Stefan

5,347
25
32

Thanks for the answer, but I still can't get it to work. Could you be more specific about the data type conversion? For example, I have a MTensor ta which contains a one dimensional array of mint, how can I convert it to a which is a int array? I tried something like a=(*ta).data but it dones't work. – xslittlegrass Apr 05 '13 at 15:02
i don't get it. is ta now a pointer to a MTensor struct or not? what is wrong with a = ta.data? (as i said before, there is no type conversion needed. the types are identical.) – Stefan Apr 05 '13 at 16:07

when I do this, I get compile error:

MMA.cc: In function 'int add(st_WolframLibraryData*, mint, MArgument*, MArgument)':
MMA.cc:35: error: invalid use of incomplete type 'struct M_TENSOR_STRUCT'
WolframLibrary.h:46: error: forward declaration of 'struct M_TENSOR_STRUCT'
MMA.cc:35: error: invalid use of incomplete type 'struct M_TENSOR_STRUCT'
WolframLibrary.h:46: error: forward declaration of 'struct M_TENSOR_STRUCT'

Beside data is a void* type, is it OK to assign it to int [] type? Thanks

– xslittlegrass Apr 05 '13 at 16:20

ok. at that it'd be better you post your actual code. it's obvious that you declared your MTensor struct but did not initialized it. – Stefan Apr 05 '13 at 18:04
let's try it anyways... say you have a function. there you declare a MTensor struct type (MTensor a;) then you assign the tensor from Mma to a. a = MArgument_getMTensor(Args[0]); now you can access the data type, since it is correctly initialized.
your question concerning the void* type... void* is a pointer pointing to nothing, so if you want that pointer to point to something you have to assign this pointer to something, but you have to cast it. it's the same with e.g. malloc which returns a void* and you assign it to an int*.

int* i_ptr = (int)malloc(sizeof(int) 8);
– Stefan Apr 05 '13 at 18:11
1

The struct M_TENSOR_STRUCT error is because I forgot to include the "WolframCompileLibrary.h" file. After fix that the compilation is OK. But I still can't get the answer right. I've add the code I use to the questions. Could you have a look? Thank you very much. – xslittlegrass Apr 05 '13 at 19:25
@xslittlegrass Had no machine with Mma yesterday. Glad that someone else helped you, although i've to say, that your original question was a bit different. In the end it was all C-coding and not a type conversion problem. My advise to you is to use MathLink at the beginning, so if you do something wrong, your kernel does not crash at all and if you're really sure about what you're doing you may switch over to LibraryLink. – Stefan Apr 06 '13 at 08:47
Yes, that's a very good advice, and thank you very much for you help! – xslittlegrass Apr 07 '13 at 02:48
@Stefan, concerning your statement: "These typedefs are just there so that you can easily distinguish between code that deals with LibraryLink and your C/C++ library side"; Maybe I am reading this wrong, but the the file WolframLibrary.h varies across platforms. For some people mint will be defined as a long or something else. So this will not lead to a portable solution. See also. – Jacob Akkerboom Dec 05 '13 at 14:02
@JacobAkkerboom hmm. I don't get your comment. Did I ever mention that these types are for portability issues? But you shall expect that on a specific platform where LibraryLink is available the typedef'd mint type is of same size than the internal used type for a signed integer by the runtime library. If someone in his code relies on a signed integer to be 4 bytes always (you shouldn't do this anyways) then this someone has indeed a problem. That is why there is a stdint.h where the integer types are well defined, according the standard (C99). which LibraryLink lacks to support...alas – Stefan Dec 05 '13 at 15:59
Herm no you never mentioned that they were for portability issues. That's not my point :). I guess it seems intuitive to expect that mint will be the same size as a signed int (int), but I found this is not the case. Quoting halirutan in the linked question above: "What I would have to do is to look up the underlying type of mint, which is long on my machine here". My guess is that long is now usually being used for 64 bit integers and int for 32 bit integers, so much so that in 64 bit operating systems int may not be most efficient integer type to work with, whereas mint does satisfy this. – Jacob Akkerboom Dec 05 '13 at 16:45
So my guess is that that is what the mint typedef is for, and that it is not for distinguishing between code that deals with librarylink vs other code – Jacob Akkerboom Dec 05 '13 at 16:48
@JacobAkkerboom as long as there is no one else using mint as a typedef for a signed 4 byte value, it is absolutely distinguishable. Actually there are two common schemes for integer size representation. LP64 and LLP64 (used by Microsoft). On a LP64 system (MacOsX, Linux etc.) LP64 is used where a long is 64- bit. On a LLP64 it has the same size as a regular int (32-bit). I don't have the header file here to check that for Unix. But I can remember for Windows there are a ladder of #ifdefs using the most appropriate signed int type on that platform. – Stefan Dec 05 '13 at 17:22
but anyways. talking about portability and using system types is always kind of a headache, but due to the fact you're dealing with a real machine and not a mathematical abstraction. Again I'd advise you to use the stdint.h or cstdint header for portability, but then WRI must change their code as well to work properly...which I doubt they do. So yes. To be portable you should use some preprocessor statements in order to find out what platform you are on. – Stefan Dec 05 '13 at 17:28
to conclude this. I thought it was appropriate for the question to make this distinction. If someone uses mint as a typedef for a LP64/LLP64 signed integer...well there is trouble to come, but this wasn't the case here and I used it inaccurately as a hint, which I wouldn't if someone is asking this kind of a question on a blog which is assigned to system programming. You may delete the line if you want to, if you fear that someone may get on the wrong track, by taking my statement universally valid. – Stefan Dec 05 '13 at 17:45
Stefan I see I also wrote a bit of nonsense in: "the file WolframLibrary.h varies across platforms." I mean that the resulting size of a mint varies across platforms, sorry for being unclear, I guess it happens :). Thanks for allowing me to edit, I will just remove the sentence and then upvote :) – Jacob Akkerboom Dec 05 '13 at 17:59
@JacobAkkerboom thank you for the upvote :) cheers – Stefan Dec 05 '13 at 18:14

How to convert data types in a LibraryLink wrapper

2 Answers2

Linked