How to maintain a large LibraryLink project?

Question

I have a LibraryLink project, which became more and more larger(about 50 LibraryLink wrapper functions, 8000 lines ANCI C code). Curently, I just use a simple *.c file to organize it as follows:

 #include " WolframLibrary.h"
 /*Part 0 including head files*/
 #include <math.h>
 #include <stdlib.h>
 #include <xxx.h>
 /*Part 1: functions declaration*/
 int func1(int a, int b);
 xxx();
 /*Part 2: LibraryLink wraper functions*/
 DLLEXPORT int func1(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res){}
 DLLEXPORT int func2(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res){}

/*Part 3: auxiliary function for LibraryLink functions*/
int func1(int a, int b){}
void func2(int a, int b){}

when I need to compile it, I used the following method:

Copy the total code to Mathematica notebook with src = all_code_of_C_file, and compile it with CreateLibrary[src, "lib_link"]

Obviously, it is not easy to maintain and organize this project via the above method.

Welcome to Mathematica.SE! 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Take the tour and check the faqs! 3) When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign! — , Oct 27 '16 at 05:14
Why are you copying the C code into the notebook? It was never meant to be done this way. Keep it in a file and use CreateLibrary[{"file.c"}, ...]. Writing code into a string is just for quick and dirty experimentation and doesn't support all features, e.g. doesn't allow C++. In fact you don't even need CreateLibrary. Use the option "ShellCommandFunction" -> Print in CreateLibrary to see the compilation command line it uses. If you want you can now put it in a makefile, or use any build system. — Szabolcs, Oct 27 '16 at 06:33
If you know some C++, you may also want to take a look at my package LTemplate. It is meant to make it quicker to write LibraryLink code by generating the boilerplate code. I estimate that in a project where I have mostly wrapper functions for a library, it cuts down the amount of code I need to write by 60-80%. However, in that project I have 162 functions. If you have 8000 lines for only 50 functions, I wonder if those are really just wrapper functions ... — Szabolcs, Oct 27 '16 at 06:47
@Szabolcs Thanks. I just refer to the Wolfram Documentation. Could you give a demo to show how to build this in a makefile or use some building system? — user123, Oct 27 '16 at 06:48
Also, if you are already so far along in your project (8000 lines), it may not be wroth rewriting it using LTemplate. Designing it using LTemplate from the beginning would have been different. If you will consider it, also keep in mind that it doesn't support all LibraryLink datatypes yet. In particular, it doesn't support MImage yet and requires fixing the underlying datatype of tensors: i.e. {Real, _, "Constant"} is allowed for arbitrary rank real tensors, but {_, _, "Constant"} isn't for arbitrary rank arbitrary type tensors. — Szabolcs, Oct 27 '16 at 06:50
@Szabolcs The total code consists of four parts. (1)Part 0 including head files. (2)Part 1: functions declaration (3)Part 2: LibraryLink wraper functions(3) main implementaions/auxiliary function for LibraryLink functions — user123, Oct 27 '16 at 06:51
@user123 The documentation says, "CreateLibrary[{file1, ...}, name] compiles a number of C source files into a library." Why don't you try this first? Personally I don't find makefiles convenient with LibraryLink because I have to deal with differences between operating systems and compilers myself. But if you really want a makefile, use CreateLibrary[..., "ShellCommandFunction" -> Print] to see the command Mathematica uses to compile your library. You can run this command yourself without Mathematica. — Szabolcs, Oct 27 '16 at 06:53
@Szabolcs In my project, I mainly use the MTensor data structure with the help of the built-in ArrarPad. For instenace, I have some ragged vectors {1,2,3,4}, {11,22,33},{111,222,333,444,555} to pass, I would use tensor = {{1,2,3,4,0}, {11,22,33,0,0},{111,222,333,444,555}} and corresponding length len = {4,3,5}. — user123, Oct 27 '16 at 06:57
@Szabolcs Thanks very much. Maybe I should separate my big *.file to some independent files aa.c bb.c ee.c,..., then applying CreateLibrary[{aa.c,bb.c,ee, ...}, name] to generate a library. — user123, Oct 27 '16 at 07:03

score 9 · Accepted Answer · edited Jun 16 '20 at 09:23

when I need to compile it, I used the following method:

Copy the total code to Mathematica notebook with src = all_code_of_C_file, and compile it with CreateLibrary[src, "lib_link"]

This is not how CreateLibrary is meant to be used. You would only type the C code in a Mathematica string if it is so short and so simple that you can't be bothered to put it in a file.

If you already have the code in one or more files, as you should for any real project (and not just quick an dirty experiment), then use the second syntax. From the documentation:

CreateLibrary[{file1, ...}, name] compiles a number of C source files into a library.

In fact putting the code in a string doesn't even support all features, e.g. it doesn't allow using C++ instead of C. Always put the code in a file, and compile the file.

I recommend that you skim through the LibraryLink and CCompileDriver tutorials before starting to work with LibraryLink. In particular, look at the section describing how to compile your library. Make sure you look at some of the basic examples that come with Mathematica and compile them yourself. The example source file are here:

SystemOpen@FileNameJoin[{$InstallationDirectory, "SystemFiles", "Links", "LibraryLink", "LibraryResources", "Source"}]

There's also a beginners tutorial here:

http://community.wolfram.com/groups/-/m/t/189735/

The first lesson shows how to compile.

If you are already comfortable with LibraryLink, and you are creating large projects, take a look at the LTemplate package. Its purpose is to reduce the amount of boilerplate/wrapper code you need to write. It automatically generates this code based on a template.

How to maintain a large LibraryLink project?

1 Answers1