Misc:MatlabOpenMP
From OpenRAVE
by Rosen Diankov
OpenMP can offer a big speed boost to most programs when run on multi-core computers. This tutorial will show how to compile internal C++ routines using OpenMP as a mex file so that they can be called from Octave or Matlab. The goal is to load all the correct libraries and files without needing root/admin access or having to move default shared object files.
Contents |
GCC/G++
First, OpenMP is only supported on gcc versions 4.2 and higher, so make sure to get it.
The biggest difficulty is getting the correct libgomp.so shared object (implementing OpenMP) for gcc. Some official libgomp.so files are generated with the nodlopen flag or with a thread local storage (TLS) model that doesn't allow for libgomp.so to be statically linked. This means that it is impossible to create plugins/mex files in the form of shared objects that use libgomp.so without compiling your own version.
Compiling libgomp.so
You will need to recompile your own libgomp.so if you get any of the following errors when trying to use the mex file:
- shared object cannot be dlopen()ed
- cannot allocate memory in static TLS block
First get the source code for the gcc you are using. After decompressing, there should be a libgomp folder.
Go into it and open the configure.tgt file. Comment out these lines:
# if test $have_tls = yes ; then # case "${target}" in # # *-*-linux*) # XCFLAGS="${XCFLAGS} -ftls-model=initial-exec" # XLDFLAGS="${XLDFLAGS} -Wl,-z,nodlopen" # ;; # esac # fi
The most important part is to make sure -ftls-model=intitial-exec and -Wl,-z,nodlopen are not executed. Then inside the libgomp folder type:
autoconf
Go back to the root gcc folder and compile only libgomp:
./configure make configure-target-libgomp
The configure script should create a folder named after your computer's architecture where it will store all its object files. On x86 linux this is called i686-pc-linux-gnu. Inside should be a libgomp folder. Go into it and type
cd $ARCH/libgomp make
This will generate two very important files: omp.h and .libs/libgomp.so.x.x.x. To be on the safe side, it is good to check whether this omp.h file is similar to the one the default gcc installation uses. Locate the default omp.h and diff it with the one just generated. If the two files are very different, you might have to update your entire gcc.
Copy these two files into a local directory, which will be referred to as $(LIBGOMP) from now.
Mex compilation
It is best to compile all OpenMP code with gcc into object files rather than directly mex'ing them for two reasons.
- Directly mex'ing the source files or inserting their .o equivalent might cause libgomp.so to fail with a "cannot allocate memory in static TLS block" error" when loading the mex file.
- There is more control over the optimization and compilation options.
The solution is to split your code into one file that contains the mexFunction entry point (let us call it mymex.cpp), and the rest of your code (let us call that myopenmp.cpp). We will first compile myopenmp.cpp with gcc into a shared object called libmyopenmp.so. Second, we will compile mymex.cpp with mex and dynamically link it with libmyopenmp.so.
# create the object file gcc -c myopenmp.o -fopenmp $(OTHERCXXFLAGS) myopenmp.cpp # create the shared object gcc -shared -Wl,-soname,libmyopenmp.so $(OTHERCXXFLAGS) -L$(LIBGOMP) $(OTHERLIBS) -o libmyopenmp.so myopenmp.o # create the Octave mex file mkoctfile --mex $(OCTCXXFLAGS) -L$(LIBMYOPENMP) -Wl,-rpath,$(LIBMYOPENMP) -lmyopenmp mymex.cpp # create the Matlab mex file mex $(MATCXXFLAGS) -L$(LIBMYOPENMP) -Wl,-rpath,$(LIBMYOPENMP) -lmyopenmp mymex.cpp
The -Wl,-rpath,$(mydir) tells the final mex file to first look for shared objects in the $(mydir) directory. If you have built your own local copy of libgomp.so, then you have to add another -rpath,$(LIBGOMP) command, or make sure it is in the same dirctory as libmyopenmp.so. This saves the hassle of changing $LD_LIBRARY_PATH. When first loading the mymex mex file, it will look for libmyopenmp.so and libgomp.so. You have to make sure these libraries are in the directory specified in rpath, or in a globally known library directory, or can be found with $LD_LIBRARY_PATH.
Issues with Matlab
Getting everything to work on Matlab is hard for several reasons: depending on the matlab version, it either won't like files compiled with gcc-4.2 (or higher) or uses outdated OpenMP implmentations that can confuse and slow down your generated mex file. Keep in mind that once Matlab is forced to load newer libraries (especially a newer libgomp), there is a chance it will start crashing randomly.
For Linux, when compiling a matlab mex file, you might get a message saying the gcc version is too high. If so, matlab will have a hard time locating the correct libstdc++.so file. In this case, go into /usr/local/share/sys/os/glnx86
and make libgcc_s and libstdc++ point to the /usr/lib versions
mv libgcc_s.so.1 libgcc_s.so.1.back ln -s /lib/libgcc_s.so.1 libgcc_s.so.1 rm libstdc++.so.6 # (this was already a symbolic link) ln -s /usr/lib/libstdc++.so.6.0.9 libstdc++.so.6
In order to get Matlab to load the correct libgomp version, set the LD_PRELOAD environment variable before launching matlab to point to libgomp.so.
export LD_PRELOAD=$(LIBGOMP)/libgomp.so
LD_PRELOAD is used to trick Matlab into using newer shared objects than what it was installed with. This is necessary because Matlab has its own OpenMP libraries (sometimes called libguide.so), which can be old and slow. Even if the mex file works with the old OpenMP libraries, it might be slower and might randomly crash. If this is the case, then it is best to preload. However, the problem with preloading the newest OpenMP libraries is that matlab modules relying on the old library will now be using the new functions and might not like it.
Windows
good luck
