How to Build Microsoft Graph Engine on Red Hat Linux

Microsoft Graph Engine is a distributed in-memory data processing engine. It’s an open source project but the official instructions claim that for Linux, it only supports Ubuntu 16.04. But what if you’re using other distributions of Linux, like Red Hat? Well, it’s possible but it does take quite some effort to build the source. Let’s get started.

Prerequisites:

You need 64 G of RAM. Otherwise, you’ll run into “virtual memory exhausted” errors during the build process.
As of 12/11/2018, if you build the latest commit of the source, the output binary won’t work with Linux. I filed this issue and until it’s resolved, you should check out earlier commits.

Install required software packages:

You need git, dotnet, openssl, and c++

sudo yum upgrade -y
sudo yum install git rh-dotnet22 openssl-devel -y
sudo yum install gcc-c++ libgcc.i686 glibc-devel.i686

Build GCC

No, devtoolset won’t work due to ABI compatibility issue. You HAVE TO build GCC 5.4 or higher. This also takes the longest time (a couple of hours).

Please follow the official instruction. I chose GCC 7.4 and I only built C & C++. So the commands I used were

wget ftp://ftp.gnu.org/gnu/gcc/gcc-7.4.0/gcc-7.4.0.tar.gz
$PWD/../gcc-7.4.0/configure --prefix=$HOME/GCC-7.4.0 --enable-languages=c,c++

Set the right environment variables

These are necessary for building CMake and later the Graphic Engine.

export PATH=~/GCC-7.4.0/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/GCC-7.4.0/lib:~/GCC-7.4.0/lib64
export CC=~/GCC-7.4.0/bin/gcc
export CXX=~/GCC-7.4.0/bin/g++

Build CMake

Since CMake 3 is needed, you have to build it from its source. It’s very quick though. One thing to note is, when doing the “make install”, you have to be root. You need to re-export LD_LIBRARY_PATH

sudo -s
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<absolutepathtoGCClib>:<absolutepathtoGCClib64>
make install

Build Graph Engine

Finally, we’re ready to build the source of Microsoft Graph Engine. Exciting?! As mentioned earlier, I checked out an earlier commit:

git clone https://github.com/Microsoft/GraphEngine.git
git checkout 4175d13a084453bd2a2bf43adf1cc4e76f9fe417
cd GraphEngine/
scl enable rh-dotnet22 bash
tools/build.sh

Voila, you have the Nuget package built under the build/ folder!