Szymon Kaliski

Kinect 2 on OSX with skeleton tracking

Howto guide for C++ development

This tutorial describes how to get Kinect 2 working on OSX with NiTE skeleton tracking. Tested on OSX 10.11, OpenFrameworks 0.9.3.

Preparation

  1. Install libfreenect2, make sure to install OpenNI2, and compile libfreenect2 with openni2 support (using make install-openni2)
  2. Downlad NiTE2 — I'm not sure about the official status of this library, so I'm not going to provide a link for now, but you should be able to find it easily if you google for NiTE-MacOSX-x64-2.2.tar.zip
  3. When compiling openni2 I had to run cmake with: -DENABLE_OPENGL=ON -DENABLE_CUDA=OFF (to disable CUDA)
  4. Put libfreenect2/build/lib/libfreenect2-openni2.0.dylib in NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI2/Drivers
  5. You should be able to run one of the samples now, like UserViewer: cd into NiTE-MacOSX-x64-2.2/Samples/Bin and run ./UserViewer, this might fail if CUDA is not available, but I had luck running it with OpenCL backend, and you can always just run on CPU (LIBFREENECT2_PIPELINE=cl|cuda|cpu ./UserViewer)

OpenFrameworks project setup

I couldn't find any OpenFrameworks addons that would work for me so here's how to use the libraries without any wrappers. Project setup is a bit complicated, but worth it.

  1. Create new project with OpenFrameworks projectGenerator, add all other addons that you will need (and watch out, re-adding addons through projectGenerator will break all the other changes we will make!)
  2. Once Xcode is ready, create two new groups: libs and includes (arbitrary names), with NiTE2 and OpenNI inside of each one:

  3. Go to NiTE-MacOSX-x64-2.2/Include and drag all *.h files into includes/NiTE2 group (no need to "copy items if needed", but remember to "add to targets")

  4. Go to /usr/local/include/ni2/ and drag all files and directories into the includes/OpenNI group (no need to "copy items if needed", but remember to "add to targets")

  5. Go to NiTE-MacOSX-x64-2.2/Redist and drag all files into the libs/NiTE2 group (select "add to targets")

  6. Go to NiTE-MacOSX-x64-2.2/Samples/Bin and drag libOpenNI2.dylib, libOpenNI2.jni.dylib, OpenNI2/, org.openni.jar and OpenNI.ini into libs/OpenNI2 group (select "add to targets"), remember that libfreenect2-openni2.0.dylib should be in OpenNI2/Drivers group!

  7. Go to "Build Phases" of the project, and clean up "Link Binary WIth Libraries": it should only contain openFrameworksDebug.a, libOpenNI2.dylib and libNiTE2.dylib

  8. Go to "Build Setings" and setup "Library Search Paths" under "Target":

  9. Go back to "Build Phases" and setup "Copy Files" (make sure that proper "Subpaths" are set):

Now we should be able to add some NiTE code and get this to run!

Working with NiTE in C++

Let's start with testing if NiTE is working, adding this to the ofApp.cpp:

#include "ofApp.h"
#include "NiTE.h"

void ofApp::setup() {
  nite::NiTE::initialize();

  nite::UserTracker userTracker;

  nite::Status niteRc = userTracker.create();

  if (niteRc != nite::STATUS_OK) {
    ofLogError() << "Couldn't create user tracker";
    return 1;
  }
  else {
    ofLogNotice() << "NiTE is working!";
  }
}

For different resource paths to work properly, we need to run the app from the Contents/MacOS directory, and we also have to specify LIBFREENECT2_PIPELINE variable, I'm usually setting up two bash scripts:

build.sh:

#!/usr/bin/env bash

xcodebuild -project *.xcodeproj -configuration Debug

run.sh (remember to change Kinect2TestDebug to your executable name):

#!/usr/bin/env bash

pushd ./bin/*.app/Contents/MacOS/
LIBFREENECT2_PIPELINE=cl ./Kinect2TestDebug
popd

We can now run ./build.sh && ./run.sh and if everything is ok, we should see "NiTE is working!" in the command line.

Now for the fun part, here's simple stick-figure code that you can treat as starting point (of course all library paths will be broken, so use just the src/* files, after setting up the project): https://github.com/szymonkaliski/of-exp-kinect2-nite-osx