Szymon Kaliski

Kinect 2 on macOS with Skeleton Tracking

This tutorial describes how to get Kinect 2 working on macOS with NiTE skeleton tracking. Tested on macOS 10.11, OpenFrameworks 0.9.3.

Thanks to George Profenza ↗ for his help in making Kinect 2 work on macOS.

Preparation

1. Install libfreenect2 ↗, make sure to install OpenNI2, and compile libfreenect2 with openni2 support (using make install-openni2)
2. Downlad NiTE2 — I'm not sure about the official status of this library, so I'm not going to provide a link for now, but you should be able to find it easily if you google for NiTE-MacOSX-x64-2.2.tar.zip
3. When compiling openni2 I had to run cmake with: -DENABLE_OPENGL=ON -DENABLE_CUDA=OFF (to disable CUDA)
4. Put libfreenect2/build/lib/libfreenect2-openni2.0.dylib in NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI2/Drivers
5. You should be able to run one of the samples now, like UserViewer: cd into NiTE-MacOSX-x64-2.2/Samples/Bin and run ./UserViewer, this might fail if CUDA is not available, but I had luck running it with OpenCL backend, and you can always just run on CPU (LIBFREENECT2_PIPELINE=cl|cuda|cpu ./UserViewer)

OpenFrameworks project setup

I couldn't find any OpenFrameworks addons that would work for me so here's how to use the libraries without any wrappers. Project setup is a bit complicated, but worth it.

1. Create new project with OpenFrameworks projectGenerator, add all other addons that you will need (and watch out, re-adding addons through projectGenerator will break all the other changes we will make!)

2. Once Xcode is ready, create two new groups: libs and includes (arbitrary names), with NiTE2 and OpenNI inside of each one:

   

3. Go to NiTE-MacOSX-x64-2.2/Include and drag all *.h files into includes/NiTE2 group (no need to "copy items if needed", but remember to "add to targets")

4. Go to /usr/local/include/ni2/ and drag all files and directories into the includes/OpenNI group (no need to "copy items if needed", but remember to "add to targets")

   

5. Go to NiTE-MacOSX-x64-2.2/Redist and drag all files into the libs/NiTE2 group (select "add to targets")

6. Go to NiTE-MacOSX-x64-2.2/Samples/Bin and drag libOpenNI2.dylib, libOpenNI2.jni.dylib, OpenNI2/, org.openni.jar and OpenNI.ini into libs/OpenNI2 group (select "add to targets"), remember that libfreenect2-openni2.0.dylib should be in OpenNI2/Drivers group!

   

7. Go to "Build Phases" of the project, and clean up "Link Binary WIth Libraries": it should only contain openFrameworksDebug.a, libOpenNI2.dylib and libNiTE2.dylib

   

8. Go to "Build Setings" and setup "Library Search Paths" under "Target":

   

9. Go back to "Build Phases" and setup "Copy Files" (make sure that proper "Subpaths" are set):

   

Now we should be able to add some NiTE code and get this to run!

Working with NiTE in C++

Let's start with testing if NiTE is working, adding this to the ofApp.cpp:

#include "ofApp.h"
#include "NiTE.h"

void ofApp::setup() {
  nite::NiTE::initialize();

  nite::UserTracker userTracker;

  nite::Status niteRc = userTracker.create();

  if (niteRc != nite::STATUS_OK) {
    ofLogError() << "Couldn't create user tracker";
    return 1;
  }
  else {
    ofLogNotice() << "NiTE is working!";
  }
}

For different resource paths to work properly, we need to run the app from the Contents/MacOS directory, and we also have to specify LIBFREENECT2_PIPELINE variable, I'm usually setting up two bash scripts:

build.sh:

#!/usr/bin/env bash

xcodebuild -project *.xcodeproj -configuration Debug

run.sh (remember to change Kinect2TestDebug to your executable name):

#!/usr/bin/env bash

pushd ./bin/*.app/Contents/MacOS/
LIBFREENECT2_PIPELINE=cl ./Kinect2TestDebug
popd

We can now run ./build.sh && ./run.sh and if everything is ok, we should see "NiTE is working!" in the command line.

Now for the fun part, here's simple stick-figure code that you can treat as starting point (of course all library paths will be broken, so use just the src/* files, after setting up the project): szymonkaliski/of-exp-kinect2-nite-osx ↗