You must definitely know the Kinect (formerly Natal Project)? This is the latest toy for the Xbox from Microsoft which allows players to control games with the help of their body.
This week I had the opportunity to test it and to realize all the possibilities it offered (precision of the movements, recognition of the skeleton, etc..). Pending an official SDK from Microsoft (announced in spring 2011), I did some tests on PC by helping me with the drivers supplied by the open source community. But if you don’t know this environment, it’s fairly complex to know what projects to use, what they do and how to install them.
The objective of this tutorial is to explain to you what are the differences between OpenKinect and OpenNI in a first time. And then we’ll see how to install the OpenNI framework in order to run the first examples on your PC (OpenNI runs on Mac OSX and Linux, but their installations/usages are a little different and therefore they’ll be not explained here).
In a second part we will see how to use some tools such as Processing and Animata to test some possibilities offered by the Kinect.
Before to start, if you bought the Xbox + Kinect pack, you must purchase an USB adapter with the power supply to connect the Kinect to the computer. It is only available on Microsoft website here.
For those who bought the Kinect without the pack, the adapter is supplied with.
- OpenKinect is an open source project built around the libfreenect driver which offering an APIs to control the motor, the LEDs, the cameras and the audio. It also provides a library to analyze the outputs of the Kinect in order to provide a layer abstraction such as the tracking of the hands or of the skeleton. Moreover several wrappers are available to allow you to develop in your preferred language (Java, C#, C++, Actionscript, Python, etc.).
- OpenNI (Open Natural Interaction) is an open source Framework (under LGPL), partially developed by PrimeSence which is one of the creators of the Kinect. It provides standard abstract programming interfaces (API) (in C++ and C #) which allow developers to write applications based on natural interactions. It’s not specific to the Kinect and that’s its main advantage because it can be plug to another audio or visual device sensors.
This scheme shows us the architecture of OpenNI. To make simple OpenNI allows you to connect the sensors that send data raw to middleware which will analyze them, will forward them to other middleware and/or will deal with them to finally sent high-level data to the application (like Push Detector, Wave Detector). This modular architecture allows you to interface different middleware or devices complying with the API defined by OpenNI. In our case NITE as middleware and the Kinect as device.
Now, I’m going to explain you how to install the various components of the OpenNI in order to run the Kinect on your PC.
To begin we are going to install OpenNI :
- Go here first and download the last OpenNI version (v22.214.171.124 at the time of writing).
- Once the download is finished, run the installer and follow the instructions.
Now we are going to install the Kinects drivers that can interface with the OpenNI’s API:
- To start, download the SensorKinect drivers here (v5.0.0) (If you don’t use a git client click on Downloads > Download.zip).
- Once the download is finished, extract the file and go to the Bin folder.
- Then run the executable and follow the instructions.
- To finish you need to install an additional driver which is located into the folder where you extracted the driver and go in the path Platform\Win32\Driver. If you are on a 32 bit system install the driver dpinst-x86.exe and if you are on a 64 bit system install the driver dpinst-amd64.exe.
Finally we are going to install the last component which allows OpenNI to run, namely NITE which is a middleware that contains all image processing algorithms and treatments of data raw from sensors:
- In the first place download the latest version of NITE here (v126.96.36.199 when I write these lines).
- Once the download is complete, run the installation and follow the instructions. During the installation it’ll ask you a key. Enter the following key: 0KOIk2JeIBYClPWVnMoRKn5cdY4= (NITE is not free but PrimeSense provided a free key to allow users to test its middleware).
All must be properly installed on your PC. To verify this you just have to connect the adapter on your Kinect then the on electric plug before connecting it to your PC. If all goes well, the green Led will begin flashing and Windows will begin installing the device.
Finally go to the OpenNI installation directory and browse to the Samples>Bin>Release folder and run one of example as NiViewer.exe. If everything works, you should see this :
(IR and RGB video)
If you need to go deeply you can read this really great book: OpenNI Cookbook.