• Kinect

Using the Kinect Depth Camera with OpenCV

This page is a starting point for learning how to use a Kinect RGB-D Depth Camera (originally for Microsoft's XBox 360) with OpenCV.

Sam Muscroft has successfully incorporated the raw depth, rgb depth map and rgb output from the Kinect sensor into an OpenCV project using the Windows CL NUI Platform.

RGB output as you'd expect is stored in an 8 bit 3 channel matrix. Depth needs to be stored in a 16 bit 1 channel matrix.

He found the easiest way to output the data (depth & RGB) was to create an image header of the appropriate bit depth and number of channels and populate with the data returned from the open-source Kinect API.

Using the C++ API from nuigroup in Windows, the data can be accessed using the following:

// Raw Depth Data
PUSHORT rawData = (PUSHORT) malloc(640*480*3);
GetNUICameraDepthFrameRAW(KinectCamera, rawData)

In OpenCV you can then use the data this way:

kinectDepthImage = cvCreateImage( cvSize(640,480), 16, 1);
cvSetData(kinectDepthImage, rawData, kinectDepthImage->widthStep);

cvReleaseImageHeader(&kinectDepthImage);

You may find it useful to convert the raw depth data to something approximating physical distance, by using a formula published by Nicolas Burrus or Stéphane Magnenat.

To access Kinect using the NUI library, visit http://www.nuigroup.com/forums/viewthread/11249/.

Other examples of Kinect with OpenCV

There is an example on how to access Kinect from OpenCV at the MoreThanTechnical blog.

There is also some info such as available enbedded platforms that support Kinect and source code how to access Kinect from OpenCV at the WhatNickLife blog.

Also there is a source code example using the Kinect camera with the OSC library and OpenCV at the ProjectAllusion blog.

The OpenFrameworks library has a Kinect module called ofxKinect, which could potentially be combined with ofxOpenCV.

To read more about how the Kinect camera works, read the article at Wired Magazine.

• Note: Open source drivers for many operating systems and the newly opened API (not source) for their body tracker are hosted at:

  • http://openni.org/

Using Kinect with ROS

• The ROS library (from Willow Garage, the same company that operates OpenCV) has a Kinect module that is officially supported and documented, so it is recommended to read the ROS Kinect Wiki.

• It includes a Getting Started tutorial, a camera calibration method, and lots of nice info on their old page about the kinect_node (which will probably move to their newer page soon?).

• There is also a mailing list "ros-kinect" for Kinect specific discussions on ROS. You can read the archived messages or subscribe to the mailing list.

• For a list of the most recent items on the ROS website related to Kinect, view the ROS Kinect entries.