Using the Kinect Depth Camera with OpenCV
This page is a starting point for learning how to use a Kinect RGB-D Depth Camera (originally for Microsoft's XBox 360) with OpenCV.
Sam Muscroft has successfully incorporated the raw depth, rgb depth map and rgb output from the Kinect sensor into an OpenCV project using the Windows CL NUI Platform.
RGB output as you'd expect is stored in an 8 bit 3 channel matrix. Depth needs to be stored in a 16 bit 1 channel matrix.
He found the easiest way to output the data (depth & RGB) was to create an image header of the appropriate bit depth and number of channels and populate with the data returned from the open-source Kinect API.
Using the C++ API from nuigroup in Windows, the data can be accessed using the following:
// Raw Depth Data PUSHORT rawData = (PUSHORT) malloc(640*480*3); GetNUICameraDepthFrameRAW(KinectCamera, rawData)
In OpenCV you can then use the data this way:
kinectDepthImage = cvCreateImage( cvSize(640,480), 16, 1); cvSetData(kinectDepthImage, rawData, kinectDepthImage->widthStep); cvReleaseImageHeader(&kinectDepthImage);
To access Kinect using the NUI library, visit http://www.nuigroup.com/forums/viewthread/11249/.
Other examples of Kinect with OpenCV
There is an example on how to access Kinect from OpenCV at the MoreThanTechnical blog.
Also there is a source code example using the Kinect camera with the OSC library and OpenCV at the ProjectAllusion blog.
To read more about how the Kinect camera works, read the article at Wired Magazine.
Note: Open source drivers for many operating systems and the newly opened API (not source) for their body tracker are hosted at:
Using Kinect with ROS
The ROS library (from Willow Garage, the same company that operates OpenCV) has a Kinect module that is officially supported and documented, so it is recommended to read the ROS Kinect Wiki.
There is also a mailing list "ros-kinect" for Kinect specific discussions on ROS. You can read the archived messages or subscribe to the mailing list.
For a list of the most recent items on the ROS website related to Kinect, view the ROS Kinect entries.