OpenCV Ideas 2009
- Sorry, Google didn't accept us in summer of code, no idea why. Most of the accepted organizations are large and general purpose (MySQL, Django, ...) but some are obscure and attempting to create functionality that already exists in OpenCV (Open Image?). It could be that they view OpenCV as too much of a niche (though 40K user group, 2M downloads) or that computer vision is too hard/specialized for most programmers (but we already had several applicants who were well qualified).
- If you attend an open source conference, and see the summer of code people there, ask. We will attempt an email conversation with them when they are less busy.
GSoC email gsoc@google.com
All coding will be done in C++/C except for the python one below.
To apply, see what we're looking for GSoC_OpenCV_Application
Ideas that need to be worked on:
Image Stitching
Goal: Implement functionality for stitching multiple images into a single panorama. -- Difficulty: Moderate (features) to High
- Make a simple highgui-based demo out of it.
Mentor: Vadim Pisarevsky, vadim.pisarevsky (+) gmail.com
Vadim is an OpenCV founder and is now lead developer
1. Test data: streetview data (courtesy of Google's Streetview team).
2. Algorithm stages (taken from http://en.wikipedia.org/wiki/Photo_stitching):
Calibration: Already implemented in OpenCV, could be further automated. This step can be skipped if vignetting, distortion etc. can be compensated at image blending stage.
Image registration: You can use SURF or other 2D features already in OpenCV (see below). Or, students can code up a list of 2D point features to be provided that we don't have yet.
Image blending: Some simple algorithm, like pyramid-based, can be implemented.Will be discussed.
We will judge the results for image stitching at the end and award prizes. Google streetview team members will help judge the results.
Feature Detectors
Goal: Develop features and Interest point detectors for recognition (see below), image stitching (see above) and planar tracking. Difficulty: Easy to Hard
Mentor1: Victor Eruhimov, relrotciv (+) googlemail.com
Victor is an OpenCV founder and is now Research Lead on the OpenCV team
Mentor2: Gary Bradski, bradski (+) willowgarage.com
Gary started OpenCV, is a Consulting Prof. at Stanford CS Dept., and Senior Scientist at Willow Garage
- Scale adaptive Harris
K. Mikolajczyk and C. Schmid, “Indexing based on scale invariant interest points” ICCV’01,pp. 525—531, 2001.
- Scale adaptive Hessian
K. Mikolajczyk and C. Schmid, “Scale and Affine invariant interest point detectors,” IJCV, Vol. 60, No. 1, pp.63-86, 2004
- Affine invariant regions
T. Tuytelaars and L. Van Gool. “Matching widely separated views based on affine invariant regions,” IJCV, Vol. 59, No. 1, pp. 61–85, 2004
T. Kadir, A. Zisserman, and M. Brady, “An affine invariant salient region detector,” ECCV’04, pp. 404-416, 2004
K. Mikolajczyk and C. Schmid, “Scale and Affine invariant interest point detectors,” IJCV, Vol. 60, No. 1, pp.63-86, 2004
GLOH Wikipedia
- Laplacian of Gaussians (LoG)
T. Lindeberg, “Feature Detection with Automatic Scale Selection”, IJCV, Vol. 30, No. 2, 1998
- Boundary fragments
Scale Invariant Object Categorization Using a Scale-Adaptive Mean-Shift Search. Bastian Leibe and Bernt Schiele. In DAGM'04 Annual Pattern Recognition Symposium, Tuebingen, Germany, Aug. 2004. Springer LNCS, Vol. 3175, pp. 145-153, 2004. DAGM Best Paper Award.
Opelt, A. , Pinz, A. and Zisserman, A. Learning an Alphabet of Shape and Appearance for Multi-Class Object Detection International Journal of Computer Vision (2008)
Combined Object Categorization and Segmentation with an Implicit Shape Model. Bastian Leibe, Ales Leonardis, and Bernt Schiele. In ECCV'04 Workshop on Statistical Learning in Computer Vision, Prague, May 2004.
SIFT interest point detection on a Difference of Gaussians pyramid Too late, we've done it
SURF interest point detection on a Haar wavelet decomposition Again, we went and did it
Maximally Stable Extremal Regions Sorry, it's in
Harris corner detector D'oh!
Good Features to Track Too late
At the end, feature performance will be assessed and prizes will be awarded
Face People and Car Detection and Recognition
Goal: Use OpenCV's Cascade, Boosting and Extremely Random Trees to train up new, improved Face and People Detectors using both mono and stereo images. Difficulty: Easy to Moderate
Mentor: Rainer Lienhart, rainer.lienhart (+) informatik.uni-augsburg.de
Rainer is a Prof. at Institut für Informatik http://www.lienhart.de/ and developed the face detector in OpenCV
Mentor2: Gary Bradski, bradski (+) willowgarage.com
Gary started OpenCV, is a Consulting Prof. at Stanford CS Dept., and Senior Scientist at Willow Garage
Mentor3: Nicolas Saunier, saunier (+) civil.ubc.ca
Nicolas is a Research Associate at the Transportation Engineering group of the University of British Columbia and uses computer vision to monitor road users and safety, http://nicolas.saunier.confins.net
- Rainer and Gary will focus on faces and people, Nicolas will focus on the detection of different types of road users (cars, trucks, bike...).
- You will help get publicly available face and people data with Rainer's assistance.
Gary will assist in getting a lot of labeled data through his Amazon Mechanical Turk connection.
- We will be using both mono imagery and stereo imagery.
- You will help prepare and preprocess the data.
- Run feature detectors over it.
- Train up classifiers.
- We will run them in real time to assess the results.
At the end, recognition results will be judged and prizes awarded according to performance.
HighGUI enhancements
Goal: make highgui, the OpenCV visualization GUI more powerful and friendly environment for computer vision experiments. Difficulty: Easy to Hard
Mentor: Vadim Pisarevsky, vadim.pisarevsky (+) gmail.com
Vadim is an OpenCV founder and is now lead developer
I will work with you, take what parts you like. What can be done:
UI part:
- Another UI backend that is based on Qt (since Qt is now LGPL and can provide native look-n-feel on every major platform: Linux, Windows, MacOSX).
- Changing window type (fixed-size/variable-size) at runtime, zooming, scrolling, 1:1 mode, per-window threads that repaint the content automatically.
Saving-restoring window locations & sizes on every platform
- Statusbars, displaying current cursor location in pixels and the pixel color.
- Saving current window content to a file.
- Overlay/transparency support, multi-win configurations.
- OpenGL graphics in highgui windows.
- Better text rendering
Video I/O part:
- FFMPEG 0.5 support.
- Consistent and predictable positioning in video streams
- robust video writing.
Also, feel free to commit yourself regardless of the SoC 
Again, prizes for elegance and features will be awarded at the end.
Python
Goal: Improve Python bindings -- Difficulty: moderate
Mentor: Mark Asbach asbach (+) ient.rwth-aachen.de
Institut für Nachrichtentechnik, RWTH Aachen University
integrate array interface for seamless integration with numpy, scipy, PyMat done
accept native types in addition to OpenCV-specific,
e.g.: (x,y) vs of cv.cvPoint(x,y) has been done
- remove double prefixes, such as cv.cv
Prizes for how well and comprehensive your work is will be awarded.
OpenCV on the iPhone
Goal: Get as much of OpenCV as you can on he iPhone. -- Difficulty: Hard
Mentor: Mark Asbach asbach (+) ient.rwth-aachen.de
Institut für Nachrichtentechnik, RWTH Aachen University
- At least HighGUI
- Optical flow
- Calibration
- Face detection
- Place recognition? (from Street view)
You bet, prizes for success.
3D Mesh
Goal: Creating 3D meshes from point clouds. -- Difficulty: Hard
Mentor: Nils Hasler nils.hasler (+) gmail.com
Nils does research at:
Max-Planck-Institut Informatik
Campus E1 4, D-66123 Saarbrücken
Phone: +49 681 9325-420
www.mpi-inf.mpg.de/~hasler
- 3D Point clouds from stereo and laser data
- Creating a mesh from the cloud
- Loading and saving of meshes
- Plot them with a projection matrix makes sense.
- maybe use OpenGL internally and allow messing with the OpenGL settings...
- If you are super fast -- 3D models from rotating textured objects.
You guessed it: Prize awarded for efficiency and comprehensive mesh solutions.
Machine Learning Methods
Goal: Implement efficient state-of-the-art classification and regression engines. Difficulty: Easy to Hard
Mentor: Victor Eruhimov, relrotciv (+) googlemail.com
Victor is an OpenCV founder and is now Research Lead on the OpenCV team
- Gradient Boosting Trees (regression, binary and multiclass classification, support for very large and high-dimensional datasets)
Friedman, J. H. "Greedy Function Approximation: A Gradient Boosting Machine." (Feb. 1999a) http://www-stat.stanford.edu/%7Ejhf/ftp/trebst.pdf
Friedman, J. H. "Stochastic Gradient Boosting ." (March 1999b) http://www-stat.stanford.edu/~jhf/ftp/stobst.pdf
- Stretch Goals
- Create a cascaded classifier based on Gradient Boosting Trees as was done for the face recognizer. Victor will discuss/assist.
- Learning people and car detectors (see Face People and Car Detection and Recognition section) with cascaded GBT
- Create a cascaded classifier based on Gradient Boosting Trees as was done for the face recognizer. Victor will discuss/assist.
We will train the Gradient Boosting and/or the Gradient Cascade on faces, people cars and award prizes accordingly.
Your own ideas
Feel free to suggest your own ideas and the mentoring team will gladly look at them. Several other ideas are in this list: