OpenCV Application to GSOC2010

Organization

Open Source Computer Vision Library (OpenCV)

Description:

The Open Source Computer Vision Library (OpenCV)is an open, free for commercial use, computer vision and machine learning software library. OpenCV contains a comprehensive set of both classic and state of the art computer vision and machine learning algorithms. These algorithms can be used, for example, to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eye from images taken using flash, follow eye movements, recognize scenes and establish markers to overlay the scenes with augmented reality to name a few. OpenCV has well over 2 million downloads, has an active user group with over 41 thousand registered members. The O'Reilly coding book on OpenCV, "Learning OpenCV" has been the best selling computer vision or machine vision book for a year and a half no. OpenCV is used extensively in companies, research groups and by governmental bodies.

Some well known companies that use OpenCV are Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, Toyota. Many startups such as Applied Minds, VideoSurf, and Zeitera make extensive use of OpenCV. OpenCV's deployed uses span the range from stitching streetview images together, detecting intrusions in surveillance video in Israel, monitoring mine equipment in China (more notoriously, OpenCV is a key part of China's "Green Dam" internet filtering software <not in our control>), helping robots navigate and pick up objects at Willowgarage, detection swimming pool drowning events in Europe, running interactive art in Spain and New York, checking run ways for derbies in Turkey, inspecting labels on products in factories around the world on to rapid face detection in Japan.

OpenCV leans mostly towards real time vision applications and takes advantage of MMX and SSE instructions when available. A CUDA interface is being developed right now. There are over 500 algorithms and about 10 times as many functions that compose or support those algorithms. OpenCV is written natively in C++ and has a templated interface that works seamlessly with std containers. Its native data type is a general matrix class that reference counts and leverages LAPACK and (in April) EIGEN. OpenCV also has a full custom Python interface and, using SWIG, OpenCV has interfaces to Matlab and Octave. OpenCV was built to provide a common infrastructure for vision applications and to accelerate the use of machine perception in the commercial products. To enable this, OpenCV has a BSD license to make it easy for businesses to use and modify the code.

Home Page:

http://opencv.willowgarage.com/wiki/

License:

BSD

Why is your organization applying to participate in GSoC 2010? What do you hope to gain by participating?

Several companies support OpenCV developers directly: Willow Garage, Intel and NVidia. However, this support, while producing open code, prioritizes applications of interest to these companies. There are many areas in computer vision that have received less attention and lower priority even though those areas are important. GSoC can help us fill critical gaps in these areas. Examples are in the list of mentor project ideas http://opencv.willowgarage.com/wiki/GSOC_OpenCV2010 , but for example, image stitching is used extensively by Google Maps and Streetview and yet we have no one working directly on algorithms in that area. Our GUI, intended for rapid debug purposes, has also become somewhat out of date and needs updating.

Did your organization participate in past GSoCs? If so, please summarize your involvement and the successes and challenges of your participation.

OpenCV has not yet participated in GSoC, though we tried, see below.

If your organization participated in past GSoCs, please let us know the ratio of students passing to students allocated, e.g. 2006: 3/6 for 3 out of 6 students passed in 2006.

N/A

If your organization has not previously participated in GSoC, have you applied in the past? If so, for what year(s)?

Yes, we applied in 2008 and 2009. I was told we were rejected because the GSoC team thought OpenCV had become abandoned/stale. This is not the case -- a total of 7 people are now paid full time to work on OpenCV, 2 more will join that number and we have many active developers http://opencv.willowgarage.com/wiki/Contributors. Three two large and one small corporation contributes ongoing financial support to OpenCV.

What is the URL for your ideas page?

http://opencv.willowgarage.com/wiki/GSOC_OpenCV2010

What is the main development mailing list for your organization? This question will be shown to students who would like to get more information about applying to your organization for GSoC 2010. If your organization uses more than one list, please make sure to include a description of the list so students know which to use.

For developers, we use:

The students and mentors will have their own list

What is the main IRC channel for your organization?

Does your organization have an application template you would like to see students use? If so, please provide it now. Please note that it is a very good idea to ask students to provide you with their contact information as part of your template. Their contact details will not be shared with you automatically via the GSoC 2010 site.

OpenCV_GSOC2010_Application

What criteria did you use to select the individuals who will act as mentors for your organization? Please be as specific as possible:

  1. All mentors but one are long time core developers of OpenCV, the 5th mentor, Nicolas Saunier has been known to us for years and just became an assistant Professor and is committed to helping OpenCV.
  2. All mentors have extensive computer vision and coding background.
  3. All mentors are people who have interns or students to manage every year, either in companies, academic settings or research groups.

What is your plan for dealing with disappearing students?

  1. We will post the orphaned project to our user group and offer a small cash prize (~$200) to someone who completes the project.

What is your plan for dealing with disappearing mentors?

  1. Send flowers to the funeral. Our mentors are all core developers or people who we've known > 10 years. Death would be the main reason that they would disappear.

  2. We have 2 backup mentors: one at Willow Garage, on at Applied Minds who will step in (after a socially minimal mourning period has passed).

What steps will you take to encourage students to interact with your project's community before, during and after the program?

  1. Before the project, we will encourage them to post a detailed project plan to the user group (also socially committing them to complete).
  2. We will set up a 2x daily twitter feed for all the summer's development projects.
  3. Project results will go up on a youtube channel that will be posted to the large user group. This should provide a large amount of feedback.
  4. We hope to set up projects with 3 goals: Quick and turn around (within a few weeks they should have a baseline accomplishment), this will be followed by the main, reasonable summer goal, but there will also be a more comprehensive "stretch goal" that we will encourage them to complete, getting feedback and feature requests from the community.

What will you do to ensure that your accepted students stick with the project after GSoC concludes?

  1. As mentioned above, the students will be left (by design) with a much more comprehensive, cooler, "stretch goal" that we hope they will be motivated to complete.

  2. If students are good, Willowgarage will hire them as interns.

Is there anything else you would like to tell the Google Summer of Code program administration team? :

One of my fears is that non-vision people at Google will look at the projects and decide "they are too hard". We have a user group of over 40,000 people, many students and I already have plenty of interest from students in applying to GSOC. Thus, there are plenty of students capable of doing this work.

Google makes use of OpenCV in various apps such as Streetview and map stitching. You may contact "Jean-Yves Bouguet" <jyb-delete-@-delete-google.com> for details on street view's use. As "pay-back", one of our projects is intended to fill out missing functionality on street view -- making it easy for someone with an Android based cell phone to stitch together a graph of Android acquired panoramas of the outside and inside views of their business. This can also be done to allow capturing parks, trails etc. I've already talked with Jean-Yves on the streetview team and he's really wants this done and will connect it to streetview functionality when done. Again, talk to him.

There will also be an app that makes it easier to overlay augmented reality onto a scene viewed through an Android cellphone.

There are many things we want to do, such as supporting cell phone use, that is just not in the scope of existing corporate support for OpenCV, yet we very much want to see these uses. GSoC will enable us to do this.

Vadim Pisarevsky, -delete-Vadim.-delete-Pisarevsky@-delete-gmail.com

GSOC_OpenCV2010

OpenCVWiki: ApplicationGSOC2010 (last edited 2010-03-18 18:57:27 by GaryBradski)