• OpenCV Monthly

OpenCV Monthly

What is new in OpenCV, particularly, what is new in the latest SVN updates

2/1/2010

• 20 new tests have been written, 7 existing tests have been extended, the whole new family of so-called “bad-arg” tests has been added to the test system. These are tests that check the reaction of the functions to invalid input paramters. Such tests helped us to cover a lot of branches in OpenCV functions that are never executed in the normal program flow.
• Considering the thousands of functions, test coverage in cxcore and cv have shown large improvements. Test coverage in calibration is dramatically improved.

  Module	Functions/Conditions Coverage, % (Dec 2009)	Functions/Conditions Coverage, % (Jan 2010)
  cxcore	84/67	91/73
  cv	71/60	78/66
  highgui	53/26	53/27
  ml	73/52	73/52
  cvaux	5/12	5/12
  calibration	65/64	93/84

In absolute numbers: 136 cxcore and cv functions and >1700 conditions have been covered during January. In total, cxcore and cv contain ~1950 functions and >28500 conditions.

• The whole cv has been modified to use exceptions rather than the old method using the global error status. This made OpenCV error handling mechanism usable again (the old method was not working properly in OpenCV 2.0) and made bad-arg tests possible.
• Several improvements in the chessboard corner detector have been implemented (that reduced corner localization error from ~1 pixel downto ~0.3 pixels) that made camera calibration somewhat more robust, but still not perfect.
• Over 11 bugs have been fixed (trac tickets #50, #53, #54, #57, #63, #53, #69, #72, #87, #88, #89)
• High resolution stereo correspondence: added stereo correspondence sample in opencv, block matching takes 15s per 10MP frame

12/31/09

• We've gotten more serious about OpenCV's software engineering. We now have a full C++ and Python interface. We've mainly been focusing on increasing regression test coverage of the code, bug fixing and improving documentation. Code test coverage (on O(5000) functions) is now at:

  • OpenCV Module	coverage % (funcs/branches) on Nov 13th	coverage % (funcs/branches) on Dec 4th
    cxcore	65 / 54	84 / 67
    cv	52 / 46	72 / 61
    highgui	17 / 3	53 / 26
    ml	66 / 47	73 / 52
    cvaux	0 / 0	5 / 12
    calibration	0/ 0	76 / 67

    • Notes

      • Charts for this may be seen here.

      • Some code (highgui) has expanded so even though there are more tests, the coverage is slightly less than last month.
      • Calibration and stereo has been a particular area of focus so it's a new category at bottom.
• 39 bugs were fixed this month out of the 54 outstanding.
• We are working on geometry and texture. This is not incorporated yet, but it is used internally:
  • We now have fairly reliable outlet localization of 2x1 white on white electrical outlets from the robot arm camera to enable new automated plugging in.

    • It is reliable in X,Y but, due to low resolution, less reliable in Z. The average localization error is <1.5mm

  • We have added automatic parking to the Willow Garage tele-presence robot.

• Added getOptimalNewCameraMatrix(...) to allow user to control how much of the rectified image is kept. You can see the images in as an example.

• New flag, CV_CALIB_CB_FAST_CHECK}} was added to {{{cvFindChessboardCorners(...) that makes chessboard finding up to 20X faster when there is no chessboard in the scene.

  • It had been frustrating to the user that chessboard detection slows down so much when there is not chessboard since the algorithm then would try hard everywhere to find a chessboard.

• 30% of the way there to updating the error handling in OpenCV to use c++ exceptions CV_Error(), CV_Assert()

12/4/09

• GrabCut integrated.

• the up-to-date C++ documentation is now available online: http://opencv.willowgarage.com/documentation/cpp/index.html

• 17 new tests have been written; >20 existing tests have been extended.

  • As a result, the test coverage was increased:

  • OpenCV Module	coverage % (funcs/branches) on Nov 13th	coverage % (funcs/branches) on Dec 4th
    cxcore	65 / 54	78 / 62
    cv	52 / 46	66 / 56
    highgui	17 / 3	55 / 27
    ml	66 / 47	73 / 52
    cvaux	0 / 0	2 / 12

• the new tests helped to catch and fix a lot of bugs too

• OpenCV SVN moved to https://code.ros.org/svn/opencv. The new bug tracker is at: https://code.ros.org/trac/opencv/report/1, although the tracker at SourceForge is still alive (until we clean it sufficiently)

11/4/09

• GrabCut draft code is ready.

Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. GrabCut - Interactive Foreground Extraction using Iterated Graph Cuts. ACM Transactions on Graphics (SIGGRAPH), 2004

• After some code cleanup it will be integrated into OpenCV

• Automated OpenCV build system is now up and running. It continuously builds and tests OpenCV in various configuration on different platforms.
  • OS tested: Windows (32-bit XP), Linux (32-bit Ubuntu 8.04, 64-bit Ubuntu 9.04), MacOSX (32-bit 10.5)
  • Compilers: MSVC 2008 and MinGW 4.3 on Windows, GCC 3.4, 4.1, 4.2, 4.3 on Linux, GCC 4.0 on MacOSX
  • Optimization options: MMX,SSE,SSE2,SSE3 on/off, OpenMP on/off, Debug/Release
  • We also build Python wrappers and the Latex-based documentation
  • Finally, we build all the ROS packages that depend on OpenCV, that give us some extra confidence in the compatibility.
  More options are going to be added later.

• ~20 bugs in 2.0 release have been fixed. These include compile errors (such as undeclared _exchange_and_add), crashes in 64-bit versions because of Lapack incompatibility etc. Please see the bug tracker for details: http://sourceforge.net/tracker/?group_id=22870&atid=376677

• Stefano Fabri committed the patch for gstreamer backend in highgui that enables reading video from remote files (probably IP cameras as well)
• SSE3 optimization has been added. That makes filtering and geometrical transformations slightly faster, single-precision FFT - ~20% faster.

9/30/09

This latest OpenCV release, 2.0, is out!

The version 2.0 features:

• the brand-new C++ interface, in addition to the existing OpenCV 1.x C API.
• a lot of new functionality, especially in the area of feature detectors and descriptors
• SSE2 optimization of many functions the revised directory structure and the completely new CMake-based build system
• completely new documentation, available
  • offline as PDF (included) and online:

  • http://opencv.willowgarage.com/documentation/index.html

and a lot more; please, see the Change Log.

Windows Installation Notes:

1. The library is better to be installed to a directory without a space in its name, such as the default C:\OpenCV2.0 - otherwise (e.g. if you install it to C:\Program Files\OpenCV2.0) you may get compile errors. This is a limitation of the current build scripts.

2. Due to many technical problems the installation package does not include pre-compiled OpenCV libraries for Visual Studio users.
   • Instead, it includes libraries built with MinGW 4.3.3 TDM-SJLJ.
     • They are good enough to run the C/C++ and Python samples and tests, but for developing your OpenCV-based applications using Visual Studio (or a Borland IDE, etc.) you need to build the libraries with your compiler using CMake, as explained here:

       • http://opencv.willowgarage.com/wiki/InstallGuide

Procedure at a glance:

•    1. Download CMake from http://www.cmake.org/cmake/resources/software.html
        and install it.
  
     2. Run CMake GUI tool and configure OpenCV there:
        2.1. select C:\OpenCV2.0 (or the installation directory you chose)
             as the source directory;
        2.2. choose some other directory name for the generated project files, e.g.
             C:\OpenCV2.0\vs2008, or D:\Work\OpenCV_MinGW etc.
        2.3. press Configure button, select your preferrable build environment
        2.4. adjust any options at your choice
        2.5. press Configure again, then press Generate.
        
     3a. In the case of Visual Studio or any other IDE, open the generated
        solution/workspace/project ..., e.g. C:\OpenCV2.0\vs2008\OpenCV.sln,
        build it in Release and Debug configurations.
     3b. In the case of command-line Makefiles, enter the destination directory
         and type "make" (or "nmake" etc.) 
        
     4. Add the output directories to the system path, e.g.:
        C:\OpenCV2.0\vs2008\bin\Debug;C:\OpenCV2.0\vs2008\bin\Release:%PATH%
        It is safe to add both directories, since the Debug
        OpenCV DLLs have the "d" suffix, which the Release DLLs do not have.
        
     5. Optionally, add C:\OpenCV2.0\include\opencv to the list of
        include directories in your IDE settings,
        and the output library directories
        (e.g. C:\OpenCV2.0\vs2008\lib\{Debug,Release})
        to the list of library paths.

  • It is important to build both release and debug configurations, and link you code with the proper libraries in each configuration, otherwise various compile-time or run-time errors are possible.

Known issues:

1. Python 2.6 bindings for OpenCV are included within the package, but not installed.
   • You can copy the subdirectory opencv/Python2.6/Lib/site-packages into the respective directory of the Python installation.
   • This procedure will be automated in the next OpenCV update.
2. Several (actually, most) SWIG-based Python samples do not work correctly now. The reason is this problem is being investigated and the intermediate update of the OpenCV Python package will be released as soon as the problem is sorted out.

3. The library has not been tested on 64-bit Windows versions. But it is known to have some limitations in the highgui part, such as the lack of FFMPEG and DirectShow support.

4. The library can be built with VS2003.NET, and all the tests pass, but DirectShow support (via videoInput library) is not included.

Differences between OpenCV-2.0.0-win32.exe and OpenCV-2.0.0a-win32.exe:

1. Visual Studio 2003.NET is now supported
2. Corrected bug in matrix routines that could produce incorrect results or even cause runtime errors.
3. A few other minor changes to sync the content with the source package for Unix.

Linux Installation Notes:

At a glance:

•    1. tar -xjf OpenCV-2.0.0.tar.bz2
     2. mkdir opencv.build
     3. cd opencv.build
     4a. cmake [<extra_options>] ../OpenCV-2.0.0 # CMake-way
        or
     4b. ../OpenCV-2.0.0/configure [<extra_options>] # configure-way
     5. make -j 2
     6. sudo make install
     7. sudo ldconfig # linux only

• For the list of prerequisities and more detailed installation guide, please, see

  • http://opencv.willowgarage.com/wiki/InstallGuide

Known issues:

• Some (actually, quite many) of SWIG-based Python bindings do not work. The problem is being investigated
• Some correctness tests occasionally fail; in 99% of cases those are known problems in the tests.
• Some tests in cvtest and mltest require test data (otherwise the respective tests will fail, which is normal), that is not included into the package to save space. The data can be retrieved from

  • http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv_extra

  • Then run cvtest as

    • ./cvtest -d <the_path_to_opencv_extra>/testdata/cv

  • and mltest as

    • ./mltest -d <the_path_to_opencv_extra>/testdata/ml

• The reference manual does not include description of most of the new functionality from cvaux. As soon as the documentation will be updated, it will be uploaded to

  • http://opencv.willowgarage.com/documentation/index.html

  • Stay tuned!

8/31/09

• More documentation improvements
  • The reference manual PDF can now be generated by OpenCV build system
  • Written ~120-page chapter on the new C++ API in cxcore.
• Added OpenMP threading to decision tree training functions, the new object detection code and the new traincascade application. On dual core machine the threaded detection code runs faster by 60%, and the traincascade is now 90% faster.
• Integrated One-Way Descriptor (implemented in the outlet detection project)

  • S. Hinterstoisser, O. Kutter, N. Navab, P. Fua, and V. Lepetit, Real-Time Learning of Accurate Patch Rectification, Conference on Computer Vision and Pattern Recognition, 2009.

• Integrated FAST corner detector (donated by the author)

  • E. Rosten and T. Drummond (May 2006). "Machine learning for high-speed corner detection,". European Conference on Computer Vision.

• Integrated Bundle Adjustment optimization engine (donated by PhaseSpace company): http://en.wikipedia.org/wiki/Bundle_adjustment

• OpenCV can now read and write gzip-compressed XML/YAML files, e.g. cvSave("bigmatrix.xml.gz", mybigmatrix);

7/22/09

• Documentation improvements

  • We now have a proof read (for English, not content) LaTeX source documentation from which we generate pdf and go from LaTeX->sphinx->html

    • This is all now up
    • It has nice search features
    • We are working on documenting all the new OpenCV features such as optional C++ interface and many new features and point finders.
  • Python documentation for the new python interface is now up too.
• We've linked in the quick links to the new python interface for OpenCV as well as posted its documentation.

• FLANN library + the OpenCV-specific frontend have been integrated. See http://people.cs.ubc.ca/~mariusm/index.php/FLANN/FLANN

• The Cascade face detector can now work with other features (its backward compatible), but now much more flexible.

  • Right now it can use the previous Haar features and also Local Binary Patterns (LBP)

         Ojala and M. Pietikainen. Multiresolution Gray-Scale and Rotation
         Invariant Texture Classification with Local Binary Patterns, IEEE Trans
         on Pattern Analysis and Machine Intelligence, Vol. 24. No.7, July, 2002.

  • But more features are on their way as well as a way for users to add features.
• Planar object detector using fern-based patch classifier and V. Lepetit keypoint detector has been added.

  •     M. Ozuysal, M. Calonder, V. Lepetit and P. Fua, Fast Keypoint Recognition using Random Ferns, accepted to IEEE Transactions on Pattern Analysis and Machine   Intelligence, 2009.

• Integrated self-similarity features (contributed by Rainer Leinhart)

  •   1. Shechtman, E., Irani, M. Matching local self-similarities across images and videos. CVPR 2007
      2. Eva Horster, Thomas Greif, Rainer Lienhart, Malcolm Slaney. Comparing Local Feature Descriptors in pLSA-Based Image Models. 30th Annual Symposium of the German Association for Pattern Recognition (DAGM) 2008, Munich, Germany, June 2008.

6/12/09

• Added new auto-documentation pages to opencv wiki.
  • These come from a LaTeX base (that will be included in the new docs directory)

    • From the LaTeX base, we will generate a pdf and go from LaTeX->sphinx->html

• Finished conversion of "cv" module to C++ interface; highgui (except for video I/O) was converted too.

5/09

• OpenCV and Willow talks at U. of Tokyo picked up by Japanese robot watch: http://robot.watch.impress.co.jp/docs/news/20090520_169668.html

• Added the new version of HOG descriptor (compatible with lear toolkit http://ralyx.inria.fr/2006/Raweb/lear/uid30.html) + people detection demo: opencv/samples/c/peopledetect.cpp

• Dense optical flow algorithm has been added (see cv::calcOpticalFlowFarneback in opencv/include/opencv/cv.hpp). This is implementation of "Two-Frame Motion Estimation Based on Polynomial Expansion" by G. Farneback.

4/09

• The new C++ API has been provided for the most of "cv" module: morpholoy, filtering, geometrical transformations, histograms, color conversions , optical flow etc. See opencv/include/opencv/cv.hpp.
• Memory footprint of tree-based training algorithms in MLL (Decision Tree, Random Trees, Boosting) has been substantially reduced (by factor of 3 at least) so that larger datasets can now be passed to train methods.
• New test module mltest has been added. It now contains the tests for MLL's tree-based algorithms.
• SSE2 optimization added for filtering functions, morphology and geometrical transformations. It mean much better performance when using MSVC2005 or higher, or GCC 4.2 or higher (SSE2 must be enabled in OpenCV's CMake config scripts)
• In-memory image encoding/decoding added in HighGUI, as well as new image I/O functions working with cv::Mat. PNG encoder performance improved by factor of 4.
• Better eye detector has been contributed by Shiqi Yu, see opencv/data/haarcascades/*[lefteye|righteye]*.xml
• Fuzzy meanshift tracker has been contributed by Farhad Dadgostar, see opencv/cvaux/src/cvfuzzymeanshifttracker.cpp

• latest_tested_snapshot tag (http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/tags/latest_tested_snapshot) has been updated.

• opencv_extra (https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv_extra/)

3/09

• Some cxcore and cv functions are optimized using SSE2 intrinsics (arithmetical & logical functions, cv::transform, DFT, filtering functions,

  • image warping (to be finished by the end of March)).

• Contributed Maximally Stable Extremal Regions (MSER) by Liu Liu

2/09

• Added "Spin images", features for describing 3D surface Andrew Johnson PhD thesis and OctTree C++ class.

• Added Extremely Random Trees (fast learning and training, high performing classification algorithm), see CvERTrees class in ml.h
• Contributed Adaptive skin detector by Farhad Dadgostar, see samples/c/adaptiveskindetector_sample.cpp

• Introduced brand-new C++ interface for cxcore (to be expanded to cv, ml & highgui):

  • IplImage, CvMat -> cv::Mat

  • cvAdd -> cv::add (can also use cv::Mat C = A + B )

  • cvReleaseMat/Image are not needed anymore, cvCreateMat/Image are often not needed either (functions [re]allocate destination images/matrices when needed).

  • C++-style XML I/O: cvWriteInt(fs, "a", 5) -> fs << "a" << 5;

  • etc.

12/08-1/09

• OpenCV repository was moved from CVS to SVN: http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary. You may be interested in using either the very latest snapshot: http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv, or some pretty recent tested snapshot:

http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/tags/latest_tested_snapshot

• Build system changed to CMake (on Linux/Unix autotools scripts will be maintained for some time too). Please, check the INSTALL document.

• Linear algebra functions (cvSolve, cvInvert, cvDet, cvEigenVV, cvSVD, cvSVBkSb) now use LAPACK (OpenCV now comes with a subset of CLapack 3.1.1.1).

Introduction

OpenCVMeetingNotes