4 Starting from OpenCV 2.0 the new modern C++ interface has been introduced.
5 It is crisp (less typing is needed to code the same thing), type-safe (no more \texttt{CvArr*} a.k.a. \texttt{void*})
6 and, in general, more convenient to use. Here is a short example of what it looks like:
10 // Simple retro-style photo effect done by adding noise to
11 // the luminance channel and reducing intensity of the chroma channels
14 // include standard OpenCV headers, same as before
18 // all the new API is put into "cv" namespace. Export its content
21 // enable/disable use of mixed API in the code below.
22 #define DEMO_MIXED_API_USE 1
24 int main( int argc, char** argv )
26 const char* imagename = argc > 1 ? argv[1] : "lena.jpg";
27 #if DEMO_MIXED_API_USE
28 // Ptr<T> is safe ref-conting pointer class
29 Ptr<IplImage> iplimg = cvLoadImage(imagename);
31 // cv::Mat replaces the CvMat and IplImage, but it's easy to convert
32 // between the old and the new data structures
33 // (by default, only the header is converted and the data is shared)
36 // the newer cvLoadImage alternative with MATLAB-style name
37 Mat img = imread(imagename);
40 if( !img.data ) // check if the image has been loaded properly
44 // convert image to YUV color space.
45 // The output image will be allocated automatically
46 cvtColor(img, img_yuv, CV_BGR2YCrCb);
48 // split the image into separate color planes
50 split(img_yuv, planes);
52 // another Mat constructor; allocates a matrix of the specified
54 Mat noise(img.size(), CV_8U);
56 // fills the matrix with normally distributed random values;
57 // there is also randu() for uniformly distributed random numbers.
58 // Scalar replaces CvScalar, Scalar::all() replaces cvScalarAll().
59 randn(noise, Scalar::all(128), Scalar::all(20));
61 // blur the noise a bit, kernel size is 3x3 and both sigma's
63 GaussianBlur(noise, noise, Size(3, 3), 0.5, 0.5);
65 const double brightness_gain = 0;
66 const double contrast_gain = 1.7;
67 #if DEMO_MIXED_API_USE
68 // it's easy to pass the new matrices to the functions that
69 // only work with IplImage or CvMat:
70 // step 1) - convert the headers, data will not be copied
71 IplImage cv_planes_0 = planes[0], cv_noise = noise;
72 // step 2) call the function; do not forget unary "&" to form pointers
73 cvAddWeighted(&cv_planes_0, contrast_gain, &cv_noise, 1,
74 -128 + brightness_gain, &cv_planes_0);
76 addWeighted(planes[0], constrast_gain, noise, 1,
77 -128 + brightness_gain, planes[0]);
79 const double color_scale = 0.5;
80 // Mat::convertTo() replaces cvConvertScale.
81 // One must explicitly specify the output matrix type
82 // (we keep it intact, i.e. pass planes[1].type())
83 planes[1].convertTo(planes[1], planes[1].type(),
84 color_scale, 128*(1-color_scale));
86 // alternative form of convertTo if we know the datatype
87 // at compile time ("uchar" here).
88 // This expression will not create any temporary arrays
89 // and should be almost as fast as the above variant
90 planes[2] = Mat_<uchar>(planes[2]*color_scale + 128*(1-color_scale));
92 // Mat::mul replaces cvMul(). Again, no temporary arrays are
93 // created in the case of simple expressions.
94 planes[0] = planes[0].mul(planes[0], 1./255);
96 // now merge the results back
97 merge(planes, img_yuv);
98 // and produce the output RGB image
99 cvtColor(img_yuv, img, CV_YCrCb2BGR);
101 // this is counterpart for cvNamedWindow
102 namedWindow("image with grain", CV_WINDOW_AUTOSIZE);
103 #if DEMO_MIXED_API_USE
104 // this is to demonstrate that img and iplimg really share the data -
105 // the result of the above processing is stored to img and thus
107 cvShowImage("image with grain", iplimg);
109 imshow("image with grain", img);
114 // all the memory will automatically be released
115 // by vector<>, Mat and Ptr<> destructors.
119 Following a summary "cheatsheet" below, the rest of the introduction will discuss the key features of the new interface in more detail.
123 \section{C++ Cheatsheet}\label{cheatSheet}
124 The section is just a summary "cheatsheet" of common things you may want to do with cv::Mat:. The code snippets below all assume the correct
132 Convert an IplImage or CvMat to an cv::Mat and a cv::Mat to an IplImage or CvMat:
134 // Assuming somewhere IplImage *iplimg; exists
135 // and has been allocated and cv::Mat Mimg has been defined
136 Mat imgMat(iplimg); //Construct an Mat image "img" out of an IplImage
137 Mimg = iplimg; //Or just set the header of pre existing cv::Mat
138 //Ming to iplimg's data (no copying is done)
140 //Convert to IplImage or CvMat, no data copying
141 IplImage ipl_img = img;
142 CvMat cvmat = img; // convert cv::Mat -> CvMat
145 A very simple way to operate on a rectanglular sub-region of an image (ROI -- "Region of Interest"):
148 Rect roi(10, 20, 100, 50);
149 //Point a cv::Mat header at it (no allocation is done)
150 Mat image_roi = image(roi);
153 A bit advanced, but should you want efficiently to sample from a circular region in an image
154 (below, instead of sampling, we just draw into a BGR image) :
157 // the function returns x boundary coordinates of
158 // the circle for each y. RxV[y1] = x1 means that
159 // when y=y1, -x1 <=x<=x1 is inside the circle
160 void getCircularROI(int R, vector < int > & RxV)
163 for( int y = 0; y <= R; y++ )
164 RxV[y] = cvRound(sqrt((double)R*R - y*y));
167 // This draws a circle in the green channel
168 // (note the "[1]" for a BGR" image,
169 // blue and red channels are not modified),
170 // but is really an example of how to *sample* from a circular region.
171 void drawCircle(Mat &image, int R, Point center)
174 getCircularROI(R, RxV);
176 Mat_<Vec3b>& img = (Mat_<Vec3b>&)image; //3 channel pointer to image
177 for( int dy = -R; dy <= R; dy++ )
179 int Rx = RxV[abs(dy)];
180 for( int dx = -Rx; dx <= Rx; dx++ )
181 img(center.y+dy, center.x+dx)[1] = 255;
186 \section{Namespace \texttt{cv} and Function Naming}
188 All the newly introduced classes and functions are placed into \texttt{cv} namespace. Therefore, to access this functionality from your code, use \texttt{cv::} specifier or \texttt{"using namespace cv;"} directive:
193 cv::Mat H = cv::findHomography(points1, points2, cv::RANSAC, 5);
203 Mat H = findHomography(points1, points2, RANSAC, 5 );
207 It is probable that some of the current or future OpenCV external names conflict with STL
208 or other libraries, in this case use explicit namespace specifiers to resolve the name conflicts:
210 Mat a(100, 100, CV_32F);
211 randu(a, Scalar::all(1), Scalar::all(std::rand()%256+1));
216 For the most of the C functions and structures from OpenCV 1.x you may find the direct counterparts in the new C++ interface. The name is usually formed by omitting \texttt{cv} or \texttt{Cv} prefix and turning the first letter to the low case (unless it's a own name, like Canny, Sobel etc). In case when there is no the new-style counterpart, it's possible to use the old functions with the new structures, as shown the first sample in the chapter.
218 \section{Memory Management}
220 When using the new interface, the most of memory deallocation and even memory allocation operations are done automatically when needed.
222 First of all, \cross{Mat}, \cross{SparseMat} and other classes have destructors
223 that deallocate memory buffers occupied by the structures when needed.
225 Secondly, this "when needed" means that the destructors do not always deallocate the buffers, they take into account possible data sharing.
226 That is, in a destructor the reference counter associated with the underlying data is decremented and the data is deallocated
227 if and only if the reference counter becomes zero, that is, when no other structures refer to the same buffer. When such a structure
228 containing a reference counter is copied, usually just the header is duplicated, while the underlying data is not; instead, the reference counter is incremented to memorize that there is another owner of the same data.
229 Also, some structures, such as \texttt{Mat}, can refer to the user-allocated data.
230 In this case the reference counter is \texttt{NULL} pointer and then no reference counting is done - the data is not deallocated by the destructors and should be deallocated manually by the user. We saw this scheme in the first example in the chapter:
232 // allocates IplImages and wraps it into shared pointer class.
233 Ptr<IplImage> iplimg = cvLoadImage(...);
235 // constructs Mat header for IplImage data;
236 // does not copy the data;
237 // the reference counter will be NULL
240 // in the end of the block img destructor is called,
241 // which does not try to deallocate the data because
242 // of NULL pointer to the reference counter.
244 // Then Ptr<IplImage> destructor is called that decrements
245 // the reference counter and, as the counter becomes 0 in this case,
246 // the destructor calls cvReleaseImage().
249 The copying semantics was mentioned in the above paragraph, but deserves a dedicated discussion.
250 By default, the new OpenCV structures implement shallow, so called O(1) (i.e. constant-time) assignment operations. It gives user possibility to pass quite big data structures to functions (though, e.g. passing \texttt{const Mat\&} is still faster than passing \texttt{Mat}), return them (e.g. see the example with \cross{findHomography} above), store them in OpenCV and STL containers etc. - and do all of this very efficiently. On the other hand, most of the new data structures provide \texttt{clone()} method that creates a full copy of an object. Here is the sample:
252 // create a big 8Mb matrix
253 Mat A(1000, 1000, CV_64F);
255 // create another header for the same matrix;
256 // this is instant operation, regardless of the matrix size.
258 // create another header for the 3-rd row of A; no data is copied either
260 // now create a separate copy of the matrix
262 // copy the 5-th row of B to C, that is, copy the 5-th row of A
263 // to the 3-rd row of A.
265 // now let A and D share the data; after that the modified version
266 // of A is still referenced by B and C.
268 // now make B an empty matrix (which references no memory buffers),
269 // but the modified version of A will still be referenced by C,
270 // despite that C is just a single row of the original A
273 // finally, make a full copy of C. In result, the big modified
274 // matrix will be deallocated, since it's not referenced by anyone
278 Memory management of the new data structures is automatic and thus easy. If, however, your code uses \cross{IplImage},
279 \cross{CvMat} or other C data structures a lot, memory management can still be automated without immediate migration
280 to \cross{Mat} by using the already mentioned template class \cross{Ptr}, similar to \texttt{shared\_ptr} from Boost and C++ TR1.
281 It wraps a pointer to an arbitrary object, provides transparent access to all the object fields and associates a reference counter with it.
282 Instance of the class can be passed to any function that expects the original pointer. For correct deallocation of the object, you should specialize \texttt{Ptr<T>::delete\_obj()} method. Such specialized methods already exist for the classical OpenCV structures, e.g.:
286 template<> inline Ptr<IplImage>::delete_obj() {
287 cvReleaseImage(&obj);
291 See \cross{Ptr} description for more details and other usage scenarios.
294 \section{Memory Management Part II. Automatic Data Allocation}\label{AutomaticMemoryManagement2}
296 With the new interface not only explicit memory deallocation is not needed anymore,
297 but the memory allocation is often done automatically too. That was demonstrated in the example
298 in the beginning of the chapter when \texttt{cvtColor} was called, and here are some more details.
300 \cross{Mat} and other array classes provide method \texttt{create} that allocates a new buffer for array
301 data if and only if the currently allocated array is not of the required size and type.
302 If a new buffer is needed, the previously allocated buffer is released
303 (by engaging all the reference counting mechanism described in the previous section).
304 Now, since it is very quick to check whether the needed memory buffer is already allocated,
305 most new OpenCV functions that have arrays as output parameters call the \texttt{create} method and
306 this way the \emph{automatic data allocation} concept is implemented. Here is the example:
313 int main(int, char**)
316 if(!cap.isOpened()) return -1;
319 namedWindow("edges",1);
324 cvtColor(frame, edges, CV_BGR2GRAY);
325 GaussianBlur(edges, edges, Size(7,7), 1.5, 1.5);
326 Canny(edges, edges, 0, 30, 3);
327 imshow("edges", edges);
328 if(waitKey(30) >= 0) break;
333 The matrix \texttt{edges} is allocated during the first frame processing and unless the resolution will suddenly change,
334 the same buffer will be reused for every next frame's edge map.
336 In many cases the output array type and size can be inferenced from the input arrays' respective characteristics, but not always.
337 In these rare cases the corresponding functions take separate input parameters that specify the data type and/or size of the output arrays,
338 like \cross{resize}. Anyway, a vast majority of the new-style array processing functions call \texttt{create}
339 for each of the output array, with just a few exceptions like \texttt{mixChannels}, \texttt{RNG::fill} and some others.
341 Note that this output array allocation semantic is only implemented in the new functions. If you want to pass the new structures to some old OpenCV function, you should first allocate the output arrays using \texttt{create} method, then make \texttt{CvMat} or \texttt{IplImage} headers and after that call the function.
343 \section{Algebraic Operations}
345 Just like in v1.x, OpenCV 2.x provides some basic functions operating on matrices, like \texttt{add},
346 \texttt{subtract}, \texttt{gemm} etc. In addition, it introduces overloaded operators that give the user a convenient
347 algebraic notation, which is nearly as fast as using the functions directly. For example, here is how the least squares problem $Ax=b$
348 can be solved using normal equations:
350 Mat x = (A.t()*A).inv()*(A.t()*b);
353 The complete list of overloaded operators can be found in \cross{Matrix Expressions}.
355 \section{Fast Element Access}
357 Historically, OpenCV provided many different ways to access image and matrix elements, and none of them was both fast and convenient.
358 With the new data structures, OpenCV 2.x introduces a few more alternatives, hopefully more convenient than before. For detailed description of the operations, please, check \cross{Mat} and \hyperref[MatT]{Mat\_} description. Here is part of the retro-photo-styling example rewritten (in simplified form) using the element access operations:
362 // split the image into separate color planes
364 split(img_yuv, planes);
366 // method 1. process Y plane using an iterator
367 MatIterator_<uchar> it = planes[0].begin<uchar>(),
368 it_end = planes[0].end<uchar>();
369 for(; it != it_end; ++it)
371 double v = *it*1.7 + rand()%21-10;
372 *it = saturate_cast<uchar>(v*v/255.);
375 // method 2. process the first chroma plane using pre-stored row pointer.
376 // method 3. process the second chroma plane using
377 // individual element access operations
378 for( int y = 0; y < img_yuv.rows; y++ )
380 uchar* Uptr = planes[1].ptr<uchar>(y);
381 for( int x = 0; x < img_yuv.cols; x++ )
383 Uptr[x] = saturate_cast<uchar>((Uptr[x]-128)/2 + 128);
384 uchar& Vxy = planes[2].at<uchar>(y, x);
385 Vxy = saturate_cast<uchar>((Vxy-128)/2 + 128);
389 merge(planes, img_yuv);
394 \section{Saturation Arithmetics}
396 In the above sample you may have noticed \hyperref[saturatecast]{saturate\_cast} operator, and that's how all the pixel processing is done in OpenCV. When a result of image operation is 8-bit image with pixel values ranging from 0 to 255, each output pixel value is clipped to this available range:
399 I(x,y)=\min(\max(value, 0), 255)
402 and the similar rules are applied to 8-bit signed and 16-bit signed and unsigned types. This "saturation" semantics (different from usual C language "wrapping" semantics, where lowest bits are taken, is implemented in every image processing function, from the simple \texttt{cv::add} to
403 \texttt{cv::cvtColor}, \texttt{cv::resize}, \texttt{cv::filter2D} etc.
404 It is not a new feature of OpenCV v2.x, it was there from very beginning. In the new version this special \hyperref[saturatecast]{saturate\_cast} template operator is introduced to simplify implementation of this semantic in your own functions.
407 \section{Error handling}
409 The modern error handling mechanism in OpenCV uses exceptions, as opposite to the manual stack unrolling used in previous versions. When OpenCV is built in DEBUG configuration, the error handler provokes memory access violation, so that the full call stack and context can be analyzed with debugger.
411 \section{Threading and Reenterability}
413 OpenCV uses OpenMP to run some time-consuming operations in parallel. Threading can be explicitly controlled by \cross{setNumThreads} function. Also, functions and "const" methods of the classes are generally re-enterable, that is, they can be called from different threads asynchronously.
418 \chapter{Introduction}
420 Starting with release 2.0, OpenCV has a new Python interface. This replaces the previous
421 \href{http://opencv.willowgarage.com/wiki/SwigPythonInterface}{SWIG-based Python interface}.
423 Some highlights of the new bindings:
426 \item{single import of all of OpenCV using \texttt{import cv}}
427 \item{OpenCV functions no longer have the "cv" prefix}
428 \item{simple types like CvRect and CvScalar use Python tuples}
429 \item{sharing of Image storage, so image transport between OpenCV and other systems (e.g. numpy and ROS) is very efficient}
430 \item{complete documentation for the Python functions}
437 Here is a collection of code fragments demonstrating some features
438 of the OpenCV Python bindings.
440 \subsection{Convert an image}
444 >>> im = cv.LoadImageM("building.jpg")
447 >>> cv.SaveImage("foo.png", im)
450 \subsection{Resize an image}
452 To resize an image in OpenCV, create a destination image of the appropriate size, then call \cross{Resize}.
456 >>> original = cv.LoadImageM("building.jpg")
457 >>> thumbnail = cv.CreateMat(original.rows / 10, original.cols / 10, cv.CV_8UC3)
458 >>> cv.Resize(original, thumbnail)
461 \subsection{Compute the Laplacian}
465 >>> im = cv.LoadImageM("building.jpg", 1)
466 >>> dst = cv.CreateImage(cv.GetSize(im), cv.IPL_DEPTH_16S, 3)
467 >>> laplace = cv.Laplace(im, dst)
468 >>> cv.SaveImage("foo-laplace.png", dst)
472 \subsection{Using GoodFeaturesToTrack}
474 To find the 10 strongest corner features in an image, use \cross{GoodFeaturesToTrack} like this:
478 >>> img = cv.LoadImageM("building.jpg", cv.CV_LOAD_IMAGE_GRAYSCALE)
479 >>> eig_image = cv.CreateMat(img.rows, img.cols, cv.CV_32FC1)
480 >>> temp_image = cv.CreateMat(img.rows, img.cols, cv.CV_32FC1)
481 >>> for (x,y) in cv.GoodFeaturesToTrack(img, eig_image, temp_image, 10, 0.04, 1.0, useHarris = True):
482 ... print "good feature at", x,y
483 good feature at 198.0 514.0
484 good feature at 791.0 260.0
485 good feature at 370.0 467.0
486 good feature at 374.0 469.0
487 good feature at 490.0 520.0
488 good feature at 262.0 278.0
489 good feature at 781.0 134.0
490 good feature at 3.0 247.0
491 good feature at 667.0 321.0
492 good feature at 764.0 304.0
496 \subsection{Using GetSubRect}
498 GetSubRect returns a rectangular part of another image. It does this without copying any data.
502 >>> img = cv.LoadImageM("building.jpg")
503 >>> sub = cv.GetSubRect(img, (60, 70, 32, 32)) # sub is 32x32 patch within img
504 >>> cv.SetZero(sub) # clear sub to zero, which also clears 32x32 pixels in img
507 \subsection{Using CreateMat, and accessing an element}
511 >>> mat = cv.CreateMat(5, 5, cv.CV_32FC1)
513 >>> mat[3,1] += 0.375
516 >>> print [mat[3,i] for i in range(5)]
517 [1.0, 1.375, 1.0, 1.0, 1.0]
520 \subsection{ROS image message to OpenCV}
522 See this tutorial: \href{http://www.ros.org/wiki/cv\_bridge/Tutorials/UsingCvBridgeToConvertBetweenROSImagesAndOpenCVImages}{Using CvBridge to convert between ROS images And OpenCV images}.
524 \subsection{PIL Image to OpenCV}
526 (For details on PIL see the \href{http://www.pythonware.com/library/pil/handbook/image.htm}{PIL handbook}.)
530 >>> pi = Image.open('building.jpg') # PIL image
531 >>> cv_im = cv.CreateImageHeader(pi.size, cv.IPL_DEPTH_8U, 3)
532 >>> cv.SetData(cv_im, pi.tostring())
533 >>> print pi.size, cv.GetSize(cv_im)
534 (868, 600) (868, 600)
535 >>> print pi.tostring() == cv_im.tostring()
539 \subsection{OpenCV to PIL Image}
543 >>> cv_im = cv.CreateImage((320,200), cv.IPL_DEPTH_8U, 1)
544 >>> pi = Image.fromstring("L", cv.GetSize(cv_im), cv_im.tostring())
549 \subsection{NumPy and OpenCV}
551 Using the \href{http://docs.scipy.org/doc/numpy/reference/arrays.interface.html}{array interface}, to use an OpenCV CvMat in NumPy:
555 >>> mat = cv.CreateMat(3, 5, cv.CV_32FC1)
557 >>> a = numpy.asarray(mat)
564 and to use a NumPy array in OpenCV:
568 >>> a = numpy.ones((480, 640))
569 >>> mat = cv.fromarray(a)
576 also, most OpenCV functions can work on NumPy arrays directly, for example:
579 >>> picture = numpy.ones((640, 480))
580 >>> cv.Smooth(picture, picture, cv.CV_GAUSSIAN, 15, 15)
584 the \cross{fromarray} function (or the implicit version shown above)
585 returns a single-channel \cross{CvMat} of the same size.
586 For a 3D array of size $j \times k \times l$, it returns a
587 \cross{CvMat} sized $j \times k$ with $l$ channels.
589 Alternatively, use \cross{fromarray} with the \texttt{allowND} option to always return a \cross{cvMatND}.