X-Git-Url: http://rtime.felk.cvut.cz/gitweb/hercules2020/kcf.git/blobdiff_plain/0adf70c02c30d0f74db1f060e2dd40165ae29fa7..03f7830ef615201e13aeda2c092f8979c488f951:/README.md diff --git a/README.md b/README.md index 537bec0..1e2d5d9 100644 --- a/README.md +++ b/README.md @@ -1,92 +1,240 @@ -## Tracking with Kernelized Correlation Filters +# KCF tracker – parallel and PREM implementations + +The goal of this project is modify KCF tracker for use in the +[HERCULES][1] project, where it will run on NVIDIA TX2 board. To +achieve the needed performance we try various ways of parallelization +of the algorithm including execution on the GPU. The aim is also to +modify the code according to the PRedictable Execution Model (PREM). + +Stable version of the tracker is available from a [CTU server][2], +development happens at [GitHub][iig]. -Code author : Tomas Vojir +[1]: http://hercules2020.eu/ +[2]: http://rtime.felk.cvut.cz/gitweb/hercules2020/kcf.git +[iig]: https://github.com/CTU-IIG/kcf +[3]: https://github.com/Shanigen/kcf + + +**Table of Contents** -________________ +- [Prerequisites](#prerequisites) +- [Compilation](#compilation) + - [Compile all supported versions](#compile-all-supported-versions) + - [Using cmake gui](#using-cmake-gui) + - [Command line](#command-line) +- [Running](#running) + - [Options](#options) +- [Automated testing](#automated-testing) +- [Authors](#authors) +- [References](#references) +- [License](#license) + + + + +## Prerequisites + +The code depends on OpenCV (version 2.4 or 3.x) library. [CMake][13] +(optionally with [Ninja][8]) is used for building. Depending on the +version to be compiled you need to have development packages for +[FFTW][4], [CUDA][5] or [OpenMP][6] installed. -This is a C++ reimplementation of algorithm presented in "High-Speed Tracking with Kernelized Correlation Filters" paper. -For more info and implementation in other languages visit the [autor's webpage!](http://home.isr.uc.pt/~henriques/circulant/). +On TX2, the following command should install what's needed: +``` shellsession +$ apt install cmake ninja-build libopencv-dev libfftw3-dev +``` -It is free for research use. If you find it useful or use it in your research, please acknowledge my git repository -and cite the original paper [1]. +[4]: http://www.fftw.org/ +[5]: https://developer.nvidia.com/cuda-downloads +[6]: http://www.openmp.org/ +[13]: https://cmake.org/ -The code depends on OpenCV 2.4+ library and is build via cmake toolchain. +## Compilation + +There are multiple ways how to compile the code. + +### Compile all supported versions + +``` shellsession +$ git submodule update --init +$ make -k +``` -_________________ -Quick start guide +This will create several `build-*` directories and compile different +versions in them. If prerequisites of some builds are missing, the +`-k` option ensures that the errors are ignored. This uses [Ninja][8] +build system, which is useful when building naively on TX2, because +builds with `ninja` are faster (better parallelized) than with `make`. -for linux: open terminal in the directory with the code +To build only a specific version run `make `. For example, +CUDA-based version can be compiled with: -$ mkdir build; cd build; cmake .. ; make +``` shellsession +$ make cufft +``` -This code compiles into binary **kcf_vot** +[8]: https://ninja-build.org/ -./kcf_vot -- using VOT 2014 methodology (http://www.votchallenge.net/) - - INPUT : expecting two files, images.txt (list of sequence images with absolute path) and - region.txt with initial bounding box in the first frame in format "top_left_x, top_left_y, width, height" or - four corner points listed clockwise starting from bottom left corner. - - OUTPUT : output.txt containing the bounding boxes in the format "top_left_x, top_left_y, width, height" +### Using cmake gui -__________ -References +```shellsession +$ git submodule update --init +$ mkdir build +$ cmake-gui . +``` -[1] João F. Henriques, Rui Caseiro, Pedro Martins, Jorge Batista, “High-Speed Tracking with Kernelized Correlation Filters“, -IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015 +- Use the just created build directory as "Where to build the + binaries". +- Press "Configure". +- Choose desired build options. Each option has a comment briefly + explaining what it does. +- Press "Generate" and close the window. -_____________________________________ -Copyright (c) 2014, Tomáš Vojíř +```shellsession +$ make -C build +``` +### Command line -Permission to use, copy, modify, and distribute this software for research -purposes is hereby granted, provided that the above copyright notice and -this permission notice appear in all copies. +```shellsession +$ git submodule update --init +$ mkdir build +$ cd build +$ cmake [options] .. # see the tables below +$ make +``` -THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES -WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF -MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR -ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES -WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN -ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF -OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +The `cmake` options below allow to select, which version to build. + +The following table shows how to configure different FFT +implementations. + +|Option| Description | +| --- | --- | +| `-DFFT=OpenCV` | Use OpenCV to calculate FFT.| +| `-DFFT=fftw` | Use fftw and its `plan_many` and "New-array execute" functions. If `std::async`, OpenMP or cuFFTW is not used the plans will use 2 threads by default.| +| `-DFFT=cuFFTW` | Use cuFFTW interface to cuFFT library.| +| `-DFFT=cuFFT` | Use cuFFT. This version also uses pure CUDA implementation of `ComplexMat` class and Gaussian correlation.| + +With all of these FFT version additional options can be added: + +|Option| Description | +| --- | --- | +| `-DBIG_BATCH=ON` | Concatenate matrices of different scales to one big matrix and perform all computations on this matrix. This improves performance of GPU FFT offloading. | +| `-DOPENMP=ON` | Parallelize certain operation with OpenMP. With `-DBIG_BATCH=OFF` it runs computations for differenct scales in parallel, with `-DBIG_BATCH=ON` it parallelizes the feature extraction, which runs on the CPU. With `fftw`, Ffftw's plans will execute in parallel.| +| `-DCUDA_DEBUG=ON` | Adds calls cudaDeviceSynchronize after every CUDA function and kernel call.| +| `-DOpenCV_DIR=/opt/opencv-3.3/share/OpenCV` | Compile against a custom OpenCV version. | +| `-DASYNC=ON` | Use C++ `std::async` to run computations for different scales in parallel. This mode of parallelization was present in the original implementation. Here, it is superseeded with -DOPENMP. This doesn't work with `BIG_BATCH` mode.| + +See also the top-level `Makefile` for other useful cmake parameters +such as extra compiler flags etc. + +## Running + +No matter which method is used to compile the code, the result will be +a `kcf_vot` binary. + +It operates on an image sequence created according to [VOT 2014 +methodology][10]. You can find some image sequences in [vot2016 +datatset][11]. + +The binary can be run as follows: + +1. `./kcf_vot [options]` + + The program looks for `groundtruth.txt` or `region.txt` and + `images.txt` files in current directory. + + - `images.txt` contains a list of images to process, each on a + separate line. + - `groundtruth.txt` contains the correct location of the tracked + object in each image as four corner points listed clockwise + starting from bottom left corner. Only the first line from this + file is used. + - `region.txt` is an alternative way of specifying the location of + the object to track via its bounding box (top_left_x, top_left_y, + width, height) in the first frame. + +2. `./kcf_vot [options] ` + + Looks for `groundtruth.txt` or `region.txt` and `images.txt` files + in the given `directory`. + +3. `./kcf_vot [options] [path/to/output.txt]` -__________________ -Additional Library +By default the program generates file `output.txt` containing the +bounding boxes of the tracked object in the format "top_left_x, +top_left_y, width, height". -NOTE: The following files are part of Piotr's Toolbox, and may had been modified for usage with c++ +[10]: http://www.votchallenge.net/ +[11]: http://www.votchallenge.net/vot2016/dataset.html - src/piotr_fhog/gradientMex.cpp - src/piotr_fhog/sse.hpp - src/piotr_fhog/wrappers.hpp - -You are encouraged to get the full version of this library at -http://vision.ucsd.edu/~pdollar/toolbox/doc/index.html +### Options -______________________________________________________________________________ +| Options | Description | +| ------- | ----------- | +| --fit, -f[W[xH]] | Specifies the dimension to which the extracted patches should be scaled. Best performance is achieved for powers of two; the smaller number the higher performance but worse accuracy. No dimension or zero rounds the dimensions to the nearest smaller power of 2, a single dimension `W` will result in patch size of `W`×`W`. The numbers should be divisible by 4. | +| --visualize, -v[delay_ms] | Visualize the output, optionally with specified delay. If the delay is 0 the program will wait for a key press. | +| --output, -o | Specify name of output file. | +| --debug, -d | Generate debug output. | +| --visual_debug, -p[p\|r] | Show graphical window with debugging information (either **p**atch or filter **r**esponse). | -Copyright (c) 2012, Piotr Dollar -All rights reserved. +## Automated testing -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: +The tracker comes with a test suite based on [vot2016 datatset][11]. +You can run the test suite as follows: -1. Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. -2. Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. + make vot2016 # This downloads the dataset (about 1GB of data) + make test -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR -ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES -(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +The above command run all tests in parallel and displays the results +in a table. If you want to measure performance, do not run multiple +tests together. This can be achieved by: -The views and conclusions contained in the software and documentation are those -of the authors and should not be interpreted as representing official policies, -either expressed or implied, of the FreeBSD Project. + make build.ninja + ninja -j1 test + +You can test only a subset of builds or image sequences by setting +BUILDS, TESTSEQ or TESTFLAGS make variables. For instance: + + make build.ninja BUILDS="cufft cufft-big fftw" TESTSEQ="bmx ball1" + ninja test + + + + +## Authors +* Vít Karafiát, Michal Sojka + +[Original C++ implementation of the KCF tracker][12] was written by +Tomas Vojir and is reimplementation of the algorithm presented in +"High-Speed Tracking with Kernelized Correlation Filters" paper \[1]. + +[12]: https://github.com/vojirt/kcf/blob/master/README.md + +## References + +\[1] João F. Henriques, Rui Caseiro, Pedro Martins, Jorge Batista, +“High-Speed Tracking with Kernelized Correlation Filters“, IEEE +Transactions on Pattern Analysis and Machine Intelligence, 2015 + +## License + +Copyright (c) 2014, Tomáš Vojíř\ +Copyright (c) 2018, Vít Karafiát\ +Copyright (c) 2018, Michal Sojka + +Permission to use, copy, modify, and distribute this software for research +purposes is hereby granted, provided that the above copyright notice and +this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + + +