init - 初始化项目

This commit is contained in:
Lee Nony
2022-05-06 01:58:53 +08:00
commit 90a5cc7cb6
6772 changed files with 2837787 additions and 0 deletions

View File

@@ -0,0 +1,4 @@
High Level GUI and Media (highgui module) {#tutorial_table_of_content_highgui}
=========================================
Content has been moved to this page: @ref tutorial_table_of_content_app

View File

@@ -0,0 +1,4 @@
Image Input and Output (imgcodecs module) {#tutorial_table_of_content_imgcodecs}
=========================================
Content has been moved to this page: @ref tutorial_table_of_content_app

View File

@@ -0,0 +1,4 @@
Video Input and Output (videoio module) {#tutorial_table_of_content_videoio}
=========================================
Content has been moved to this page: @ref tutorial_table_of_content_app

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 135 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -0,0 +1,85 @@
Using Creative Senz3D and other Intel RealSense SDK compatible depth sensors {#tutorial_intelperc}
=======================================================================================
@tableofcontents
@prev_tutorial{tutorial_orbbec_astra}
**Note**: This tutorial is partially obsolete since PerC SDK has been replaced with RealSense SDK
Depth sensors compatible with Intel® RealSense SDK are supported through VideoCapture
class. Depth map, RGB image and some other formats of output can be retrieved by using familiar
interface of VideoCapture.
In order to use depth sensor with OpenCV you should do the following preliminary steps:
-# Install Intel RealSense SDK 2.0 (from here <https://github.com/IntelRealSense/librealsense>).
-# Configure OpenCV with Intel RealSense SDK support by setting WITH_LIBREALSENSE flag in
CMake. If Intel RealSense SDK is found in install folders OpenCV will be built with
Intel Realsense SDK library (see a status LIBREALSENSE in CMake log).
-# Build OpenCV.
VideoCapture can retrieve the following data:
-# data given from depth generator:
- CAP_INTELPERC_DEPTH_MAP - each pixel is a 16-bit integer. The value indicates the
distance from an object to the camera's XY plane or the Cartesian depth. (CV_16UC1)
- CAP_INTELPERC_UVDEPTH_MAP - each pixel contains two 32-bit floating point values in
the range of 0-1, representing the mapping of depth coordinates to the color
coordinates. (CV_32FC2)
- CAP_INTELPERC_IR_MAP - each pixel is a 16-bit integer. The value indicates the
intensity of the reflected laser beam. (CV_16UC1)
-# data given from RGB image generator:
- CAP_INTELPERC_IMAGE - color image. (CV_8UC3)
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. :
@code{.cpp}
VideoCapture capture( CAP_REALSENSE );
for(;;)
{
Mat depthMap;
capture >> depthMap;
if( waitKey( 30 ) >= 0 )
break;
}
@endcode
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. :
@code{.cpp}
VideoCapture capture(CAP_REALSENSE);
for(;;)
{
Mat depthMap;
Mat image;
Mat irImage;
capture.grab();
capture.retrieve( depthMap, CAP_INTELPERC_DEPTH_MAP );
capture.retrieve( image, CAP_INTELPERC_IMAGE );
capture.retrieve( irImage, CAP_INTELPERC_IR_MAP);
if( waitKey( 30 ) >= 0 )
break;
}
@endcode
For setting and getting some property of sensor\` data generators use VideoCapture::set and
VideoCapture::get methods respectively, e.g. :
@code{.cpp}
VideoCapture capture(CAP_REALSENSE);
capture.set( CAP_INTELPERC_DEPTH_GENERATOR | CAP_PROP_INTELPERC_PROFILE_IDX, 0 );
cout << "FPS " << capture.get( CAP_INTELPERC_DEPTH_GENERATOR+CAP_PROP_FPS ) << endl;
@endcode
Since two types of sensor's data generators are supported (image generator and depth generator),
there are two flags that should be used to set/get property of the needed generator:
- CAP_INTELPERC_IMAGE_GENERATOR -- a flag for access to the image generator properties.
- CAP_INTELPERC_DEPTH_GENERATOR -- a flag for access to the depth generator properties. This
flag value is assumed by default if neither of the two possible values of the property is set.
For more information please refer to the example of usage
[videocapture_intelperc.cpp](https://github.com/opencv/opencv/tree/master/samples/cpp/videocapture_intelperc.cpp)
in opencv/samples/cpp folder.

View File

@@ -0,0 +1,144 @@
Using Kinect and other OpenNI compatible depth sensors {#tutorial_kinect_openni}
======================================================
@tableofcontents
@prev_tutorial{tutorial_video_write}
@next_tutorial{tutorial_orbbec_astra}
Depth sensors compatible with OpenNI (Kinect, XtionPRO, ...) are supported through VideoCapture
class. Depth map, BGR image and some other formats of output can be retrieved by using familiar
interface of VideoCapture.
In order to use depth sensor with OpenCV you should do the following preliminary steps:
-# Install OpenNI library (from here <http://www.openni.org/downloadfiles>) and PrimeSensor Module
for OpenNI (from here <https://github.com/avin2/SensorKinect>). The installation should be done
to default folders listed in the instructions of these products, e.g.:
@code{.text}
OpenNI:
Linux & MacOSX:
Libs into: /usr/lib
Includes into: /usr/include/ni
Windows:
Libs into: c:/Program Files/OpenNI/Lib
Includes into: c:/Program Files/OpenNI/Include
PrimeSensor Module:
Linux & MacOSX:
Bins into: /usr/bin
Windows:
Bins into: c:/Program Files/Prime Sense/Sensor/Bin
@endcode
If one or both products were installed to the other folders, the user should change
corresponding CMake variables OPENNI_LIB_DIR, OPENNI_INCLUDE_DIR or/and
OPENNI_PRIME_SENSOR_MODULE_BIN_DIR.
-# Configure OpenCV with OpenNI support by setting WITH_OPENNI flag in CMake. If OpenNI is found
in install folders OpenCV will be built with OpenNI library (see a status OpenNI in CMake log)
whereas PrimeSensor Modules can not be found (see a status OpenNI PrimeSensor Modules in CMake
log). Without PrimeSensor module OpenCV will be successfully compiled with OpenNI library, but
VideoCapture object will not grab data from Kinect sensor.
-# Build OpenCV.
VideoCapture can retrieve the following data:
-# data given from depth generator:
- CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1)
- CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3)
- CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1)
- CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1)
- CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not occluded, not shaded etc.)
(CV_8UC1)
-# data given from BGR image generator:
- CAP_OPENNI_BGR_IMAGE - color image (CV_8UC3)
- CAP_OPENNI_GRAY_IMAGE - gray image (CV_8UC1)
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. :
@code{.cpp}
VideoCapture capture( CAP_OPENNI );
for(;;)
{
Mat depthMap;
capture >> depthMap;
if( waitKey( 30 ) >= 0 )
break;
}
@endcode
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. :
@code{.cpp}
VideoCapture capture(0); // or CAP_OPENNI
for(;;)
{
Mat depthMap;
Mat bgrImage;
capture.grab();
capture.retrieve( depthMap, CAP_OPENNI_DEPTH_MAP );
capture.retrieve( bgrImage, CAP_OPENNI_BGR_IMAGE );
if( waitKey( 30 ) >= 0 )
break;
}
@endcode
For setting and getting some property of sensor\` data generators use VideoCapture::set and
VideoCapture::get methods respectively, e.g. :
@code{.cpp}
VideoCapture capture( CAP_OPENNI );
capture.set( CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE, CAP_OPENNI_VGA_30HZ );
cout << "FPS " << capture.get( CAP_OPENNI_IMAGE_GENERATOR+CAP_PROP_FPS ) << endl;
@endcode
Since two types of sensor's data generators are supported (image generator and depth generator),
there are two flags that should be used to set/get property of the needed generator:
- CAP_OPENNI_IMAGE_GENERATOR -- A flag for access to the image generator properties.
- CAP_OPENNI_DEPTH_GENERATOR -- A flag for access to the depth generator properties. This flag
value is assumed by default if neither of the two possible values of the property is not set.
Some depth sensors (for example XtionPRO) do not have image generator. In order to check it you can
get CAP_OPENNI_IMAGE_GENERATOR_PRESENT property.
@code{.cpp}
bool isImageGeneratorPresent = capture.get( CAP_PROP_OPENNI_IMAGE_GENERATOR_PRESENT ) != 0; // or == 1
@endcode
Flags specifying the needed generator type must be used in combination with particular generator
property. The following properties of cameras available through OpenNI interfaces are supported:
- For image generator:
- CAP_PROP_OPENNI_OUTPUT_MODE -- Three output modes are supported: CAP_OPENNI_VGA_30HZ
used by default (image generator returns images in VGA resolution with 30 FPS),
CAP_OPENNI_SXGA_15HZ (image generator returns images in SXGA resolution with 15 FPS) and
CAP_OPENNI_SXGA_30HZ (image generator returns images in SXGA resolution with 30 FPS, the
mode is supported by XtionPRO Live); depth generator's maps are always in VGA resolution.
- For depth generator:
- CAP_PROP_OPENNI_REGISTRATION -- Flag that registers the remapping depth map to image map
by changing depth generator's view point (if the flag is "on") or sets this view point to
its normal one (if the flag is "off"). The registration processs resulting images are
pixel-aligned,which means that every pixel in the image is aligned to a pixel in the depth
image.
Next properties are available for getting only:
- CAP_PROP_OPENNI_FRAME_MAX_DEPTH -- A maximum supported depth of Kinect in mm.
- CAP_PROP_OPENNI_BASELINE -- Baseline value in mm.
- CAP_PROP_OPENNI_FOCAL_LENGTH -- A focal length in pixels.
- CAP_PROP_FRAME_WIDTH -- Frame width in pixels.
- CAP_PROP_FRAME_HEIGHT -- Frame height in pixels.
- CAP_PROP_FPS -- Frame rate in FPS.
- Some typical flags combinations "generator type + property" are defined as single flags:
- CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE = CAP_OPENNI_IMAGE_GENERATOR + CAP_PROP_OPENNI_OUTPUT_MODE
- CAP_OPENNI_DEPTH_GENERATOR_BASELINE = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_BASELINE
- CAP_OPENNI_DEPTH_GENERATOR_FOCAL_LENGTH = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_FOCAL_LENGTH
- CAP_OPENNI_DEPTH_GENERATOR_REGISTRATION = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_REGISTRATION
For more information please refer to the example of usage
[videocapture_openni.cpp](https://github.com/opencv/opencv/tree/master/samples/cpp/videocapture_openni.cpp) in
opencv/samples/cpp folder.

View File

@@ -0,0 +1,169 @@
Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra}
======================================================
@tableofcontents
@prev_tutorial{tutorial_kinect_openni}
@next_tutorial{tutorial_intelperc}
### Introduction
This tutorial is devoted to the Astra Series of Orbbec 3D cameras (https://orbbec3d.com/product-astra-pro/).
That cameras have a depth sensor in addition to a common color sensor. The depth sensors can be read using
the open source OpenNI API with @ref cv::VideoCapture class. The video stream is provided through the regular
camera interface.
### Installation Instructions
In order to use the Astra camera's depth sensor with OpenCV you should do the following steps:
-# Download the latest version of Orbbec OpenNI SDK (from here <https://orbbec3d.com/develop/>).
Unzip the archive, choose the build according to your operating system and follow installation
steps provided in the Readme file. For instance, if you use 64bit GNU/Linux run:
@code{.bash}
$ cd Linux/OpenNI-Linux-x64-2.3.0.63/
$ sudo ./install.sh
@endcode
When you are done with the installation, make sure to replug your device for udev rules to take
effect. The camera should now work as a general camera device. Note that your current user should
belong to group `video` to have access to the camera. Also, make sure to source `OpenNIDevEnvironment` file:
@code{.bash}
$ source OpenNIDevEnvironment
@endcode
-# Run the following commands to verify that OpenNI library and header files can be found. You should see
something similar in your terminal:
@code{.bash}
$ echo $OPENNI2_INCLUDE
/home/user/OpenNI_2.3.0.63/Linux/OpenNI-Linux-x64-2.3.0.63/Include
$ echo $OPENNI2_REDIST
/home/user/OpenNI_2.3.0.63/Linux/OpenNI-Linux-x64-2.3.0.63/Redist
@endcode
If the above two variables are empty, then you need to source `OpenNIDevEnvironment` again. Now you can
configure OpenCV with OpenNI support enabled by setting the `WITH_OPENNI2` flag in CMake.
You may also like to enable the `BUILD_EXAMPLES` flag to get a code sample working with your Astra camera.
Run the following commands in the directory containing OpenCV source code to enable OpenNI support:
@code{.bash}
$ mkdir build
$ cd build
$ cmake -DWITH_OPENNI2=ON ..
@endcode
If the OpenNI library is found, OpenCV will be built with OpenNI2 support. You can see the status of OpenNI2
support in the CMake log:
@code{.text}
-- Video I/O:
-- DC1394: YES (2.2.6)
-- FFMPEG: YES
-- avcodec: YES (58.91.100)
-- avformat: YES (58.45.100)
-- avutil: YES (56.51.100)
-- swscale: YES (5.7.100)
-- avresample: NO
-- GStreamer: YES (1.18.1)
-- OpenNI2: YES (2.3.0)
-- v4l/v4l2: YES (linux/videodev2.h)
@endcode
-# Build OpenCV:
@code{.bash}
$ make
@endcode
### Code
The Astra Pro camera has two sensors -- a depth sensor and a color sensor. The depth sensor
can be read using the OpenNI interface with @ref cv::VideoCapture class. The video stream is
not available through OpenNI API and is only provided via the regular camera interface.
So, to get both depth and color frames, two @ref cv::VideoCapture objects should be created:
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Open streams
The first object will use the OpenNI2 API to retrieve depth data. The second one uses the
Video4Linux2 interface to access the color sensor. Note that the example above assumes that
the Astra camera is the first camera in the system. If you have more than one camera connected,
you may need to explicitly set the proper camera number.
Before using the created VideoCapture objects you may want to set up stream parameters by setting
objects' properties. The most important parameters are frame width, frame height and fps.
For this example, well configure width and height of both streams to VGA resolution, which is
the maximum resolution available for both sensors, and wed like both stream parameters to be the
same for easier color-to-depth data registration:
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Setup streams
For setting and retrieving some property of sensor data generators use @ref cv::VideoCapture::set and
@ref cv::VideoCapture::get methods respectively, e.g. :
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Get properties
The following properties of cameras available through OpenNI interface are supported for the depth
generator:
- @ref cv::CAP_PROP_FRAME_WIDTH -- Frame width in pixels.
- @ref cv::CAP_PROP_FRAME_HEIGHT -- Frame height in pixels.
- @ref cv::CAP_PROP_FPS -- Frame rate in FPS.
- @ref cv::CAP_PROP_OPENNI_REGISTRATION -- Flag that registers the remapping depth map to image map
by changing the depth generator's viewpoint (if the flag is "on") or sets this view point to
its normal one (if the flag is "off"). The registration process resulting images are
pixel-aligned, which means that every pixel in the image is aligned to a pixel in the depth
image.
- @ref cv::CAP_PROP_OPENNI2_MIRROR -- Flag to enable or disable mirroring for this stream. Set to 0
to disable mirroring
Next properties are available for getting only:
- @ref cv::CAP_PROP_OPENNI_FRAME_MAX_DEPTH -- A maximum supported depth of the camera in mm.
- @ref cv::CAP_PROP_OPENNI_BASELINE -- Baseline value in mm.
After the VideoCapture objects have been set up, you can start reading frames from them.
@note
OpenCV's VideoCapture provides synchronous API, so you have to grab frames in a new thread
to avoid one stream blocking while another stream is being read. VideoCapture is not a
thread-safe class, so you need to be careful to avoid any possible deadlocks or data races.
As there are two video sources that should be read simultaneously, its necessary to create two
threads to avoid blocking. Example implementation that gets frames from each sensor in a new thread
and stores them in a list along with their timestamps:
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Read streams
VideoCapture can retrieve the following data:
-# data given from the depth generator:
- @ref cv::CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1)
- @ref cv::CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3)
- @ref cv::CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1)
- @ref cv::CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1)
- @ref cv::CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not occluded, not shaded, etc.)
(CV_8UC1)
-# data given from the color sensor is a regular BGR image (CV_8UC3).
When new data are available, each reading thread notifies the main thread using a condition variable.
A frame is stored in the ordered list -- the first frame in the list is the earliest captured,
the last frame is the latest captured. As depth and color frames are read from independent sources
two video streams may become out of sync even when both streams are set up for the same frame rate.
A post-synchronization procedure can be applied to the streams to combine depth and color frames into
pairs. The sample code below demonstrates this procedure:
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Pair frames
In the code snippet above the execution is blocked until there are some frames in both frame lists.
When there are new frames, their timestamps are being checked -- if they differ more than a half of
the frame period then one of the frames is dropped. If timestamps are close enough, then two frames
are paired. Now, we have two frames: one containing color information and another one -- depth information.
In the example above retrieved frames are simply shown with cv::imshow function, but you can insert
any other processing code here.
In the sample images below you can see the color frame and the depth frame representing the same scene.
Looking at the color frame it's hard to distinguish plant leaves from leaves painted on a wall,
but the depth data makes it easy.
![Color frame](images/astra_color.jpg)
![Depth frame](images/astra_depth.png)
The complete implementation can be found in
[orbbec_astra.cpp](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp)
in `samples/cpp/tutorial_code/videoio` directory.

View File

@@ -0,0 +1,107 @@
Reading Geospatial Raster files with GDAL {#tutorial_raster_io_gdal}
=========================================
@tableofcontents
@prev_tutorial{tutorial_trackbar}
@next_tutorial{tutorial_video_input_psnr_ssim}
| | |
| -: | :- |
| Original author | Marvin Smith |
| Compatibility | OpenCV >= 3.0 |
Geospatial raster data is a heavily used product in Geographic Information Systems and
Photogrammetry. Raster data typically can represent imagery and Digital Elevation Models (DEM). The
standard library for loading GIS imagery is the Geographic Data Abstraction Library [(GDAL)](http://www.gdal.org). In this
example, we will show techniques for loading GIS raster formats using native OpenCV functions. In
addition, we will show some an example of how OpenCV can use this data for novel and interesting
purposes.
Goals
-----
The primary objectives for this tutorial:
- How to use OpenCV [imread](@ref imread) to load satellite imagery.
- How to use OpenCV [imread](@ref imread) to load SRTM Digital Elevation Models
- Given the corner coordinates of both the image and DEM, correlate the elevation data to the
image to find elevations for each pixel.
- Show a basic, easy-to-implement example of a terrain heat map.
- Show a basic use of DEM data coupled with ortho-rectified imagery.
To implement these goals, the following code takes a Digital Elevation Model as well as a GeoTiff
image of San Francisco as input. The image and DEM data is processed and generates a terrain heat
map of the image as well as labels areas of the city which would be affected should the water level
of the bay rise 10, 50, and 100 meters.
Code
----
@include cpp/tutorial_code/imgcodecs/GDAL_IO/gdal-image.cpp
How to Read Raster Data using GDAL
----------------------------------
This demonstration uses the default OpenCV imread function. The primary difference is that in order
to force GDAL to load the image, you must use the appropriate flag.
@snippet cpp/tutorial_code/imgcodecs/GDAL_IO/gdal-image.cpp load1
When loading digital elevation models, the actual numeric value of each pixel is essential and
cannot be scaled or truncated. For example, with image data a pixel represented as a double with a
value of 1 has an equal appearance to a pixel which is represented as an unsigned character with a
value of 255. With terrain data, the pixel value represents the elevation in meters. In order to
ensure that OpenCV preserves the native value, use the GDAL flag in imread with the ANYDEPTH flag.
@snippet cpp/tutorial_code/imgcodecs/GDAL_IO/gdal-image.cpp load2
If you know beforehand the type of DEM model you are loading, then it may be a safe bet to test the
Mat::type() or Mat::depth() using an assert or other mechanism. NASA or DOD specification documents
can provide the input types for various elevation models. The major types, SRTM and DTED, are both
signed shorts.
Notes
-----
### Lat/Lon (Geographic) Coordinates should normally be avoided
The Geographic Coordinate System is a spherical coordinate system, meaning that using them with
Cartesian mathematics is technically incorrect. This demo uses them to increase the readability and
is accurate enough to make the point. A better coordinate system would be Universal Transverse
Mercator.
### Finding the corner coordinates
One easy method to find the corner coordinates of an image is to use the command-line tool gdalinfo.
For imagery which is ortho-rectified and contains the projection information, you can use the [USGS
EarthExplorer](http://http://earthexplorer.usgs.gov).
@code{.bash}
\f$> gdalinfo N37W123.hgt
Driver: SRTMHGT/SRTMHGT File Format
Files: N37W123.hgt
Size is 3601, 3601
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
... more output ...
Corner Coordinates:
Upper Left (-123.0001389, 38.0001389) (123d 0' 0.50"W, 38d 0' 0.50"N)
Lower Left (-123.0001389, 36.9998611) (123d 0' 0.50"W, 36d59'59.50"N)
Upper Right (-121.9998611, 38.0001389) (121d59'59.50"W, 38d 0' 0.50"N)
Lower Right (-121.9998611, 36.9998611) (121d59'59.50"W, 36d59'59.50"N)
Center (-122.5000000, 37.5000000) (122d30' 0.00"W, 37d30' 0.00"N)
... more output ...
@endcode
Results
-------
Below is the output of the program. Use the first image as the input. For the DEM model, download
the SRTM file located at the USGS here.
[<http://dds.cr.usgs.gov/srtm/version2_1/SRTM1/Region_04/N37W123.hgt.zip>](http://dds.cr.usgs.gov/srtm/version2_1/SRTM1/Region_04/N37W123.hgt.zip)
![Input Image](images/gdal_output.jpg)
![Heat Map](images/gdal_heat-map.jpg)
![Heat Map Overlay](images/gdal_flood-zone.jpg)

View File

@@ -0,0 +1,10 @@
Application utils (highgui, imgcodecs, videoio modules) {#tutorial_table_of_content_app}
=======================================================
- @subpage tutorial_trackbar
- @subpage tutorial_raster_io_gdal
- @subpage tutorial_video_input_psnr_ssim
- @subpage tutorial_video_write
- @subpage tutorial_kinect_openni
- @subpage tutorial_orbbec_astra
- @subpage tutorial_intelperc

View File

@@ -0,0 +1,142 @@
Adding a Trackbar to our applications! {#tutorial_trackbar}
======================================
@tableofcontents
@next_tutorial{tutorial_raster_io_gdal}
| | |
| -: | :- |
| Original author | Ana Huamán |
| Compatibility | OpenCV >= 3.0 |
- In the previous tutorials (about @ref tutorial_adding_images and the @ref tutorial_basic_linear_transform)
you might have noted that we needed to give some **input** to our programs, such
as \f$\alpha\f$ and \f$beta\f$. We accomplished that by entering this data using the Terminal.
- Well, it is time to use some fancy GUI tools. OpenCV provides some GUI utilities (**highgui** module)
for you. An example of this is a **Trackbar**.
![](images/Adding_Trackbars_Tutorial_Trackbar.png)
- In this tutorial we will just modify our two previous programs so that they get the input
information from the trackbar.
Goals
-----
In this tutorial you will learn how to:
- Add a Trackbar in an OpenCV window by using @ref cv::createTrackbar
Code
----
Let's modify the program made in the tutorial @ref tutorial_adding_images. We will let the user enter the
\f$\alpha\f$ value by using the Trackbar.
@add_toggle_cpp
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp)
@include cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp
@end_toggle
@add_toggle_java
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/master/samples/java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java)
@include java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java
@end_toggle
@add_toggle_python
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/master/samples/python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py)
@include python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py
@end_toggle
Explanation
-----------
We only analyze the code that is related to Trackbar:
- First, we load two images, which are going to be blended.
@add_toggle_cpp
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp load
@end_toggle
@add_toggle_java
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java load
@end_toggle
@add_toggle_python
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py load
@end_toggle
- To create a trackbar, first we have to create the window in which it is going to be located. So:
@add_toggle_cpp
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp window
@end_toggle
@add_toggle_java
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java window
@end_toggle
@add_toggle_python
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py window
@end_toggle
- Now we can create the Trackbar:
@add_toggle_cpp
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp create_trackbar
@end_toggle
@add_toggle_java
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java create_trackbar
@end_toggle
@add_toggle_python
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py create_trackbar
@end_toggle
Note the following (C++ code):
- Our Trackbar has a label **TrackbarName**
- The Trackbar is located in the window named **Linear Blend**
- The Trackbar values will be in the range from \f$0\f$ to **alpha_slider_max** (the minimum
limit is always **zero**).
- The numerical value of Trackbar is stored in **alpha_slider**
- Whenever the user moves the Trackbar, the callback function **on_trackbar** is called
Finally, we have to define the callback function **on_trackbar** for C++ and Python code, using an anonymous inner class listener in Java
@add_toggle_cpp
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp on_trackbar
@end_toggle
@add_toggle_java
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java on_trackbar
@end_toggle
@add_toggle_python
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py on_trackbar
@end_toggle
Note that (C++ code):
- We use the value of **alpha_slider** (integer) to get a double value for **alpha**.
- **alpha_slider** is updated each time the trackbar is displaced by the user.
- We define *src1*, *src2*, *dist*, *alpha*, *alpha_slider* and *beta* as global variables,
so they can be used everywhere.
Result
------
- Our program produces the following output:
![](images/Adding_Trackbars_Tutorial_Result_0.jpg)
- As a manner of practice, you can also add two trackbars for the program made in
@ref tutorial_basic_linear_transform. One trackbar to set \f$\alpha\f$ and another for set \f$\beta\f$. The output might
look like:
![](images/Adding_Trackbars_Tutorial_Result_1.jpg)

View File

@@ -0,0 +1,203 @@
Video Input with OpenCV and similarity measurement {#tutorial_video_input_psnr_ssim}
==================================================
@tableofcontents
@prev_tutorial{tutorial_raster_io_gdal}
@next_tutorial{tutorial_video_write}
| | |
| -: | :- |
| Original author | Bernát Gábor |
| Compatibility | OpenCV >= 3.0 |
Goal
----
Today it is common to have a digital video recording system at your disposal. Therefore, you will
eventually come to the situation that you no longer process a batch of images, but video streams.
These may be of two kinds: real-time image feed (in the case of a webcam) or prerecorded and hard
disk drive stored files. Luckily OpenCV treats these two in the same manner, with the same C++
class. So here's what you'll learn in this tutorial:
- How to open and read video streams
- Two ways for checking image similarity: PSNR and SSIM
The source code
---------------
As a test case where to show off these using OpenCV I've created a small program that reads in two
video files and performs a similarity check between them. This is something you could use to check
just how well a new video compressing algorithms works. Let there be a reference (original) video
like [this small Megamind clip
](https://github.com/opencv/opencv/tree/master/samples/data/Megamind.avi) and [a compressed
version of it ](https://github.com/opencv/opencv/tree/master/samples/data/Megamind_bugy.avi).
You may also find the source code and these video file in the
`samples/data` folder of the OpenCV source library.
@add_toggle_cpp
@include cpp/tutorial_code/videoio/video-input-psnr-ssim/video-input-psnr-ssim.cpp
@end_toggle
@add_toggle_python
@include samples/python/tutorial_code/videoio/video-input-psnr-ssim.py
@end_toggle
How to read a video stream (online-camera or offline-file)?
-----------------------------------------------------------
Essentially, all the functionalities required for video manipulation is integrated in the @ref cv::VideoCapture
C++ class. This on itself builds on the FFmpeg open source library. This is a basic
dependency of OpenCV so you shouldn't need to worry about this. A video is composed of a succession
of images, we refer to these in the literature as frames. In case of a video file there is a *frame
rate* specifying just how long is between two frames. While for the video cameras usually there is a
limit of just how many frames they can digitize per second, this property is less important as at
any time the camera sees the current snapshot of the world.
The first task you need to do is to assign to a @ref cv::VideoCapture class its source. You can do
this either via the @ref cv::VideoCapture::VideoCapture or its @ref cv::VideoCapture::open function. If this argument is an
integer then you will bind the class to a camera, a device. The number passed here is the ID of the
device, assigned by the operating system. If you have a single camera attached to your system its ID
will probably be zero and further ones increasing from there. If the parameter passed to these is a
string it will refer to a video file, and the string points to the location and name of the file.
For example, to the upper source code a valid command line is:
@code{.bash}
video/Megamind.avi video/Megamind_bug.avi 35 10
@endcode
We do a similarity check. This requires a reference and a test case video file. The first two
arguments refer to this. Here we use a relative address. This means that the application will look
into its current working directory and open the video folder and try to find inside this the
*Megamind.avi* and the *Megamind_bug.avi*.
@code{.cpp}
const string sourceReference = argv[1],sourceCompareWith = argv[2];
VideoCapture captRefrnc(sourceReference);
// or
VideoCapture captUndTst;
captUndTst.open(sourceCompareWith);
@endcode
To check if the binding of the class to a video source was successful or not use the @ref cv::VideoCapture::isOpened
function:
@code{.cpp}
if ( !captRefrnc.isOpened())
{
cout << "Could not open reference " << sourceReference << endl;
return -1;
}
@endcode
Closing the video is automatic when the objects destructor is called. However, if you want to close
it before this you need to call its @ref cv::VideoCapture::release function. The frames of the video are just
simple images. Therefore, we just need to extract them from the @ref cv::VideoCapture object and put
them inside a *Mat* one. The video streams are sequential. You may get the frames one after another
by the @ref cv::VideoCapture::read or the overloaded \>\> operator:
@code{.cpp}
Mat frameReference, frameUnderTest;
captRefrnc >> frameReference;
captUndTst.read(frameUnderTest);
@endcode
The upper read operations will leave empty the *Mat* objects if no frame could be acquired (either
cause the video stream was closed or you got to the end of the video file). We can check this with a
simple if:
@code{.cpp}
if( frameReference.empty() || frameUnderTest.empty())
{
// exit the program
}
@endcode
A read method is made of a frame grab and a decoding applied on that. You may call explicitly these
two by using the @ref cv::VideoCapture::grab and then the @ref cv::VideoCapture::retrieve functions.
Videos have many-many information attached to them besides the content of the frames. These are
usually numbers, however in some case it may be short character sequences (4 bytes or less). Due to
this to acquire these information there is a general function named @ref cv::VideoCapture::get that returns double
values containing these properties. Use bitwise operations to decode the characters from a double
type and conversions where valid values are only integers. Its single argument is the ID of the
queried property. For example, here we get the size of the frames in the reference and test case
video file; plus the number of frames inside the reference.
@code{.cpp}
Size refS = Size((int) captRefrnc.get(CAP_PROP_FRAME_WIDTH),
(int) captRefrnc.get(CAP_PROP_FRAME_HEIGHT)),
cout << "Reference frame resolution: Width=" << refS.width << " Height=" << refS.height
<< " of nr#: " << captRefrnc.get(CAP_PROP_FRAME_COUNT) << endl;
@endcode
When you are working with videos you may often want to control these values yourself. To do this
there is a @ref cv::VideoCapture::set function. Its first argument remains the name of the property you want to
change and there is a second of double type containing the value to be set. It will return true if
it succeeds and false otherwise. Good examples for this is seeking in a video file to a given time
or frame:
@code{.cpp}
captRefrnc.set(CAP_PROP_POS_MSEC, 1.2); // go to the 1.2 second in the video
captRefrnc.set(CAP_PROP_POS_FRAMES, 10); // go to the 10th frame of the video
// now a read operation would read the frame at the set position
@endcode
For properties you can read and change look into the documentation of the @ref cv::VideoCapture::get and
@ref cv::VideoCapture::set functions.
### Image similarity - PSNR and SSIM
We want to check just how imperceptible our video converting operation went, therefore we need a
system to check frame by frame the similarity or differences. The most common algorithm used for
this is the PSNR (aka **Peak signal-to-noise ratio**). The simplest definition of this starts out
from the *mean squared error*. Let there be two images: I1 and I2; with a two dimensional size i and
j, composed of c number of channels.
\f[MSE = \frac{1}{c*i*j} \sum{(I_1-I_2)^2}\f]
Then the PSNR is expressed as:
\f[PSNR = 10 \cdot \log_{10} \left( \frac{MAX_I^2}{MSE} \right)\f]
Here the \f$MAX_I\f$ is the maximum valid value for a pixel. In case of the simple single byte image
per pixel per channel this is 255. When two images are the same the MSE will give zero, resulting in
an invalid divide by zero operation in the PSNR formula. In this case the PSNR is undefined and as
we'll need to handle this case separately. The transition to a logarithmic scale is made because the
pixel values have a very wide dynamic range. All this translated to OpenCV and a function looks
like:
@add_toggle_cpp
@snippet cpp/tutorial_code/videoio/video-input-psnr-ssim/video-input-psnr-ssim.cpp get-psnr
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/videoio/video-input-psnr-ssim.py get-psnr
@end_toggle
Typically result values are anywhere between 30 and 50 for video compression, where higher is
better. If the images significantly differ you'll get much lower ones like 15 and so. This
similarity check is easy and fast to calculate, however in practice it may turn out somewhat
inconsistent with human eye perception. The **structural similarity** algorithm aims to correct
this.
Describing the methods goes well beyond the purpose of this tutorial. For that I invite you to read
the article introducing it. Nevertheless, you can get a good image of it by looking at the OpenCV
implementation below.
@note
SSIM is described more in-depth in the: "Z. Wang, A. C. Bovik, H. R. Sheikh and E. P.
Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE
Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004." article.
@add_toggle_cpp
@snippet samples/cpp/tutorial_code/videoio/video-input-psnr-ssim/video-input-psnr-ssim.cpp get-mssim
@end_toggle
@add_toggle_python
@snippet samples/python/tutorial_code/videoio/video-input-psnr-ssim.py get-mssim
@end_toggle
This will return a similarity index for each channel of the image. This value is between zero and
one, where one corresponds to perfect fit. Unfortunately, the many Gaussian blurring is quite
costly, so while the PSNR may work in a real time like environment (24 frame per second) this will
take significantly more than to accomplish similar performance results.
Therefore, the source code presented at the start of the tutorial will perform the PSNR measurement
for each frame, and the SSIM only for the frames where the PSNR falls below an input value. For
visualization purpose we show both images in an OpenCV window and print the PSNR and MSSIM values to
the console. Expect to see something like:
![](images/outputVideoInput.png)
You may observe a runtime instance of this on the [YouTube here](https://www.youtube.com/watch?v=iOcNljutOgg).
@youtube{iOcNljutOgg}

View File

@@ -0,0 +1,166 @@
Creating a video with OpenCV {#tutorial_video_write}
============================
@tableofcontents
@prev_tutorial{tutorial_video_input_psnr_ssim}
@next_tutorial{tutorial_kinect_openni}
| | |
| -: | :- |
| Original author | Bernát Gábor |
| Compatibility | OpenCV >= 3.0 |
Goal
----
Whenever you work with video feeds you may eventually want to save your image processing result in a
form of a new video file. For simple video outputs you can use the OpenCV built-in @ref cv::VideoWriter
class, designed for this.
- How to create a video file with OpenCV
- What type of video files you can create with OpenCV
- How to extract a given color channel from a video
As a simple demonstration I'll just extract one of the BGR color channels of an input video file
into a new video. You can control the flow of the application from its console line arguments:
- The first argument points to the video file to work on
- The second argument may be one of the characters: R G B. This will specify which of the channels
to extract.
- The last argument is the character Y (Yes) or N (No). If this is no, the codec used for the
input video file will be the same as for the output. Otherwise, a window will pop up and allow
you to select yourself the codec to use.
For example, a valid command line would look like:
@code{.bash}
video-write.exe video/Megamind.avi R Y
@endcode
The source code
---------------
You may also find the source code and these video file in the
`samples/cpp/tutorial_code/videoio/video-write/` folder of the OpenCV source library or [download it
from here ](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/videoio/video-write/video-write.cpp).
@include cpp/tutorial_code/videoio/video-write/video-write.cpp
The structure of a video
------------------------
For start, you should have an idea of just how a video file looks. Every video file in itself is a
container. The type of the container is expressed in the files extension (for example *avi*, *mov*
or *mkv*). This contains multiple elements like: video feeds, audio feeds or other tracks (like for
example subtitles). How these feeds are stored is determined by the codec used for each one of them.
In case of the audio tracks commonly used codecs are *mp3* or *aac*. For the video files the list is
somehow longer and includes names such as *XVID*, *DIVX*, *H264* or *LAGS* (*Lagarith Lossless
Codec*). The full list of codecs you may use on a system depends on just what one you have
installed.
![](images/videoFileStructure.png)
As you can see things can get really complicated with videos. However, OpenCV is mainly a computer
vision library, not a video stream, codec and write one. Therefore, the developers tried to keep
this part as simple as possible. Due to this OpenCV for video containers supports only the *avi*
extension, its first version. A direct limitation of this is that you cannot save a video file
larger than 2 GB. Furthermore you can only create and expand a single video track inside the
container. No audio or other track editing support here. Nevertheless, any video codec present on
your system might work. If you encounter some of these limitations you will need to look into more
specialized video writing libraries such as *FFmpeg* or codecs as *HuffYUV*, *CorePNG* and *LCL*. As
an alternative, create the video track with OpenCV and expand it with sound tracks or convert it to
other formats by using video manipulation programs such as *VirtualDub* or *AviSynth*.
The VideoWriter class
-----------------------
The content written here builds on the assumption you
already read the @ref tutorial_video_input_psnr_ssim tutorial and you know how to read video files. To create a
video file you just need to create an instance of the @ref cv::VideoWriter class. You can specify
its properties either via parameters in the constructor or later on via the @ref cv::VideoWriter::open function.
Either way, the parameters are the same: 1. The name of the output that contains the container type
in its extension. At the moment only *avi* is supported. We construct this from the input file, add
to this the name of the channel to use, and finish it off with the container extension.
@code{.cpp}
const string source = argv[1]; // the source file name
string::size_type pAt = source.find_last_of('.'); // Find extension point
const string NAME = source.substr(0, pAt) + argv[2][0] + ".avi"; // Form the new name with container
@endcode
-# The codec to use for the video track. Now all the video codecs have a unique short name of
maximum four characters. Hence, the *XVID*, *DIVX* or *H264* names. This is called a four
character code. You may also ask this from an input video by using its *get* function. Because
the *get* function is a general function it always returns double values. A double value is
stored on 64 bits. Four characters are four bytes, meaning 32 bits. These four characters are
coded in the lower 32 bits of the *double*. A simple way to throw away the upper 32 bits would
be to just convert this value to *int*:
@code{.cpp}
VideoCapture inputVideo(source); // Open input
int ex = static_cast<int>(inputVideo.get(CAP_PROP_FOURCC)); // Get Codec Type- Int form
@endcode
OpenCV internally works with this integer type and expect this as its second parameter. Now to
convert from the integer form to string we may use two methods: a bitwise operator and a union
method. The first one extracting from an int the characters looks like (an "and" operation, some
shifting and adding a 0 at the end to close the string):
@code{.cpp}
char EXT[] = {ex & 0XFF , (ex & 0XFF00) >> 8,(ex & 0XFF0000) >> 16,(ex & 0XFF000000) >> 24, 0};
@endcode
You can do the same thing with the *union* as:
@code{.cpp}
union { int v; char c[5];} uEx ;
uEx.v = ex; // From Int to char via union
uEx.c[4]='\0';
@endcode
The advantage of this is that the conversion is done automatically after assigning, while for
the bitwise operator you need to do the operations whenever you change the codec type. In case
you know the codecs four character code beforehand, you can use the *CV_FOURCC* macro to build
the integer:
@code{.cpp}
CV_FOURCC('P','I','M,'1') // this is an MPEG1 codec from the characters to integer
@endcode
If you pass for this argument minus one then a window will pop up at runtime that contains all
the codec installed on your system and ask you to select the one to use:
![](images/videoCompressSelect.png)
-# The frame per second for the output video. Again, here I keep the input videos frame per second
by using the *get* function.
-# The size of the frames for the output video. Here too I keep the input videos frame size per
second by using the *get* function.
-# The final argument is an optional one. By default is true and says that the output will be a
colorful one (so for write you will send three channel images). To create a gray scale video
pass a false parameter here.
Here it is, how I use it in the sample:
@code{.cpp}
VideoWriter outputVideo;
Size S = Size((int) inputVideo.get(CAP_PROP_FRAME_WIDTH), //Acquire input size
(int) inputVideo.get(CAP_PROP_FRAME_HEIGHT));
outputVideo.open(NAME , ex, inputVideo.get(CAP_PROP_FPS),S, true);
@endcode
Afterwards, you use the @ref cv::VideoWriter::isOpened() function to find out if the open operation succeeded or
not. The video file automatically closes when the *VideoWriter* object is destroyed. After you open
the object with success you can send the frames of the video in a sequential order by using the
@ref cv::VideoWriter::write function of the class. Alternatively, you can use its overloaded operator \<\< :
@code{.cpp}
outputVideo.write(res); //or
outputVideo << res;
@endcode
Extracting a color channel from an BGR image means to set to zero the BGR values of the other
channels. You can either do this with image scanning operations or by using the split and merge
operations. You first split the channels up into different images, set the other channels to zero
images of the same size and type and finally merge them back:
@code{.cpp}
split(src, spl); // process - extract only the correct channel
for( int i =0; i < 3; ++i)
if (i != channel)
spl[i] = Mat::zeros(S, spl[0].type());
merge(spl, res);
@endcode
Put all this together and you'll get the upper source code, whose runtime result will show something
around the idea:
![](images/resultOutputWideoWrite.png)
You may observe a runtime instance of this on the [YouTube
here](https://www.youtube.com/watch?v=jpBwHxsl1_0).
@youtube{jpBwHxsl1_0}