init - 初始化项目
4
doc/tutorials/app/_old/table_of_content_highgui.markdown
Normal file
@@ -0,0 +1,4 @@
|
||||
High Level GUI and Media (highgui module) {#tutorial_table_of_content_highgui}
|
||||
=========================================
|
||||
|
||||
Content has been moved to this page: @ref tutorial_table_of_content_app
|
||||
@@ -0,0 +1,4 @@
|
||||
Image Input and Output (imgcodecs module) {#tutorial_table_of_content_imgcodecs}
|
||||
=========================================
|
||||
|
||||
Content has been moved to this page: @ref tutorial_table_of_content_app
|
||||
4
doc/tutorials/app/_old/table_of_content_videoio.markdown
Normal file
@@ -0,0 +1,4 @@
|
||||
Video Input and Output (videoio module) {#tutorial_table_of_content_videoio}
|
||||
=========================================
|
||||
|
||||
Content has been moved to this page: @ref tutorial_table_of_content_app
|
||||
BIN
doc/tutorials/app/images/Adding_Trackbars_Tutorial_Result_0.jpg
Normal file
|
After Width: | Height: | Size: 9.2 KiB |
BIN
doc/tutorials/app/images/Adding_Trackbars_Tutorial_Result_1.jpg
Normal file
|
After Width: | Height: | Size: 29 KiB |
BIN
doc/tutorials/app/images/Adding_Trackbars_Tutorial_Trackbar.png
Normal file
|
After Width: | Height: | Size: 2.0 KiB |
BIN
doc/tutorials/app/images/astra_color.jpg
Normal file
|
After Width: | Height: | Size: 135 KiB |
BIN
doc/tutorials/app/images/astra_depth.png
Normal file
|
After Width: | Height: | Size: 29 KiB |
BIN
doc/tutorials/app/images/gdal_flood-zone.jpg
Normal file
|
After Width: | Height: | Size: 111 KiB |
BIN
doc/tutorials/app/images/gdal_heat-map.jpg
Normal file
|
After Width: | Height: | Size: 53 KiB |
BIN
doc/tutorials/app/images/gdal_output.jpg
Normal file
|
After Width: | Height: | Size: 120 KiB |
BIN
doc/tutorials/app/images/outputVideoInput.png
Normal file
|
After Width: | Height: | Size: 48 KiB |
BIN
doc/tutorials/app/images/resultOutputWideoWrite.png
Normal file
|
After Width: | Height: | Size: 39 KiB |
BIN
doc/tutorials/app/images/videoCompressSelect.png
Normal file
|
After Width: | Height: | Size: 7.2 KiB |
BIN
doc/tutorials/app/images/videoFileStructure.png
Normal file
|
After Width: | Height: | Size: 12 KiB |
85
doc/tutorials/app/intelperc.markdown
Normal file
@@ -0,0 +1,85 @@
|
||||
Using Creative Senz3D and other Intel RealSense SDK compatible depth sensors {#tutorial_intelperc}
|
||||
=======================================================================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_orbbec_astra}
|
||||
|
||||
**Note**: This tutorial is partially obsolete since PerC SDK has been replaced with RealSense SDK
|
||||
|
||||
Depth sensors compatible with Intel® RealSense SDK are supported through VideoCapture
|
||||
class. Depth map, RGB image and some other formats of output can be retrieved by using familiar
|
||||
interface of VideoCapture.
|
||||
|
||||
In order to use depth sensor with OpenCV you should do the following preliminary steps:
|
||||
|
||||
-# Install Intel RealSense SDK 2.0 (from here <https://github.com/IntelRealSense/librealsense>).
|
||||
|
||||
-# Configure OpenCV with Intel RealSense SDK support by setting WITH_LIBREALSENSE flag in
|
||||
CMake. If Intel RealSense SDK is found in install folders OpenCV will be built with
|
||||
Intel Realsense SDK library (see a status LIBREALSENSE in CMake log).
|
||||
|
||||
-# Build OpenCV.
|
||||
|
||||
VideoCapture can retrieve the following data:
|
||||
|
||||
-# data given from depth generator:
|
||||
- CAP_INTELPERC_DEPTH_MAP - each pixel is a 16-bit integer. The value indicates the
|
||||
distance from an object to the camera's XY plane or the Cartesian depth. (CV_16UC1)
|
||||
- CAP_INTELPERC_UVDEPTH_MAP - each pixel contains two 32-bit floating point values in
|
||||
the range of 0-1, representing the mapping of depth coordinates to the color
|
||||
coordinates. (CV_32FC2)
|
||||
- CAP_INTELPERC_IR_MAP - each pixel is a 16-bit integer. The value indicates the
|
||||
intensity of the reflected laser beam. (CV_16UC1)
|
||||
|
||||
-# data given from RGB image generator:
|
||||
- CAP_INTELPERC_IMAGE - color image. (CV_8UC3)
|
||||
|
||||
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. :
|
||||
@code{.cpp}
|
||||
VideoCapture capture( CAP_REALSENSE );
|
||||
for(;;)
|
||||
{
|
||||
Mat depthMap;
|
||||
capture >> depthMap;
|
||||
|
||||
if( waitKey( 30 ) >= 0 )
|
||||
break;
|
||||
}
|
||||
@endcode
|
||||
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. :
|
||||
@code{.cpp}
|
||||
VideoCapture capture(CAP_REALSENSE);
|
||||
for(;;)
|
||||
{
|
||||
Mat depthMap;
|
||||
Mat image;
|
||||
Mat irImage;
|
||||
|
||||
capture.grab();
|
||||
|
||||
capture.retrieve( depthMap, CAP_INTELPERC_DEPTH_MAP );
|
||||
capture.retrieve( image, CAP_INTELPERC_IMAGE );
|
||||
capture.retrieve( irImage, CAP_INTELPERC_IR_MAP);
|
||||
|
||||
if( waitKey( 30 ) >= 0 )
|
||||
break;
|
||||
}
|
||||
@endcode
|
||||
For setting and getting some property of sensor\` data generators use VideoCapture::set and
|
||||
VideoCapture::get methods respectively, e.g. :
|
||||
@code{.cpp}
|
||||
VideoCapture capture(CAP_REALSENSE);
|
||||
capture.set( CAP_INTELPERC_DEPTH_GENERATOR | CAP_PROP_INTELPERC_PROFILE_IDX, 0 );
|
||||
cout << "FPS " << capture.get( CAP_INTELPERC_DEPTH_GENERATOR+CAP_PROP_FPS ) << endl;
|
||||
@endcode
|
||||
Since two types of sensor's data generators are supported (image generator and depth generator),
|
||||
there are two flags that should be used to set/get property of the needed generator:
|
||||
|
||||
- CAP_INTELPERC_IMAGE_GENERATOR -- a flag for access to the image generator properties.
|
||||
- CAP_INTELPERC_DEPTH_GENERATOR -- a flag for access to the depth generator properties. This
|
||||
flag value is assumed by default if neither of the two possible values of the property is set.
|
||||
|
||||
For more information please refer to the example of usage
|
||||
[videocapture_intelperc.cpp](https://github.com/opencv/opencv/tree/master/samples/cpp/videocapture_intelperc.cpp)
|
||||
in opencv/samples/cpp folder.
|
||||
144
doc/tutorials/app/kinect_openni.markdown
Normal file
@@ -0,0 +1,144 @@
|
||||
Using Kinect and other OpenNI compatible depth sensors {#tutorial_kinect_openni}
|
||||
======================================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_video_write}
|
||||
@next_tutorial{tutorial_orbbec_astra}
|
||||
|
||||
|
||||
Depth sensors compatible with OpenNI (Kinect, XtionPRO, ...) are supported through VideoCapture
|
||||
class. Depth map, BGR image and some other formats of output can be retrieved by using familiar
|
||||
interface of VideoCapture.
|
||||
|
||||
In order to use depth sensor with OpenCV you should do the following preliminary steps:
|
||||
|
||||
-# Install OpenNI library (from here <http://www.openni.org/downloadfiles>) and PrimeSensor Module
|
||||
for OpenNI (from here <https://github.com/avin2/SensorKinect>). The installation should be done
|
||||
to default folders listed in the instructions of these products, e.g.:
|
||||
@code{.text}
|
||||
OpenNI:
|
||||
Linux & MacOSX:
|
||||
Libs into: /usr/lib
|
||||
Includes into: /usr/include/ni
|
||||
Windows:
|
||||
Libs into: c:/Program Files/OpenNI/Lib
|
||||
Includes into: c:/Program Files/OpenNI/Include
|
||||
PrimeSensor Module:
|
||||
Linux & MacOSX:
|
||||
Bins into: /usr/bin
|
||||
Windows:
|
||||
Bins into: c:/Program Files/Prime Sense/Sensor/Bin
|
||||
@endcode
|
||||
If one or both products were installed to the other folders, the user should change
|
||||
corresponding CMake variables OPENNI_LIB_DIR, OPENNI_INCLUDE_DIR or/and
|
||||
OPENNI_PRIME_SENSOR_MODULE_BIN_DIR.
|
||||
|
||||
-# Configure OpenCV with OpenNI support by setting WITH_OPENNI flag in CMake. If OpenNI is found
|
||||
in install folders OpenCV will be built with OpenNI library (see a status OpenNI in CMake log)
|
||||
whereas PrimeSensor Modules can not be found (see a status OpenNI PrimeSensor Modules in CMake
|
||||
log). Without PrimeSensor module OpenCV will be successfully compiled with OpenNI library, but
|
||||
VideoCapture object will not grab data from Kinect sensor.
|
||||
|
||||
-# Build OpenCV.
|
||||
|
||||
VideoCapture can retrieve the following data:
|
||||
|
||||
-# data given from depth generator:
|
||||
- CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1)
|
||||
- CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3)
|
||||
- CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1)
|
||||
- CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1)
|
||||
- CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not occluded, not shaded etc.)
|
||||
(CV_8UC1)
|
||||
|
||||
-# data given from BGR image generator:
|
||||
- CAP_OPENNI_BGR_IMAGE - color image (CV_8UC3)
|
||||
- CAP_OPENNI_GRAY_IMAGE - gray image (CV_8UC1)
|
||||
|
||||
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. :
|
||||
@code{.cpp}
|
||||
VideoCapture capture( CAP_OPENNI );
|
||||
for(;;)
|
||||
{
|
||||
Mat depthMap;
|
||||
capture >> depthMap;
|
||||
|
||||
if( waitKey( 30 ) >= 0 )
|
||||
break;
|
||||
}
|
||||
@endcode
|
||||
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. :
|
||||
@code{.cpp}
|
||||
VideoCapture capture(0); // or CAP_OPENNI
|
||||
for(;;)
|
||||
{
|
||||
Mat depthMap;
|
||||
Mat bgrImage;
|
||||
|
||||
capture.grab();
|
||||
|
||||
capture.retrieve( depthMap, CAP_OPENNI_DEPTH_MAP );
|
||||
capture.retrieve( bgrImage, CAP_OPENNI_BGR_IMAGE );
|
||||
|
||||
if( waitKey( 30 ) >= 0 )
|
||||
break;
|
||||
}
|
||||
@endcode
|
||||
For setting and getting some property of sensor\` data generators use VideoCapture::set and
|
||||
VideoCapture::get methods respectively, e.g. :
|
||||
@code{.cpp}
|
||||
VideoCapture capture( CAP_OPENNI );
|
||||
capture.set( CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE, CAP_OPENNI_VGA_30HZ );
|
||||
cout << "FPS " << capture.get( CAP_OPENNI_IMAGE_GENERATOR+CAP_PROP_FPS ) << endl;
|
||||
@endcode
|
||||
Since two types of sensor's data generators are supported (image generator and depth generator),
|
||||
there are two flags that should be used to set/get property of the needed generator:
|
||||
|
||||
- CAP_OPENNI_IMAGE_GENERATOR -- A flag for access to the image generator properties.
|
||||
- CAP_OPENNI_DEPTH_GENERATOR -- A flag for access to the depth generator properties. This flag
|
||||
value is assumed by default if neither of the two possible values of the property is not set.
|
||||
|
||||
Some depth sensors (for example XtionPRO) do not have image generator. In order to check it you can
|
||||
get CAP_OPENNI_IMAGE_GENERATOR_PRESENT property.
|
||||
@code{.cpp}
|
||||
bool isImageGeneratorPresent = capture.get( CAP_PROP_OPENNI_IMAGE_GENERATOR_PRESENT ) != 0; // or == 1
|
||||
@endcode
|
||||
Flags specifying the needed generator type must be used in combination with particular generator
|
||||
property. The following properties of cameras available through OpenNI interfaces are supported:
|
||||
|
||||
- For image generator:
|
||||
|
||||
- CAP_PROP_OPENNI_OUTPUT_MODE -- Three output modes are supported: CAP_OPENNI_VGA_30HZ
|
||||
used by default (image generator returns images in VGA resolution with 30 FPS),
|
||||
CAP_OPENNI_SXGA_15HZ (image generator returns images in SXGA resolution with 15 FPS) and
|
||||
CAP_OPENNI_SXGA_30HZ (image generator returns images in SXGA resolution with 30 FPS, the
|
||||
mode is supported by XtionPRO Live); depth generator's maps are always in VGA resolution.
|
||||
|
||||
- For depth generator:
|
||||
|
||||
- CAP_PROP_OPENNI_REGISTRATION -- Flag that registers the remapping depth map to image map
|
||||
by changing depth generator's view point (if the flag is "on") or sets this view point to
|
||||
its normal one (if the flag is "off"). The registration process’s resulting images are
|
||||
pixel-aligned,which means that every pixel in the image is aligned to a pixel in the depth
|
||||
image.
|
||||
|
||||
Next properties are available for getting only:
|
||||
|
||||
- CAP_PROP_OPENNI_FRAME_MAX_DEPTH -- A maximum supported depth of Kinect in mm.
|
||||
- CAP_PROP_OPENNI_BASELINE -- Baseline value in mm.
|
||||
- CAP_PROP_OPENNI_FOCAL_LENGTH -- A focal length in pixels.
|
||||
- CAP_PROP_FRAME_WIDTH -- Frame width in pixels.
|
||||
- CAP_PROP_FRAME_HEIGHT -- Frame height in pixels.
|
||||
- CAP_PROP_FPS -- Frame rate in FPS.
|
||||
|
||||
- Some typical flags combinations "generator type + property" are defined as single flags:
|
||||
|
||||
- CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE = CAP_OPENNI_IMAGE_GENERATOR + CAP_PROP_OPENNI_OUTPUT_MODE
|
||||
- CAP_OPENNI_DEPTH_GENERATOR_BASELINE = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_BASELINE
|
||||
- CAP_OPENNI_DEPTH_GENERATOR_FOCAL_LENGTH = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_FOCAL_LENGTH
|
||||
- CAP_OPENNI_DEPTH_GENERATOR_REGISTRATION = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_REGISTRATION
|
||||
|
||||
For more information please refer to the example of usage
|
||||
[videocapture_openni.cpp](https://github.com/opencv/opencv/tree/master/samples/cpp/videocapture_openni.cpp) in
|
||||
opencv/samples/cpp folder.
|
||||
169
doc/tutorials/app/orbbec_astra.markdown
Normal file
@@ -0,0 +1,169 @@
|
||||
Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra}
|
||||
======================================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_kinect_openni}
|
||||
@next_tutorial{tutorial_intelperc}
|
||||
|
||||
|
||||
### Introduction
|
||||
|
||||
This tutorial is devoted to the Astra Series of Orbbec 3D cameras (https://orbbec3d.com/product-astra-pro/).
|
||||
That cameras have a depth sensor in addition to a common color sensor. The depth sensors can be read using
|
||||
the open source OpenNI API with @ref cv::VideoCapture class. The video stream is provided through the regular
|
||||
camera interface.
|
||||
|
||||
### Installation Instructions
|
||||
|
||||
In order to use the Astra camera's depth sensor with OpenCV you should do the following steps:
|
||||
|
||||
-# Download the latest version of Orbbec OpenNI SDK (from here <https://orbbec3d.com/develop/>).
|
||||
Unzip the archive, choose the build according to your operating system and follow installation
|
||||
steps provided in the Readme file. For instance, if you use 64bit GNU/Linux run:
|
||||
@code{.bash}
|
||||
$ cd Linux/OpenNI-Linux-x64-2.3.0.63/
|
||||
$ sudo ./install.sh
|
||||
@endcode
|
||||
When you are done with the installation, make sure to replug your device for udev rules to take
|
||||
effect. The camera should now work as a general camera device. Note that your current user should
|
||||
belong to group `video` to have access to the camera. Also, make sure to source `OpenNIDevEnvironment` file:
|
||||
@code{.bash}
|
||||
$ source OpenNIDevEnvironment
|
||||
@endcode
|
||||
|
||||
-# Run the following commands to verify that OpenNI library and header files can be found. You should see
|
||||
something similar in your terminal:
|
||||
@code{.bash}
|
||||
$ echo $OPENNI2_INCLUDE
|
||||
/home/user/OpenNI_2.3.0.63/Linux/OpenNI-Linux-x64-2.3.0.63/Include
|
||||
$ echo $OPENNI2_REDIST
|
||||
/home/user/OpenNI_2.3.0.63/Linux/OpenNI-Linux-x64-2.3.0.63/Redist
|
||||
@endcode
|
||||
If the above two variables are empty, then you need to source `OpenNIDevEnvironment` again. Now you can
|
||||
configure OpenCV with OpenNI support enabled by setting the `WITH_OPENNI2` flag in CMake.
|
||||
You may also like to enable the `BUILD_EXAMPLES` flag to get a code sample working with your Astra camera.
|
||||
Run the following commands in the directory containing OpenCV source code to enable OpenNI support:
|
||||
@code{.bash}
|
||||
$ mkdir build
|
||||
$ cd build
|
||||
$ cmake -DWITH_OPENNI2=ON ..
|
||||
@endcode
|
||||
If the OpenNI library is found, OpenCV will be built with OpenNI2 support. You can see the status of OpenNI2
|
||||
support in the CMake log:
|
||||
@code{.text}
|
||||
-- Video I/O:
|
||||
-- DC1394: YES (2.2.6)
|
||||
-- FFMPEG: YES
|
||||
-- avcodec: YES (58.91.100)
|
||||
-- avformat: YES (58.45.100)
|
||||
-- avutil: YES (56.51.100)
|
||||
-- swscale: YES (5.7.100)
|
||||
-- avresample: NO
|
||||
-- GStreamer: YES (1.18.1)
|
||||
-- OpenNI2: YES (2.3.0)
|
||||
-- v4l/v4l2: YES (linux/videodev2.h)
|
||||
@endcode
|
||||
|
||||
-# Build OpenCV:
|
||||
@code{.bash}
|
||||
$ make
|
||||
@endcode
|
||||
|
||||
### Code
|
||||
|
||||
The Astra Pro camera has two sensors -- a depth sensor and a color sensor. The depth sensor
|
||||
can be read using the OpenNI interface with @ref cv::VideoCapture class. The video stream is
|
||||
not available through OpenNI API and is only provided via the regular camera interface.
|
||||
So, to get both depth and color frames, two @ref cv::VideoCapture objects should be created:
|
||||
|
||||
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Open streams
|
||||
|
||||
The first object will use the OpenNI2 API to retrieve depth data. The second one uses the
|
||||
Video4Linux2 interface to access the color sensor. Note that the example above assumes that
|
||||
the Astra camera is the first camera in the system. If you have more than one camera connected,
|
||||
you may need to explicitly set the proper camera number.
|
||||
|
||||
Before using the created VideoCapture objects you may want to set up stream parameters by setting
|
||||
objects' properties. The most important parameters are frame width, frame height and fps.
|
||||
For this example, we’ll configure width and height of both streams to VGA resolution, which is
|
||||
the maximum resolution available for both sensors, and we’d like both stream parameters to be the
|
||||
same for easier color-to-depth data registration:
|
||||
|
||||
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Setup streams
|
||||
|
||||
For setting and retrieving some property of sensor data generators use @ref cv::VideoCapture::set and
|
||||
@ref cv::VideoCapture::get methods respectively, e.g. :
|
||||
|
||||
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Get properties
|
||||
|
||||
The following properties of cameras available through OpenNI interface are supported for the depth
|
||||
generator:
|
||||
|
||||
- @ref cv::CAP_PROP_FRAME_WIDTH -- Frame width in pixels.
|
||||
- @ref cv::CAP_PROP_FRAME_HEIGHT -- Frame height in pixels.
|
||||
- @ref cv::CAP_PROP_FPS -- Frame rate in FPS.
|
||||
- @ref cv::CAP_PROP_OPENNI_REGISTRATION -- Flag that registers the remapping depth map to image map
|
||||
by changing the depth generator's viewpoint (if the flag is "on") or sets this view point to
|
||||
its normal one (if the flag is "off"). The registration process’ resulting images are
|
||||
pixel-aligned, which means that every pixel in the image is aligned to a pixel in the depth
|
||||
image.
|
||||
- @ref cv::CAP_PROP_OPENNI2_MIRROR -- Flag to enable or disable mirroring for this stream. Set to 0
|
||||
to disable mirroring
|
||||
|
||||
Next properties are available for getting only:
|
||||
|
||||
- @ref cv::CAP_PROP_OPENNI_FRAME_MAX_DEPTH -- A maximum supported depth of the camera in mm.
|
||||
- @ref cv::CAP_PROP_OPENNI_BASELINE -- Baseline value in mm.
|
||||
|
||||
After the VideoCapture objects have been set up, you can start reading frames from them.
|
||||
|
||||
@note
|
||||
OpenCV's VideoCapture provides synchronous API, so you have to grab frames in a new thread
|
||||
to avoid one stream blocking while another stream is being read. VideoCapture is not a
|
||||
thread-safe class, so you need to be careful to avoid any possible deadlocks or data races.
|
||||
|
||||
As there are two video sources that should be read simultaneously, it’s necessary to create two
|
||||
threads to avoid blocking. Example implementation that gets frames from each sensor in a new thread
|
||||
and stores them in a list along with their timestamps:
|
||||
|
||||
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Read streams
|
||||
|
||||
VideoCapture can retrieve the following data:
|
||||
|
||||
-# data given from the depth generator:
|
||||
- @ref cv::CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1)
|
||||
- @ref cv::CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3)
|
||||
- @ref cv::CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1)
|
||||
- @ref cv::CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1)
|
||||
- @ref cv::CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not occluded, not shaded, etc.)
|
||||
(CV_8UC1)
|
||||
|
||||
-# data given from the color sensor is a regular BGR image (CV_8UC3).
|
||||
|
||||
When new data are available, each reading thread notifies the main thread using a condition variable.
|
||||
A frame is stored in the ordered list -- the first frame in the list is the earliest captured,
|
||||
the last frame is the latest captured. As depth and color frames are read from independent sources
|
||||
two video streams may become out of sync even when both streams are set up for the same frame rate.
|
||||
A post-synchronization procedure can be applied to the streams to combine depth and color frames into
|
||||
pairs. The sample code below demonstrates this procedure:
|
||||
|
||||
@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Pair frames
|
||||
|
||||
In the code snippet above the execution is blocked until there are some frames in both frame lists.
|
||||
When there are new frames, their timestamps are being checked -- if they differ more than a half of
|
||||
the frame period then one of the frames is dropped. If timestamps are close enough, then two frames
|
||||
are paired. Now, we have two frames: one containing color information and another one -- depth information.
|
||||
In the example above retrieved frames are simply shown with cv::imshow function, but you can insert
|
||||
any other processing code here.
|
||||
|
||||
In the sample images below you can see the color frame and the depth frame representing the same scene.
|
||||
Looking at the color frame it's hard to distinguish plant leaves from leaves painted on a wall,
|
||||
but the depth data makes it easy.
|
||||
|
||||

|
||||

|
||||
|
||||
The complete implementation can be found in
|
||||
[orbbec_astra.cpp](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp)
|
||||
in `samples/cpp/tutorial_code/videoio` directory.
|
||||
107
doc/tutorials/app/raster_io_gdal.markdown
Normal file
@@ -0,0 +1,107 @@
|
||||
Reading Geospatial Raster files with GDAL {#tutorial_raster_io_gdal}
|
||||
=========================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_trackbar}
|
||||
@next_tutorial{tutorial_video_input_psnr_ssim}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Marvin Smith |
|
||||
| Compatibility | OpenCV >= 3.0 |
|
||||
|
||||
Geospatial raster data is a heavily used product in Geographic Information Systems and
|
||||
Photogrammetry. Raster data typically can represent imagery and Digital Elevation Models (DEM). The
|
||||
standard library for loading GIS imagery is the Geographic Data Abstraction Library [(GDAL)](http://www.gdal.org). In this
|
||||
example, we will show techniques for loading GIS raster formats using native OpenCV functions. In
|
||||
addition, we will show some an example of how OpenCV can use this data for novel and interesting
|
||||
purposes.
|
||||
|
||||
Goals
|
||||
-----
|
||||
|
||||
The primary objectives for this tutorial:
|
||||
|
||||
- How to use OpenCV [imread](@ref imread) to load satellite imagery.
|
||||
- How to use OpenCV [imread](@ref imread) to load SRTM Digital Elevation Models
|
||||
- Given the corner coordinates of both the image and DEM, correlate the elevation data to the
|
||||
image to find elevations for each pixel.
|
||||
- Show a basic, easy-to-implement example of a terrain heat map.
|
||||
- Show a basic use of DEM data coupled with ortho-rectified imagery.
|
||||
|
||||
To implement these goals, the following code takes a Digital Elevation Model as well as a GeoTiff
|
||||
image of San Francisco as input. The image and DEM data is processed and generates a terrain heat
|
||||
map of the image as well as labels areas of the city which would be affected should the water level
|
||||
of the bay rise 10, 50, and 100 meters.
|
||||
|
||||
Code
|
||||
----
|
||||
|
||||
@include cpp/tutorial_code/imgcodecs/GDAL_IO/gdal-image.cpp
|
||||
|
||||
How to Read Raster Data using GDAL
|
||||
----------------------------------
|
||||
|
||||
This demonstration uses the default OpenCV imread function. The primary difference is that in order
|
||||
to force GDAL to load the image, you must use the appropriate flag.
|
||||
@snippet cpp/tutorial_code/imgcodecs/GDAL_IO/gdal-image.cpp load1
|
||||
When loading digital elevation models, the actual numeric value of each pixel is essential and
|
||||
cannot be scaled or truncated. For example, with image data a pixel represented as a double with a
|
||||
value of 1 has an equal appearance to a pixel which is represented as an unsigned character with a
|
||||
value of 255. With terrain data, the pixel value represents the elevation in meters. In order to
|
||||
ensure that OpenCV preserves the native value, use the GDAL flag in imread with the ANYDEPTH flag.
|
||||
@snippet cpp/tutorial_code/imgcodecs/GDAL_IO/gdal-image.cpp load2
|
||||
If you know beforehand the type of DEM model you are loading, then it may be a safe bet to test the
|
||||
Mat::type() or Mat::depth() using an assert or other mechanism. NASA or DOD specification documents
|
||||
can provide the input types for various elevation models. The major types, SRTM and DTED, are both
|
||||
signed shorts.
|
||||
|
||||
Notes
|
||||
-----
|
||||
|
||||
### Lat/Lon (Geographic) Coordinates should normally be avoided
|
||||
|
||||
The Geographic Coordinate System is a spherical coordinate system, meaning that using them with
|
||||
Cartesian mathematics is technically incorrect. This demo uses them to increase the readability and
|
||||
is accurate enough to make the point. A better coordinate system would be Universal Transverse
|
||||
Mercator.
|
||||
|
||||
### Finding the corner coordinates
|
||||
|
||||
One easy method to find the corner coordinates of an image is to use the command-line tool gdalinfo.
|
||||
For imagery which is ortho-rectified and contains the projection information, you can use the [USGS
|
||||
EarthExplorer](http://http://earthexplorer.usgs.gov).
|
||||
@code{.bash}
|
||||
\f$> gdalinfo N37W123.hgt
|
||||
|
||||
Driver: SRTMHGT/SRTMHGT File Format
|
||||
Files: N37W123.hgt
|
||||
Size is 3601, 3601
|
||||
Coordinate System is:
|
||||
GEOGCS["WGS 84",
|
||||
DATUM["WGS_1984",
|
||||
|
||||
... more output ...
|
||||
|
||||
Corner Coordinates:
|
||||
Upper Left (-123.0001389, 38.0001389) (123d 0' 0.50"W, 38d 0' 0.50"N)
|
||||
Lower Left (-123.0001389, 36.9998611) (123d 0' 0.50"W, 36d59'59.50"N)
|
||||
Upper Right (-121.9998611, 38.0001389) (121d59'59.50"W, 38d 0' 0.50"N)
|
||||
Lower Right (-121.9998611, 36.9998611) (121d59'59.50"W, 36d59'59.50"N)
|
||||
Center (-122.5000000, 37.5000000) (122d30' 0.00"W, 37d30' 0.00"N)
|
||||
|
||||
... more output ...
|
||||
@endcode
|
||||
Results
|
||||
-------
|
||||
|
||||
Below is the output of the program. Use the first image as the input. For the DEM model, download
|
||||
the SRTM file located at the USGS here.
|
||||
[<http://dds.cr.usgs.gov/srtm/version2_1/SRTM1/Region_04/N37W123.hgt.zip>](http://dds.cr.usgs.gov/srtm/version2_1/SRTM1/Region_04/N37W123.hgt.zip)
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
10
doc/tutorials/app/table_of_content_app.markdown
Normal file
@@ -0,0 +1,10 @@
|
||||
Application utils (highgui, imgcodecs, videoio modules) {#tutorial_table_of_content_app}
|
||||
=======================================================
|
||||
|
||||
- @subpage tutorial_trackbar
|
||||
- @subpage tutorial_raster_io_gdal
|
||||
- @subpage tutorial_video_input_psnr_ssim
|
||||
- @subpage tutorial_video_write
|
||||
- @subpage tutorial_kinect_openni
|
||||
- @subpage tutorial_orbbec_astra
|
||||
- @subpage tutorial_intelperc
|
||||
142
doc/tutorials/app/trackbar.markdown
Normal file
@@ -0,0 +1,142 @@
|
||||
Adding a Trackbar to our applications! {#tutorial_trackbar}
|
||||
======================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@next_tutorial{tutorial_raster_io_gdal}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Ana Huamán |
|
||||
| Compatibility | OpenCV >= 3.0 |
|
||||
|
||||
|
||||
- In the previous tutorials (about @ref tutorial_adding_images and the @ref tutorial_basic_linear_transform)
|
||||
you might have noted that we needed to give some **input** to our programs, such
|
||||
as \f$\alpha\f$ and \f$beta\f$. We accomplished that by entering this data using the Terminal.
|
||||
- Well, it is time to use some fancy GUI tools. OpenCV provides some GUI utilities (**highgui** module)
|
||||
for you. An example of this is a **Trackbar**.
|
||||
|
||||

|
||||
|
||||
- In this tutorial we will just modify our two previous programs so that they get the input
|
||||
information from the trackbar.
|
||||
|
||||
Goals
|
||||
-----
|
||||
|
||||
In this tutorial you will learn how to:
|
||||
|
||||
- Add a Trackbar in an OpenCV window by using @ref cv::createTrackbar
|
||||
|
||||
Code
|
||||
----
|
||||
|
||||
Let's modify the program made in the tutorial @ref tutorial_adding_images. We will let the user enter the
|
||||
\f$\alpha\f$ value by using the Trackbar.
|
||||
|
||||
@add_toggle_cpp
|
||||
This tutorial code's is shown lines below. You can also download it from
|
||||
[here](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp)
|
||||
@include cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
This tutorial code's is shown lines below. You can also download it from
|
||||
[here](https://github.com/opencv/opencv/tree/master/samples/java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java)
|
||||
@include java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
This tutorial code's is shown lines below. You can also download it from
|
||||
[here](https://github.com/opencv/opencv/tree/master/samples/python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py)
|
||||
@include python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py
|
||||
@end_toggle
|
||||
|
||||
Explanation
|
||||
-----------
|
||||
|
||||
We only analyze the code that is related to Trackbar:
|
||||
|
||||
- First, we load two images, which are going to be blended.
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp load
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java load
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py load
|
||||
@end_toggle
|
||||
|
||||
- To create a trackbar, first we have to create the window in which it is going to be located. So:
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp window
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java window
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py window
|
||||
@end_toggle
|
||||
|
||||
- Now we can create the Trackbar:
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp create_trackbar
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java create_trackbar
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py create_trackbar
|
||||
@end_toggle
|
||||
|
||||
Note the following (C++ code):
|
||||
- Our Trackbar has a label **TrackbarName**
|
||||
- The Trackbar is located in the window named **Linear Blend**
|
||||
- The Trackbar values will be in the range from \f$0\f$ to **alpha_slider_max** (the minimum
|
||||
limit is always **zero**).
|
||||
- The numerical value of Trackbar is stored in **alpha_slider**
|
||||
- Whenever the user moves the Trackbar, the callback function **on_trackbar** is called
|
||||
|
||||
Finally, we have to define the callback function **on_trackbar** for C++ and Python code, using an anonymous inner class listener in Java
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet cpp/tutorial_code/HighGUI/AddingImagesTrackbar.cpp on_trackbar
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet java/tutorial_code/highgui/trackbar/AddingImagesTrackbar.java on_trackbar
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet python/tutorial_code/highgui/trackbar/AddingImagesTrackbar.py on_trackbar
|
||||
@end_toggle
|
||||
|
||||
Note that (C++ code):
|
||||
- We use the value of **alpha_slider** (integer) to get a double value for **alpha**.
|
||||
- **alpha_slider** is updated each time the trackbar is displaced by the user.
|
||||
- We define *src1*, *src2*, *dist*, *alpha*, *alpha_slider* and *beta* as global variables,
|
||||
so they can be used everywhere.
|
||||
|
||||
Result
|
||||
------
|
||||
|
||||
- Our program produces the following output:
|
||||
|
||||

|
||||
|
||||
- As a manner of practice, you can also add two trackbars for the program made in
|
||||
@ref tutorial_basic_linear_transform. One trackbar to set \f$\alpha\f$ and another for set \f$\beta\f$. The output might
|
||||
look like:
|
||||
|
||||

|
||||
203
doc/tutorials/app/video_input_psnr_ssim.markdown
Normal file
@@ -0,0 +1,203 @@
|
||||
Video Input with OpenCV and similarity measurement {#tutorial_video_input_psnr_ssim}
|
||||
==================================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_raster_io_gdal}
|
||||
@next_tutorial{tutorial_video_write}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Bernát Gábor |
|
||||
| Compatibility | OpenCV >= 3.0 |
|
||||
|
||||
Goal
|
||||
----
|
||||
|
||||
Today it is common to have a digital video recording system at your disposal. Therefore, you will
|
||||
eventually come to the situation that you no longer process a batch of images, but video streams.
|
||||
These may be of two kinds: real-time image feed (in the case of a webcam) or prerecorded and hard
|
||||
disk drive stored files. Luckily OpenCV treats these two in the same manner, with the same C++
|
||||
class. So here's what you'll learn in this tutorial:
|
||||
|
||||
- How to open and read video streams
|
||||
- Two ways for checking image similarity: PSNR and SSIM
|
||||
|
||||
The source code
|
||||
---------------
|
||||
|
||||
As a test case where to show off these using OpenCV I've created a small program that reads in two
|
||||
video files and performs a similarity check between them. This is something you could use to check
|
||||
just how well a new video compressing algorithms works. Let there be a reference (original) video
|
||||
like [this small Megamind clip
|
||||
](https://github.com/opencv/opencv/tree/master/samples/data/Megamind.avi) and [a compressed
|
||||
version of it ](https://github.com/opencv/opencv/tree/master/samples/data/Megamind_bugy.avi).
|
||||
You may also find the source code and these video file in the
|
||||
`samples/data` folder of the OpenCV source library.
|
||||
|
||||
@add_toggle_cpp
|
||||
@include cpp/tutorial_code/videoio/video-input-psnr-ssim/video-input-psnr-ssim.cpp
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@include samples/python/tutorial_code/videoio/video-input-psnr-ssim.py
|
||||
@end_toggle
|
||||
|
||||
How to read a video stream (online-camera or offline-file)?
|
||||
-----------------------------------------------------------
|
||||
|
||||
Essentially, all the functionalities required for video manipulation is integrated in the @ref cv::VideoCapture
|
||||
C++ class. This on itself builds on the FFmpeg open source library. This is a basic
|
||||
dependency of OpenCV so you shouldn't need to worry about this. A video is composed of a succession
|
||||
of images, we refer to these in the literature as frames. In case of a video file there is a *frame
|
||||
rate* specifying just how long is between two frames. While for the video cameras usually there is a
|
||||
limit of just how many frames they can digitize per second, this property is less important as at
|
||||
any time the camera sees the current snapshot of the world.
|
||||
|
||||
The first task you need to do is to assign to a @ref cv::VideoCapture class its source. You can do
|
||||
this either via the @ref cv::VideoCapture::VideoCapture or its @ref cv::VideoCapture::open function. If this argument is an
|
||||
integer then you will bind the class to a camera, a device. The number passed here is the ID of the
|
||||
device, assigned by the operating system. If you have a single camera attached to your system its ID
|
||||
will probably be zero and further ones increasing from there. If the parameter passed to these is a
|
||||
string it will refer to a video file, and the string points to the location and name of the file.
|
||||
For example, to the upper source code a valid command line is:
|
||||
@code{.bash}
|
||||
video/Megamind.avi video/Megamind_bug.avi 35 10
|
||||
@endcode
|
||||
We do a similarity check. This requires a reference and a test case video file. The first two
|
||||
arguments refer to this. Here we use a relative address. This means that the application will look
|
||||
into its current working directory and open the video folder and try to find inside this the
|
||||
*Megamind.avi* and the *Megamind_bug.avi*.
|
||||
@code{.cpp}
|
||||
const string sourceReference = argv[1],sourceCompareWith = argv[2];
|
||||
|
||||
VideoCapture captRefrnc(sourceReference);
|
||||
// or
|
||||
VideoCapture captUndTst;
|
||||
captUndTst.open(sourceCompareWith);
|
||||
@endcode
|
||||
To check if the binding of the class to a video source was successful or not use the @ref cv::VideoCapture::isOpened
|
||||
function:
|
||||
@code{.cpp}
|
||||
if ( !captRefrnc.isOpened())
|
||||
{
|
||||
cout << "Could not open reference " << sourceReference << endl;
|
||||
return -1;
|
||||
}
|
||||
@endcode
|
||||
Closing the video is automatic when the objects destructor is called. However, if you want to close
|
||||
it before this you need to call its @ref cv::VideoCapture::release function. The frames of the video are just
|
||||
simple images. Therefore, we just need to extract them from the @ref cv::VideoCapture object and put
|
||||
them inside a *Mat* one. The video streams are sequential. You may get the frames one after another
|
||||
by the @ref cv::VideoCapture::read or the overloaded \>\> operator:
|
||||
@code{.cpp}
|
||||
Mat frameReference, frameUnderTest;
|
||||
captRefrnc >> frameReference;
|
||||
captUndTst.read(frameUnderTest);
|
||||
@endcode
|
||||
The upper read operations will leave empty the *Mat* objects if no frame could be acquired (either
|
||||
cause the video stream was closed or you got to the end of the video file). We can check this with a
|
||||
simple if:
|
||||
@code{.cpp}
|
||||
if( frameReference.empty() || frameUnderTest.empty())
|
||||
{
|
||||
// exit the program
|
||||
}
|
||||
@endcode
|
||||
A read method is made of a frame grab and a decoding applied on that. You may call explicitly these
|
||||
two by using the @ref cv::VideoCapture::grab and then the @ref cv::VideoCapture::retrieve functions.
|
||||
|
||||
Videos have many-many information attached to them besides the content of the frames. These are
|
||||
usually numbers, however in some case it may be short character sequences (4 bytes or less). Due to
|
||||
this to acquire these information there is a general function named @ref cv::VideoCapture::get that returns double
|
||||
values containing these properties. Use bitwise operations to decode the characters from a double
|
||||
type and conversions where valid values are only integers. Its single argument is the ID of the
|
||||
queried property. For example, here we get the size of the frames in the reference and test case
|
||||
video file; plus the number of frames inside the reference.
|
||||
@code{.cpp}
|
||||
Size refS = Size((int) captRefrnc.get(CAP_PROP_FRAME_WIDTH),
|
||||
(int) captRefrnc.get(CAP_PROP_FRAME_HEIGHT)),
|
||||
|
||||
cout << "Reference frame resolution: Width=" << refS.width << " Height=" << refS.height
|
||||
<< " of nr#: " << captRefrnc.get(CAP_PROP_FRAME_COUNT) << endl;
|
||||
@endcode
|
||||
When you are working with videos you may often want to control these values yourself. To do this
|
||||
there is a @ref cv::VideoCapture::set function. Its first argument remains the name of the property you want to
|
||||
change and there is a second of double type containing the value to be set. It will return true if
|
||||
it succeeds and false otherwise. Good examples for this is seeking in a video file to a given time
|
||||
or frame:
|
||||
@code{.cpp}
|
||||
captRefrnc.set(CAP_PROP_POS_MSEC, 1.2); // go to the 1.2 second in the video
|
||||
captRefrnc.set(CAP_PROP_POS_FRAMES, 10); // go to the 10th frame of the video
|
||||
// now a read operation would read the frame at the set position
|
||||
@endcode
|
||||
For properties you can read and change look into the documentation of the @ref cv::VideoCapture::get and
|
||||
@ref cv::VideoCapture::set functions.
|
||||
|
||||
### Image similarity - PSNR and SSIM
|
||||
|
||||
We want to check just how imperceptible our video converting operation went, therefore we need a
|
||||
system to check frame by frame the similarity or differences. The most common algorithm used for
|
||||
this is the PSNR (aka **Peak signal-to-noise ratio**). The simplest definition of this starts out
|
||||
from the *mean squared error*. Let there be two images: I1 and I2; with a two dimensional size i and
|
||||
j, composed of c number of channels.
|
||||
|
||||
\f[MSE = \frac{1}{c*i*j} \sum{(I_1-I_2)^2}\f]
|
||||
|
||||
Then the PSNR is expressed as:
|
||||
|
||||
\f[PSNR = 10 \cdot \log_{10} \left( \frac{MAX_I^2}{MSE} \right)\f]
|
||||
|
||||
Here the \f$MAX_I\f$ is the maximum valid value for a pixel. In case of the simple single byte image
|
||||
per pixel per channel this is 255. When two images are the same the MSE will give zero, resulting in
|
||||
an invalid divide by zero operation in the PSNR formula. In this case the PSNR is undefined and as
|
||||
we'll need to handle this case separately. The transition to a logarithmic scale is made because the
|
||||
pixel values have a very wide dynamic range. All this translated to OpenCV and a function looks
|
||||
like:
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet cpp/tutorial_code/videoio/video-input-psnr-ssim/video-input-psnr-ssim.cpp get-psnr
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/videoio/video-input-psnr-ssim.py get-psnr
|
||||
@end_toggle
|
||||
|
||||
Typically result values are anywhere between 30 and 50 for video compression, where higher is
|
||||
better. If the images significantly differ you'll get much lower ones like 15 and so. This
|
||||
similarity check is easy and fast to calculate, however in practice it may turn out somewhat
|
||||
inconsistent with human eye perception. The **structural similarity** algorithm aims to correct
|
||||
this.
|
||||
|
||||
Describing the methods goes well beyond the purpose of this tutorial. For that I invite you to read
|
||||
the article introducing it. Nevertheless, you can get a good image of it by looking at the OpenCV
|
||||
implementation below.
|
||||
|
||||
@note
|
||||
SSIM is described more in-depth in the: "Z. Wang, A. C. Bovik, H. R. Sheikh and E. P.
|
||||
Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE
|
||||
Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004." article.
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/videoio/video-input-psnr-ssim/video-input-psnr-ssim.cpp get-mssim
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/videoio/video-input-psnr-ssim.py get-mssim
|
||||
@end_toggle
|
||||
|
||||
This will return a similarity index for each channel of the image. This value is between zero and
|
||||
one, where one corresponds to perfect fit. Unfortunately, the many Gaussian blurring is quite
|
||||
costly, so while the PSNR may work in a real time like environment (24 frame per second) this will
|
||||
take significantly more than to accomplish similar performance results.
|
||||
|
||||
Therefore, the source code presented at the start of the tutorial will perform the PSNR measurement
|
||||
for each frame, and the SSIM only for the frames where the PSNR falls below an input value. For
|
||||
visualization purpose we show both images in an OpenCV window and print the PSNR and MSSIM values to
|
||||
the console. Expect to see something like:
|
||||
|
||||

|
||||
|
||||
You may observe a runtime instance of this on the [YouTube here](https://www.youtube.com/watch?v=iOcNljutOgg).
|
||||
|
||||
@youtube{iOcNljutOgg}
|
||||
166
doc/tutorials/app/video_write.markdown
Normal file
@@ -0,0 +1,166 @@
|
||||
Creating a video with OpenCV {#tutorial_video_write}
|
||||
============================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_video_input_psnr_ssim}
|
||||
@next_tutorial{tutorial_kinect_openni}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Bernát Gábor |
|
||||
| Compatibility | OpenCV >= 3.0 |
|
||||
|
||||
Goal
|
||||
----
|
||||
|
||||
Whenever you work with video feeds you may eventually want to save your image processing result in a
|
||||
form of a new video file. For simple video outputs you can use the OpenCV built-in @ref cv::VideoWriter
|
||||
class, designed for this.
|
||||
|
||||
- How to create a video file with OpenCV
|
||||
- What type of video files you can create with OpenCV
|
||||
- How to extract a given color channel from a video
|
||||
|
||||
As a simple demonstration I'll just extract one of the BGR color channels of an input video file
|
||||
into a new video. You can control the flow of the application from its console line arguments:
|
||||
|
||||
- The first argument points to the video file to work on
|
||||
- The second argument may be one of the characters: R G B. This will specify which of the channels
|
||||
to extract.
|
||||
- The last argument is the character Y (Yes) or N (No). If this is no, the codec used for the
|
||||
input video file will be the same as for the output. Otherwise, a window will pop up and allow
|
||||
you to select yourself the codec to use.
|
||||
|
||||
For example, a valid command line would look like:
|
||||
@code{.bash}
|
||||
video-write.exe video/Megamind.avi R Y
|
||||
@endcode
|
||||
The source code
|
||||
---------------
|
||||
|
||||
You may also find the source code and these video file in the
|
||||
`samples/cpp/tutorial_code/videoio/video-write/` folder of the OpenCV source library or [download it
|
||||
from here ](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/videoio/video-write/video-write.cpp).
|
||||
|
||||
@include cpp/tutorial_code/videoio/video-write/video-write.cpp
|
||||
|
||||
The structure of a video
|
||||
------------------------
|
||||
|
||||
For start, you should have an idea of just how a video file looks. Every video file in itself is a
|
||||
container. The type of the container is expressed in the files extension (for example *avi*, *mov*
|
||||
or *mkv*). This contains multiple elements like: video feeds, audio feeds or other tracks (like for
|
||||
example subtitles). How these feeds are stored is determined by the codec used for each one of them.
|
||||
In case of the audio tracks commonly used codecs are *mp3* or *aac*. For the video files the list is
|
||||
somehow longer and includes names such as *XVID*, *DIVX*, *H264* or *LAGS* (*Lagarith Lossless
|
||||
Codec*). The full list of codecs you may use on a system depends on just what one you have
|
||||
installed.
|
||||
|
||||

|
||||
|
||||
As you can see things can get really complicated with videos. However, OpenCV is mainly a computer
|
||||
vision library, not a video stream, codec and write one. Therefore, the developers tried to keep
|
||||
this part as simple as possible. Due to this OpenCV for video containers supports only the *avi*
|
||||
extension, its first version. A direct limitation of this is that you cannot save a video file
|
||||
larger than 2 GB. Furthermore you can only create and expand a single video track inside the
|
||||
container. No audio or other track editing support here. Nevertheless, any video codec present on
|
||||
your system might work. If you encounter some of these limitations you will need to look into more
|
||||
specialized video writing libraries such as *FFmpeg* or codecs as *HuffYUV*, *CorePNG* and *LCL*. As
|
||||
an alternative, create the video track with OpenCV and expand it with sound tracks or convert it to
|
||||
other formats by using video manipulation programs such as *VirtualDub* or *AviSynth*.
|
||||
|
||||
The VideoWriter class
|
||||
-----------------------
|
||||
|
||||
The content written here builds on the assumption you
|
||||
already read the @ref tutorial_video_input_psnr_ssim tutorial and you know how to read video files. To create a
|
||||
video file you just need to create an instance of the @ref cv::VideoWriter class. You can specify
|
||||
its properties either via parameters in the constructor or later on via the @ref cv::VideoWriter::open function.
|
||||
Either way, the parameters are the same: 1. The name of the output that contains the container type
|
||||
in its extension. At the moment only *avi* is supported. We construct this from the input file, add
|
||||
to this the name of the channel to use, and finish it off with the container extension.
|
||||
@code{.cpp}
|
||||
const string source = argv[1]; // the source file name
|
||||
string::size_type pAt = source.find_last_of('.'); // Find extension point
|
||||
const string NAME = source.substr(0, pAt) + argv[2][0] + ".avi"; // Form the new name with container
|
||||
@endcode
|
||||
-# The codec to use for the video track. Now all the video codecs have a unique short name of
|
||||
maximum four characters. Hence, the *XVID*, *DIVX* or *H264* names. This is called a four
|
||||
character code. You may also ask this from an input video by using its *get* function. Because
|
||||
the *get* function is a general function it always returns double values. A double value is
|
||||
stored on 64 bits. Four characters are four bytes, meaning 32 bits. These four characters are
|
||||
coded in the lower 32 bits of the *double*. A simple way to throw away the upper 32 bits would
|
||||
be to just convert this value to *int*:
|
||||
@code{.cpp}
|
||||
VideoCapture inputVideo(source); // Open input
|
||||
int ex = static_cast<int>(inputVideo.get(CAP_PROP_FOURCC)); // Get Codec Type- Int form
|
||||
@endcode
|
||||
OpenCV internally works with this integer type and expect this as its second parameter. Now to
|
||||
convert from the integer form to string we may use two methods: a bitwise operator and a union
|
||||
method. The first one extracting from an int the characters looks like (an "and" operation, some
|
||||
shifting and adding a 0 at the end to close the string):
|
||||
@code{.cpp}
|
||||
char EXT[] = {ex & 0XFF , (ex & 0XFF00) >> 8,(ex & 0XFF0000) >> 16,(ex & 0XFF000000) >> 24, 0};
|
||||
@endcode
|
||||
You can do the same thing with the *union* as:
|
||||
@code{.cpp}
|
||||
union { int v; char c[5];} uEx ;
|
||||
uEx.v = ex; // From Int to char via union
|
||||
uEx.c[4]='\0';
|
||||
@endcode
|
||||
The advantage of this is that the conversion is done automatically after assigning, while for
|
||||
the bitwise operator you need to do the operations whenever you change the codec type. In case
|
||||
you know the codecs four character code beforehand, you can use the *CV_FOURCC* macro to build
|
||||
the integer:
|
||||
@code{.cpp}
|
||||
CV_FOURCC('P','I','M,'1') // this is an MPEG1 codec from the characters to integer
|
||||
@endcode
|
||||
If you pass for this argument minus one then a window will pop up at runtime that contains all
|
||||
the codec installed on your system and ask you to select the one to use:
|
||||
|
||||

|
||||
|
||||
-# The frame per second for the output video. Again, here I keep the input videos frame per second
|
||||
by using the *get* function.
|
||||
-# The size of the frames for the output video. Here too I keep the input videos frame size per
|
||||
second by using the *get* function.
|
||||
-# The final argument is an optional one. By default is true and says that the output will be a
|
||||
colorful one (so for write you will send three channel images). To create a gray scale video
|
||||
pass a false parameter here.
|
||||
|
||||
Here it is, how I use it in the sample:
|
||||
@code{.cpp}
|
||||
VideoWriter outputVideo;
|
||||
Size S = Size((int) inputVideo.get(CAP_PROP_FRAME_WIDTH), //Acquire input size
|
||||
(int) inputVideo.get(CAP_PROP_FRAME_HEIGHT));
|
||||
outputVideo.open(NAME , ex, inputVideo.get(CAP_PROP_FPS),S, true);
|
||||
@endcode
|
||||
Afterwards, you use the @ref cv::VideoWriter::isOpened() function to find out if the open operation succeeded or
|
||||
not. The video file automatically closes when the *VideoWriter* object is destroyed. After you open
|
||||
the object with success you can send the frames of the video in a sequential order by using the
|
||||
@ref cv::VideoWriter::write function of the class. Alternatively, you can use its overloaded operator \<\< :
|
||||
@code{.cpp}
|
||||
outputVideo.write(res); //or
|
||||
outputVideo << res;
|
||||
@endcode
|
||||
Extracting a color channel from an BGR image means to set to zero the BGR values of the other
|
||||
channels. You can either do this with image scanning operations or by using the split and merge
|
||||
operations. You first split the channels up into different images, set the other channels to zero
|
||||
images of the same size and type and finally merge them back:
|
||||
@code{.cpp}
|
||||
split(src, spl); // process - extract only the correct channel
|
||||
for( int i =0; i < 3; ++i)
|
||||
if (i != channel)
|
||||
spl[i] = Mat::zeros(S, spl[0].type());
|
||||
merge(spl, res);
|
||||
@endcode
|
||||
Put all this together and you'll get the upper source code, whose runtime result will show something
|
||||
around the idea:
|
||||
|
||||

|
||||
|
||||
You may observe a runtime instance of this on the [YouTube
|
||||
here](https://www.youtube.com/watch?v=jpBwHxsl1_0).
|
||||
|
||||
@youtube{jpBwHxsl1_0}
|
||||