PW 1J - Image processing basics [OpenCV/JavaCV]

This practical work illustrates image processing basics using the OpenCV library within a Java environment and JavaCV.

1. Introduction

1.1. OpenCV

OpenCV (Open Computer Vision) is a graphics library. It is specialized in image processing, whether for images or video. 

OpenCV provides data structures dedicated to images representation and a set of over 2,500 computer vision algorithms, accessible via APIs. This makes it possible to perform a wide range of computer vision functions. The library currently covers various fields including for example 2D geometry, 3D geometry, image processing, 3D reconstruction, object detection and machine learning (CNN / DNN).

OpenCV is developped in C++ and provide wrappers for many others languages and SDL (Python, Java, MatLab, ...)

The versatility and multitude of functions make OpenCV the most widely used library in the computer vision community today. The library is the basis for many image/video processing programs. Today, it is developed, maintained, documented and used by a community of over 40,000 active members.

OpenCV website: https://opencv.org

OpenCV Github: https://github.com/opencv/opencv

1.2. JavaCV

JavaCV is a Java library that aims to provide a full access to OpenCV functionnalities from a Java program.

JavaCV uses wrappers from the JavaCPP Presets of commonly used libraries by researchers in the field of computer vision (OpenCV, FFmpeg, libdc1394, FlyCapture, Spinnaker, OpenKinect, librealsense, CL PS3 Eye Driver, videoInput, ARToolKitPlus, flandmark, Leptonica, and Tesseract) and provides utility classes to make their functionality easier to use on the Java platform, including Android.

JavaCV Github: https://github.com/bytedeco/javacv

2. Getting started

JavaCV is a library that provides OpenCV's native functions from within a Java application. It is freely distributed and can be used for any project requiring computer vision or image processing functions. The library features Maven dependencies and can be easily deployed in an existing project.


Exercise 1.

Download the Maven project archive at http://www.seinturier.fr/teaching/computer_vision/javacv/javacv-basic.zip and unzip it then within the project's root directory (javacv-basic), type the following commands to check its validity:

  • mvn clean
  • mvn compile

check for any errors.

If you are using an IDE (Eclipse, IntelliJ, ...), you can import the Maven project in order to work with.


3. Images I/O

One of OpenCV's essential features is its ability to work with images. Among other things, the library lets you load and save images.

3.1. Image loading

The function imread(filename, flag) enables to load an image from a file. The function takes in parameter the name of the file to load and a flag that represent the color format to use and return a Mat object that represents the image. If the image loading is not a success, the function return an empty Mat. The JavaCV library provides a static wrapping to the imread function.


Exercise 2.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/ImageBasic.java and append the following code to the main method:

String file = "data/images/lake-mountain.png";
		
// Load an image, using its color model
Mat image = imread(file, IMREAD_UNCHANGED);
		
// The image load has failed
if (image.empty()) {
	System.out.println("No data available within image "+file);
}  else {
	System.out.println("Image "+file+" successfully loaed.");
}

To access Mat class,  imread method and IMREAD_UNCHANGED value to be reconized, the following imports have to be added:

import org.bytedeco.opencv.opencv_core.Mat;
import static org.bytedeco.opencv.global.opencv_imgcodecs.imread;
import static org.bytedeco.opencv.global.opencv_imgcodecs.IMREAD_UNCHANGED ;

Compile and run the program in order to check if the image loading is successful.


It is possible to convert image to various color formats using the imread function. The flag parameter enable to specify the image conversion:

  • IMREAD_UNCHANGED keep the image original color format.
  • IMREAD_GRAYSCALE convert the image to gray.
  • IMREAD_COLOR convert the image to BGR color.
  • Other specific flags can be found by reading the OpenCV documentation.

3.2. Image saving

The function imwrite(filename, image) enable to save the given image to a file identified by its filename. The function return true if the image is successfully saved and false othrwise. If an error occurs during the image writing, a RuntimeException is raised.


Exercise 3.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/ImageBasic.java and modify the main method as follows:

public static void main(String[] args) {
	String file = "data/images/lake-mountain.png";
			
	// Load an image, using its color model
	Mat image = imread(file, IMREAD_UNCHANGED);
			
	// The image load has failed, program exit
	if (image.empty()) System.exit(1);
			
	// The filename that locates the place where the image has to be copied
	String copy = "data/images/lake-mountain-copy.png";
			
	// Write the image to the desired location
	try {
		  
		if (imwrite(copy, image)) {
			System.out.println("Image written to "+copy);
		} else {
			System.out.println("Cannot write image to "+copy);
		}
	} catch (Exception e) {
		System.err.println("Cannot write image to file "+copy+": "+e.getMessage());
		e.printStackTrace(System.err);
	}
}

To access imwrite method, the following imports have to be added:

import static org.bytedeco.opencv.global.opencv_imgcodecs.imwrite;

Execute the program and ensure that the image have been copied to the desired location.


Exercise 4.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/ImageBasic.java and modify the main method in order to make the copy of the image to be gray. 


3.3. Image display

OpenCV library provide a simple way to display image. However, the JavaCV implementation does not wrap this functionnality as it relies on external libraries (such as Qt) that are not available in Java. The java-cv project contains a class dedicated to image display that relies on JavaFX. The class fr.utln.javacv.jfx.JavaCVJFXImageDisplay provide a static method display(String, Mat) that enable to display an image represented by a Mat within a JavaFX frame named according to  the given String.


Exercise 5.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/ImageBasic.java and modify the main method as follows:

public static void main(String[] args) {
	String file = "data/images/lake-mountain.png";
		
	// Load an image, using its color model
	Mat image = imread(file, IMREAD_UNCHANGED);
		
	// The image load has failed, program exit
	if (image.empty()) System.exit(1);
		
	JavaCVJFXImageDisplay.display(file, image);
}

In order to access to the JavaCVJFXImageDisplay class, the following import has to be added:

import fr.utln.javacv.jfx.JavaCVJFXImageDisplay;

Run the program and ensure that the image is displayed.


Using the JavaFX image display class enables to control images during processing. Any kind of image can be displayed.

4. Underlying representation: Mat

OpenCV represents images as a matrix that is implemented with the Mat class. A Mat is quite similar to a standard mathematical matrix.

4.1. General properties

A Mat provides general properties that describe its structure.

4.1.1. Size

The size of a Mat is composed of its width (the number of its columns) and height (the number of its rows). Getting a Mat m size can be obtained by calling method:

Size s = m.size()

From the Size object s that is returned, it is possible to get the number of columns of the Mat by calling method:

s.width()

In the same way, the number of rows of a Mat can be obtained from a Size s by caling method:

s.height()

There is also conveniences methods to obtain the number of rows and columns directely from a Mat m by calling methods:

m.rows()
m.cols()

Exercise 6.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following static method:

public static void ex6() {
			
	String file = "data/images/lake-mountain.png";
				
	// Load an image, using its color model
	Mat mat = imread(file, IMREAD_UNCHANGED);
}

Remember to add following imports in order to access Mat, imread and IMREAD_UNCHANGED:

import org.bytedeco.opencv.opencv_core.Mat;
import static org.bytedeco.opencv.global.opencv_imgcodecs.imread;
import static org.bytedeco.opencv.global.opencv_imgcodecs.IMREAD_UNCHANGED ;

Modify the main method in order to call ex6():

public static void main(String[] args) {
	ex6();
}

Complete the ex6() method in order to display the number of rows and columns of mat using the size() method and also the rows() and cols() methods.

Run the program and ensure that the two way of getting the number of rows / cols are giving the same results and also that the returned values correspond to the input image resolution.


4.1.2. Depth

The type of the values that are stored within the cells of a Mat is called depth. There is a finite set of depth that OpenCV can handles and that correspond to numerical type. All available depth are coded on an integer constant:

Mat depth (numerical type) Value size (bits) Sign
Native type JavaCV constant
8U 8 unsigned unsigned byte CV_8U
8S 8 signed byte CV_8S
16U 16 unsigned unsigned short int CV_16U
16S 16 signed short int CV_16S
32S 32 signed int CV_32S
32F 32 signed float  CV_32F
64F 64 signed double CV_64F

 

The depth of a Mat instance m can be obtained using the method:

int d = m.depth()

the value of d is one of the constants CV_8U, CV_8S, CV_16U, CV_16S, CV_32S, CV_32F, CV_64F.


Exercise 7.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the getDepthAsString() method as follows:

/**
  * Convert a {@link Mat#depth() Mat depth} into a {@link String}. 
  * If the given <code>depth</code> is not valid, "XX" is returned. 
  * @param depth the depth of a Mat (obtained using {@link Mat#depth()})
  * @return the String representation of a Mat depth
  */
public static String getDepthAsString(int depth) {
	switch(depth) {
	case CV_8U: return "8U";
	case CV_8S: return "8S";
	case CV_16S: return "16S";
	case CV_16U: return "16U";
	case CV_32F: return "32F";
	case CV_32S: return "32S";
	case CV_64F: return "64F";
	default: return "XX";
	}
}

To access to the CV_XX constants, the following imports are needed:

import static org.bytedeco.opencv.global.opencv_core.CV_8U;
import static org.bytedeco.opencv.global.opencv_core.CV_8S;
import static org.bytedeco.opencv.global.opencv_core.CV_16S;
import static org.bytedeco.opencv.global.opencv_core.CV_16U;
import static org.bytedeco.opencv.global.opencv_core.CV_32F;
import static org.bytedeco.opencv.global.opencv_core.CV_32S;
import static org.bytedeco.opencv.global.opencv_core.CV_64F;

Edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following method:

public static void ex7() {
		
	String file = "data/images/lake-mountain.png";
			
	// Load an image, using its color model
	Mat m = imread(file, IMREAD_UNCHANGED);
}

Modify now the ex7 method in order to display the size and the depth of m.

Modify the main method to run ex7.

Run the program and ensure that the Mat depth correspond to an unsigned byte (8U).


4.1.3. Channels

Values contained within cells of a Mat are defined by their depth (their numerical type). However, Mat can store multiple values within a single cell. The number and the ordering of these values is called channels. If a Mat has only one channel, then each of its cells contains only one value whose type is determined by its depth. If a Mat has several channels, each of its cells contains an ordered list of values whose number is equal to the number of channels.

The following example illustrate a 2x2 1-channelled Mat. Each cell contains only one value: 

[ 1 2 ]
[ 3 4 ]

The following example illustrate a 2x2 2-channelled Mat. Each cell contains a list of 2 values.

[ (1, 2) (3,4) ]
[ (5, 6) (7, 8)]

It is possible to know the number of channels that provide a Mat m with a call to the method:

m.channels()

Exercise 8.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add a getChannelAsString method as follows:

/**
  * Convert a {@link Mat#channels() Mat channels count} into a {@link String}. 
  * If the given <code>channels<code> count is not valid, "CY" is returned.
  * @param channels the number of channels for a Mat (obtained using {@link Mat#channels()})
  * @return the String representation of a Mat channels count
  */
public static String getChannelAsString(int channels) {
	if ((channels >= 1) && (channels <= 4)) {
			return "C"+channels;
	}
		
	return "CY";
}
	

add to the class the following method:

public static void ex8() {
		
	String file = "data/images/lake-mountain.png";
		
	// Load an image, using its color model
	Mat m = imread(file, IMREAD_UNCHANGED);
}

Modify now the ex8 method in order to display the size, the depth and the channels of m.

Modify the main method to run ex8.

Run the program and make sure it displays the expected information. Wich kind of relation can be established between image properties and the number of channels ?


4.1.4. Type

A Mat's type is the combination of its depth and channels. As OpenCV has a limited number of depths and a limited number of channels, the number of possible types is also limited. The types recognized by OpenCV are constructed as follows:

Depth
Channels
1 2 3 4
8U 8UC1 8UC2 8UC3 8UC4
8S 8SC1 8SC2 8SC3 8SC4
16U 16UC1 16UC2 16UC3 16UC4
16S 16SC1 16SC2 16SC3 16SC4
32S 32SC1 32SC2 32SC3 32SC4
32F 32FC1 32FC2 32FC3 32FC4
64F 64FC1 64FC1 64FC3 64FC4

 

Each type is represented as a constant integer defined within JavaCV by static constant within package org.bytedeco.opencv.global.opencv_core that is named CV_<depth><channels>. For example, the 8UC1 type is defined by:

org.bytedeco.opencv.global.opencv_core.CV_8UC1;

The type of a Mat m can be obtained by a call to the method:

m.type()

Exercise 9.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add a getTypeAsString method as follows:

/**
  * Convert a {@link Mat#type() Mat type} into a {@link String}. 
  * If the given <code>type<code> is not valid, "XXYCZ" is returned.
  * @param type the type of a Mat (obtained using {@link Mat#type()})
  * @return the String representation of a Mat channels type
  */
public static String getTypeAsString(int type) {
	if (type == CV_8UC1)  return "8UC1";
	if (type == CV_8UC2)  return "8UC2";
	if (type == CV_8UC3)  return "8UC3";
	if (type == CV_8UC4)  return "8UC4";
	if (type == CV_8SC1)  return "8SC1";
	if (type == CV_8SC2)  return "8SC2";
	if (type == CV_8SC3)  return "8SC3";
	if (type == CV_8SC4)  return "8SC4";
	if (type == CV_16UC1) return "16UC1";
	if (type == CV_16UC2) return "16UC2";
	if (type == CV_16UC3) return "16UC3";
	if (type == CV_16UC4) return "16UC4";
	if (type == CV_16SC1) return "16SC1";
	if (type == CV_16SC2) return "16SC2";
	if (type == CV_16SC3) return "16SC3";
	if (type == CV_16SC4) return "16SC4";
	if (type == CV_32SC1) return "32SC1";
	if (type == CV_32SC2) return "32SC2";
	if (type == CV_32SC3) return "32SC3";
	if (type == CV_32SC4) return "32SC4";
	if (type == CV_32FC1) return "32FC1";
	if (type == CV_32FC2) return "32FC2";
	if (type == CV_32FC3) return "32FC3";
	if (type == CV_32FC4) return "32FC4";
	if (type == CV_64FC1) return "64FC1";
	if (type == CV_64FC2) return "64FC2";
	if (type == CV_64FC3) return "64FC3";
	if (type == CV_64FC4) return "64FC4";
	else return "XXYCZ";
}

This method enable to convert a type into a String. Accessing to all the available types needs the following imports:

import static org.bytedeco.opencv.global.opencv_core.CV_8UC1;
import static org.bytedeco.opencv.global.opencv_core.CV_8UC2;
import static org.bytedeco.opencv.global.opencv_core.CV_8UC3;
import static org.bytedeco.opencv.global.opencv_core.CV_8UC4;
import static org.bytedeco.opencv.global.opencv_core.CV_8SC1;
import static org.bytedeco.opencv.global.opencv_core.CV_8SC2;
import static org.bytedeco.opencv.global.opencv_core.CV_8SC3;
import static org.bytedeco.opencv.global.opencv_core.CV_8SC4;
import static org.bytedeco.opencv.global.opencv_core.CV_16UC1;
import static org.bytedeco.opencv.global.opencv_core.CV_16UC2;
import static org.bytedeco.opencv.global.opencv_core.CV_16UC3;
import static org.bytedeco.opencv.global.opencv_core.CV_16UC4;
import static org.bytedeco.opencv.global.opencv_core.CV_16SC1;
import static org.bytedeco.opencv.global.opencv_core.CV_16SC2;
import static org.bytedeco.opencv.global.opencv_core.CV_16SC3;
import static org.bytedeco.opencv.global.opencv_core.CV_16SC4;
import static org.bytedeco.opencv.global.opencv_core.CV_32SC1;
import static org.bytedeco.opencv.global.opencv_core.CV_32SC2;
import static org.bytedeco.opencv.global.opencv_core.CV_32SC3;
import static org.bytedeco.opencv.global.opencv_core.CV_32SC4;
import static org.bytedeco.opencv.global.opencv_core.CV_32FC1;
import static org.bytedeco.opencv.global.opencv_core.CV_32FC2;
import static org.bytedeco.opencv.global.opencv_core.CV_32FC3;
import static org.bytedeco.opencv.global.opencv_core.CV_32FC4;
import static org.bytedeco.opencv.global.opencv_core.CV_64FC1;
import static org.bytedeco.opencv.global.opencv_core.CV_64FC2;
import static org.bytedeco.opencv.global.opencv_core.CV_64FC3;
import static org.bytedeco.opencv.global.opencv_core.CV_64FC4;

add to the class the following method:

public static void ex9() {
		
	String file = "data/images/lake-mountain.png";
		
	// Load an image, using its color model
	Mat m = imread(file, IMREAD_UNCHANGED);
	
}

Modify the ex9 method in order to display the size, the depth, the channels and the type of m.

Modify the main method to run ex9.

Run the program and make sure it displays the expected information as the type must be the concatenation of the String representations of the depth and the channels.


4.2. Instanciation

Mat objects can be obtained using imread function but can also be instanciated from scratch.

4.2.1. Construction

The Mat class provide a constructor that ca be call using:

new Mat(rows, cols, type)

where rows is the desired number of rows, cols is the desired number of columns and type is the desired type. For example the following instruction:

Mat m = new Mat(4, 3, CV_32FC3);

creates a Mat m with 4 rows, 3 columns and a type 32FC3 (a depth of 32F and 3 channels).

As a Mat is a wrap to a native object, it is not guaranted that the values whithin the created Mat are initialized. 

4.2.2. Initialization

OpenCV / JavaCV provides some static helper in order to create Mat with initialized values.

Zeros filled Mat

a Mat can be created and initialized with all its values equal to 0 using:

Mat m = Mat.zeros(r, c, t).asMat();

This instruction creates a Mat m with r rows, c columns and a type t with all the values of m equal to 0. For example, the instruction:

Mat m = Mat.zeros(4, 3, CV_64FC2).asMat();

creates a Mat m with 4 rows, 3 columns, a 64FC2 type and with all its values initialized to 0.

One filled Mat

a Mat can be created and initialized with all its values equal to 1 using:

Mat m = Mat.ones(r, c, t).asMat();

This instruction creates a Mat m with r rows, c columns and a type t with all the values of m equal to 0. For example, the instruction:

Mat m = Mat.ones(5, 7, CV_8UC1).asMat();

creates a Mat m with 5 rows, 7 columns, a 8UC1 type and with all its values initialized to 1.

Diagonal ones Mat

A Mat can be created and initialized with all its values equal to 0 except on its main diagonal, where its first channel values are then 1. This can be done using:

Mat m = Mat.eye(r, c, t).asMat();

This instruction creates a Mat m with r rows, c columns, a type t and with all the values of m equal to 0 except for its main diagonal where the first channel values equal 1. For example, the instruction:

Mat m = Mat.eye(4, 3, CV_32FC3).asMat();

creates a Mat m with 4 rows, 3 columns, a 32FC3 type that is such as:

[ ( 1.0 0.0 0.0 ) ( 0.0 0.0 0.0 ) ( 0.0 0.0 0.0 ) ]
[ ( 0.0 0.0 0.0 ) ( 1.0 0.0 0.0 ) ( 0.0 0.0 0.0 ) ]
[ ( 0.0 0.0 0.0 ) ( 0.0 0.0 0.0 ) ( 1.0 0.0 0.0 ) ]
[ ( 0.0 0.0 0.0 ) ( 0.0 0.0 0.0 ) ( 0.0 0.0 0.0 ) ]

If the desired Mat is square and if it count 1 channel, the eye function creates an identity matrix. For example:

Mat m = Mat.eye(3, 3, CV_32FC1).asMat();

creates the Mat:

[ 1.0 0.0 0.0 ]
[ 0.0 1.0 0.0 ]
[ 0.0 0.0 1.0 ]

4.3. Data access

JavaCV Mat object is a wrapper to underelying native OpenCV Mat. Accessing data that are stored within a Mat needs to access native memory.

To make native data access transparent, JavaCV offers a system of Indexer, which can both read and write native data from Java.

An instance of the Mat class can provide an indexer to access its data using the method:

createIndexer()

The previous method returns an implementation of th Indexer interface that depends on the depth (numerical type) of the Mat. Following table illustrate the possible type of indexer:

Mat depth (numerical type) Native type Indexer class Java type
8U unsigned byte UByteIndexer int
8S byte ByteIndexer byte
16U unsigned short int UShortIndexer int
16S short int ShortIndexer short
32S int IntIndexer int
32F float FloatIndexer float
64F double DoubleIndexer double

 

4.3.1. Data reading

Reading data from a Mat is performed by calling Indexer get methods according to the number of channels of the Mat:

Single channel Mat

For a single channel Mat (C1), the method get(r, c) return the value of the cell at row r and column c.


Exercise 10.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following method:

public static void ex10() {
	Mat m = MatUtils.rowMajorMat(4, 3, CV_8UC1);

	// Create an indexer for accessing Mat data
	UByteIndexer indexer = m.createIndexer();
        
	for(int row = 0; row < m.rows(); row++) {
		System.out.print("[");
		for(int col = 0; col < m.cols(); col++) {	
				
			// Display the value of the Mat cell at (row, col)
			System.out.print(" "+indexer.get(row, col));
		}

		System.out.println(" ]");
	}
}
	

The access to UByteIndexer class needs the import:

import org.bytedeco.javacpp.indexer.UByteIndexer;

The ex10 method creates a 4x3 Mat of unigned bytes with 1 channel populated with successive integers. The line:

Mat m = MatUtils.rowMajorMat(4, 3, CV_8UC1);

is a convenience for creating an initialized Mat with specific values. The line:

UByteIndexer indexer = m.createIndexer();

Creates an Indexer for the Mat. As the Mat depth is unsigned byte, the created indexer is an instance of UByteIndexer (see Indexer table). The line:

System.out.print(" "+indexer.get(row, col));

displays the value that the indexer can get from the cell located at (row, col) within the Mat.

Modify the main method to run ex10.

Run the program and ensure that the Mat is successfully displayed. Expected result is:

[ 1 2 3 ]
[ 4 5 6 ]
[ 7 8 9 ]
[ 10 11 12 ]

Multiple channels Mat

For a multiple channels Mat (C2, C3, C4), two get methods are available:

    • get(r, c, ch) return the value of the cell at row r, column c for the channel ch.
    • get(r, c, T[]) fills the array T with the values contained within cell at row r and column c. The type of T depends on the class of the indexer (see Indexer table). The size of T is the same as the number of channels.

Exercice 11.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following method:

public static void ex11() {
		
	// Create a new Mat with 5 rows, 7 cols and 3 channel made of float 
	Mat m = MatUtils.rowMajorMat(5, 7, CV_32FC3);
			
	System.out.println("Mat size: "+m.rows()+"x"+m.cols()+", type: "+getTypeAsString(m.type()));
			
	// Access to the native image raster (the Mat)
	FloatIndexer indexer = m.createIndexer();
		
	for(int row = 0; row < m.rows(); row++) {
		System.out.print("[");
		for(int col = 0; col < m.cols(); col++) {
				
			System.out.print(" ( ");
			for(int channel = 0; channel < m.channels(); channel++) {
				System.out.print(indexer.get(row, col, channel)+" ");
			}
			System.out.print(")");
		}
		System.out.println(" ]");
	}
}

The FloatIndexer class is made available by adding the import:

import org.bytedeco.javacpp.indexer.FloatIndexer;

The line:

FloatIndexer indexer = m.createIndexer();

Creates an Indexer for the Mat. As the Mat depth is float, the created indexer is an instance of FloatIndexer (see Indexer table). The line:

System.out.print(indexer.get(row, col, channel)+" ");

displays the value of the cell located at (row, col) for the desired channel.

Modify the main method to run ex11.

Run the program and ensure that the Mat is successfully displayed. Expected result is:

Mat size: 5x7, type: 32FC3
[ ( 1.1 1.2 1.3 ) ( 2.1 2.2 2.3 ) ( 3.1 3.2 3.3 ) ( 4.1 4.2 4.3 ) ( 5.1 5.2 5.3 ) ( 6.1 6.2 6.3 ) ( 7.1 7.2 7.3 ) ]
[ ( 8.1 8.2 8.3 ) ( 9.1 9.2 9.3 ) ( 10.1 10.2 10.3 ) ( 11.1 11.2 11.3 ) ( 12.1 12.2 12.3 ) ( 13.1 13.2 13.3 ) ( 14.1 14.2 14.3 ) ]
[ ( 15.1 15.2 15.3 ) ( 16.1 16.2 16.3 ) ( 17.1 17.2 17.3 ) ( 18.1 18.2 18.3 ) ( 19.1 19.2 19.3 ) ( 20.1 20.2 20.3 ) ( 21.1 21.2 21.3 ) ]
[ ( 22.1 22.2 22.3 ) ( 23.1 23.2 23.3 ) ( 24.1 24.2 24.3 ) ( 25.1 25.2 25.3 ) ( 26.1 26.2 26.3 ) ( 27.1 27.2 27.3 ) ( 28.1 28.2 28.3 ) ]
[ ( 29.1 29.2 29.3 ) ( 30.1 30.2 30.3 ) ( 31.1 31.2 31.3 ) ( 32.1 32.2 32.3 ) ( 33.1 33.2 33.3 ) ( 34.1 34.2 34.3 ) ( 35.1 35.2 35.3 ) ]

Exercise 12.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following method:

public static void ex12() {
		
	// Create a new Mat with 6 rows, 6 cols and 3 channel made of float 
	Mat m = MatUtils.rowMajorMat(6, 6, CV_64FC2);
			
	System.out.println("Mat size: "+m.rows()+"x"+m.cols()+", type: "+getTypeAsString(m.type()));
			
	// Access to the native image raster (the Mat)
	DoubleIndexer indexer = m.createIndexer();
			
	// Prepare the channel buffer
	double[] values = new double[m.channels()];
		
	for(int row = 0; row < m.rows(); row++) {
		System.out.print("[");
		for(int col = 0; col < m.cols(); col++) {
				
			// Get the channel values as an array
			indexer.get(row, col, values);
				
			System.out.print(" ( ");
			for(int channel = 0; channel < m.channels(); channel++) {
				System.out.print(values[channel]+" ");
			}
			System.out.print(")");
		}
		System.out.println(" ]");
	}
}

The use of DoubleIndexer requires the import:

import org.bytedeco.javacpp.indexer.DoubleIndexer;

The line:

DoubleIndexer indexer = m.createIndexer();

Creates an Indexer for the Mat. As the Mat depth is double, the created indexer is an instance of DoubleIndexer (see Indexer table). The line:

double[] values = new double[m.channels()];

prepare the values buffer. The array has a size that is equals to the number of channels. The line:

indexer.get(row, col, values);

fills the values with all the ones available. values[0] correspond to the 1st channel value and values[1] to the 2nd channel value.

Modify the main method to run ex12.

Run the program and ensure that the Mat is successfully displayed. Expected result is:

Mat size: 6x6, type: 64FC2
[ ( 1.1 1.2 ) ( 2.1 2.2 ) ( 3.1 3.2 ) ( 4.1 4.2 ) ( 5.1 5.2 ) ( 6.1 6.2 ) ]
[ ( 7.1 7.2 ) ( 8.1 8.2 ) ( 9.1 9.2 ) ( 10.1 10.2 ) ( 11.1 11.2 ) ( 12.1 12.2 ) ]
[ ( 13.1 13.2 ) ( 14.1 14.2 ) ( 15.1 15.2 ) ( 16.1 16.2 ) ( 17.1 17.2 ) ( 18.1 18.2 ) ]
[ ( 19.1 19.2 ) ( 20.1 20.2 ) ( 21.1 21.2 ) ( 22.1 22.2 ) ( 23.1 23.2 ) ( 24.1 24.2 ) ]
[ ( 25.1 25.2 ) ( 26.1 26.2 ) ( 27.1 27.2 ) ( 28.1 28.2 ) ( 29.1 29.2 ) ( 30.1 30.2 ) ]
[ ( 31.1 31.2 ) ( 32.1 32.2 ) ( 33.1 33.2 ) ( 34.1 34.2 ) ( 35.1 35.2 ) ( 36.1 36.2 ) ]

4.3.2. Data writting

Writing data from a Mat is performed by calling Indexer put methods according to the number of channels of the Mat:

Single channel Mat

For a single channel Mat (C1), the method put(r, c, v) set the value of the cell (r, c) to v.


Exercise 13.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following method:

public static void ex13() {
		
	// Create a 4x3 mat initialized to 0
	Mat m = Mat.zeros(4, 3, CV_8UC1).asMat();

	// Create an indexer for accessing Mat data
	UByteIndexer indexer = m.createIndexer();
	      
	// Set the values of the Mat
	int i = 1;
	for(int row = 0; row < m.rows(); row++) {
		for(int col = 0; col < m.cols(); col++) {	
			indexer.put(row, col, i);
			i++;
		}
	}
		
	// Display the modified Mat
	for(int row = 0; row < m.rows(); row++) {
		System.out.print("[");
		for(int col = 0; col < m.cols(); col++) {	
					
			// Display the value of the Mat cell at (row, col)
			System.out.print(" "+indexer.get(row, col));
		}

		System.out.println(" ]");
	}
}

The instruction:

Mat m = Mat.zeros(4, 3, CV_8UC1).asMat();

create a 4x3 Mat that is filled with 0. The line:

indexer.put(row, col, i);

set the value of the cell (row, col) to the value i.

Modify the main method to run ex13.

Run the program and ensure that the Mat is successfully displayed. Expected result is:

[ 1 2 3 ]
[ 4 5 6 ]
[ 7 8 9 ]
[ 10 11 12 ]

Multiple channels Mat

For a multiple channels Mat (C2, C3, C4), two put methods are available:

    • put(r, c, ch, v) set the value of the cell at row r, column c for the channel ch to v.
    • put(r, c, T[]) set the values of the cell at row r, column c to the ones contained within the array T. The type of T depends on the class of the indexer (see Indexer table). The size of T is the same as the number of channels.

Exercise 14.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/basic/MatBasic.java and add the following method:

public static void ex14() {
		
	// Create a 5x7 mat initialized to 0
	Mat m = Mat.zeros(5, 7, CV_32FC3).asMat();

	// Create an indexer for accessing Mat data
	FloatIndexer indexer = m.createIndexer();
	      
	// Set the values of the Mat
	int i = 1;
	for(int row = 0; row < m.rows(); row++) {
		for(int col = 0; col < m.cols(); col++) {	
			for(int channel = 0; channel < m.channels(); channel++)
			indexer.put(row, col, channel, i);
			i++;
		}
	}
		
	// Display the modified Mat
	for(int row = 0; row < m.rows(); row++) {
		System.out.print("[");
		for(int col = 0; col < m.cols(); col++) {
				
			System.out.print(" ( ");
			for(int channel = 0; channel < m.channels(); channel++) {
				System.out.print(indexer.get(row, col, channel)+" ");
			}
			System.out.print(")");
		}
		System.out.println(" ]");
	}
}

The instruction:

indexer.put(row, col, channel, i);

set the value of the cell (row, col) for the channel to the value i.

Modify the main method to run ex14.

Run the program and ensure that the Mat is successfully displayed. Expected result is:

[ ( 8.0 8.0 8.0 ) ( 9.0 9.0 9.0 ) ( 10.0 10.0 10.0 ) ( 11.0 11.0 11.0 ) ( 12.0 12.0 12.0 ) ( 13.0 13.0 13.0 ) ( 14.0 14.0 14.0 ) ]
[ ( 15.0 15.0 15.0 ) ( 16.0 16.0 16.0 ) ( 17.0 17.0 17.0 ) ( 18.0 18.0 18.0 ) ( 19.0 19.0 19.0 ) ( 20.0 20.0 20.0 ) ( 21.0 21.0 21.0 ) ]
[ ( 22.0 22.0 22.0 ) ( 23.0 23.0 23.0 ) ( 24.0 24.0 24.0 ) ( 25.0 25.0 25.0 ) ( 26.0 26.0 26.0 ) ( 27.0 27.0 27.0 ) ( 28.0 28.0 28.0 ) ]
[ ( 29.0 29.0 29.0 ) ( 30.0 30.0 30.0 ) ( 31.0 31.0 31.0 ) ( 32.0 32.0 32.0 ) ( 33.0 33.0 33.0 ) ( 34.0 34.0 34.0 ) ( 35.0 35.0 35.0 ) ]

5. Mat manipulation

OpenCV provides functionnalities for Mat manipulation and mathematical operations. This section illustrates most common Mat maniupulation. For the rest of this section, let $M$ be a Mat of size $r\times{}c$ such as:

\[
M\ =\ 
\begin{bmatrix}
m_{0, 0}  & \cdots & m_{0,j}   & \cdots & m_{0, c}  \\
\vdots    & \ddots & \vdots    & \ddots & \vdots   \\
m_{i,0}   & \cdots & m_{i,j}   & \cdots & m_{i,c}      \\
\vdots    & \ddots & \vdots    & \ddots & \vdots   \\
m_{r,0} & \cdots & m_{r,j} & \cdots & m_{r,c}
\end{bmatrix}
\]

5.1. Addition & subtraction

 

 

5.2. Multiplication / division

scalar Multiplication

\[
s\ \times\ M\ =\ 
\begin{bmatrix}
sm_{0, 0}  & \cdots & sm_{0,j}   & \cdots & sm_{0, n-1} \\
\vdots     & \ddots & \vdots     & \ddots & \vdots     \\
sm_{i,0}   & \cdots & sm_{i,j}   & \cdots & sm_{i,n-1}  \\
\vdots     & \ddots & \vdots     & \ddots & \vdots     \\
sm_{n-1,0} & \cdots & sm_{n-1,j} & \cdots & sm_{n-1,n-1}
\end{bmatrix}
\]

scalar division

\[
\frac{M}{s} =
\begin{bmatrix}
\frac{m_{0, 0}}{s}  & \cdots & \frac{m_{0, j}}{s}   & \cdots & \frac{m_{0, n-1}}{s}  \\
\vdots              & \ddots & \vdots               & \ddots & \vdots                \\
\frac{m_{i,0}}{s}   & \cdots & \frac{m_{i, j}}{s}   & \cdots & \frac{m_{i, n-1}}{s}  \\
\vdots              & \ddots & \vdots               & \ddots & \vdots                \\
\frac{m_{n-1,0}}{s} & \cdots & \frac{m_{n-1, j}}{s} & \cdots & \frac{m_{n-1, n-1}}{s}
\end{bmatrix}
\]

todo

5.3. Concatenation

todo

6. Images manipulation using Mat

It is now possible to manipulate images.


Exercise 15.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/intermediate/ImageModifying.java and add a method:

/**
  * Extract a sub-Mat located within a rectangle from the given <code>source</code>
  * @param source the source MAt
  * @param x the X coordinate of the rectangle upper left corner
  * @param y the y coordinate of the rectangle upper left corner
  * @param width the rectangle width
  * @param height the rectangle height
  * @return the extracted sut Mat
  */
public static Mat extract(Mat source, int x, int y, int width, int height)

Implement this method to return the part of the input mat within the rectangle defined by the upper left corner (x, y) and a size (width, height).

Add the main method as follows:

public static void main(String[] args) {
	String file = "data/images/lake-mountain.png";
		
	// Load an image, using its color model
	Mat image = imread(file, IMREAD_UNCHANGED);
		
	Mat m = extract(image, 200, 250, 100, 100);
			
	JavaCVJFXImageDisplay.display("Extract", m);
		
}

Run the program and ensure that the extraction is successfull.


Exercise 16.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/intermediate/ImageModifying.java and add the method:

/**
  * Split the <code>source</code> Mat into an array <code>m</code> of Mat where each <code>m[i]</code> contains values from source channel <code>i</code>.  
  * @param source the source Mat
  * @return an array of Mat
  */
public static Mat[] splitChannels(Mat source)

Implement this method in order to return an array of Mat where each Mat[i] contains values from the given source for the channel i.

Modify the main method in order as follows:

public static void main(String[] args) {
	String file = "data/images/lake-mountain.png";
		
	// Load an image, using its color model
	Mat image = imread(file, IMREAD_UNCHANGED);
		
	Mat[] splitted = splitChannels(image);
			
	if (splitted != null){
		for(int i = 0; i < splitted.length; i++){
			imwrite("splitted_"+i+".png", splitted[i]);
		}
	}
		
}

Run the program and check the results.


Exercise 17.

Within the javacv-basic project, edit the file src/main/java/fr/utln/javacv/intermediate/ImageModifying.java and add the method:

/**
 * Return a new {@link Mat} that is the result of a right rotation 
 * of the given <code>source</code> {@link Mat} by 90°.
 * @param source the source Mat
 * @return the rotated Mat
 */
public static Mat rot90(Mat source)

Implements the method in order to return the result of the rotation of the source by 90°.

Modify the main method in order as follows:

public static void main(String[] args) {
	String file = "data/images/lake-mountain.png";
		
	// Load an image, using its color model
	Mat image = imread(file, IMREAD_UNCHANGED);
		
	Mat m = rot90(image);
				
	imwrite("Rot90.png", m);
		
}

Run the program and check the results.