Image Recognition Algorithm in C++
Image recognition is a technology in which we identify the places, logos, people, objects, buildings, and other variables in digital images. For humans like you, it can be easy to recognize the image of a cat and differentiate it from an image of a horse, but it is difficult for a computer to recognize. A digital image contains elements of a picture that are also referred to as pixels, each with finite, discrete quantities of numeric representation for its intensity or grey level. So, a computer sees an image like a number of values of these pixels, and to identify a certain image, it has to find the patterns and the regularities in the data.
Image recognition is helpful in object detection, in which we analyze an image and find various objects in the image, while image recognition deals with finding the images and classifying them into various categories.
How does Image Recognition Work?
The task of identifying an image involves the creation of a neural network that processes the individual pixels of an image. These neural networks are given as input of pre-labelled images in order to train them how to identify similar images. These types of networks are called as convolutional neural networks.
Working of Convolutional Neural Network:
Convolutional Neural Networks are also known as convolutional networks. These are the specialized kinds of neural networks that are used for processing data that has known grid-like topology. Convolutional networks can be used in time-series data, which can be seen as a 1-D grid taking samples at regular intervals of time, on image data, which can be thought of as a 2-D grid of pixels. Instead of using a matrix for calculation, convolution neural networks use convolution operations to perform calculations. Convolutions are a special kind of linear operation. The word Convolutional in Convolutional Neural Network indicates that networks use a mathematical operation called convolution. Convolution between two functions in mathematics produces a third function expressing how the shape of one function is modified by another. When the convolution is applied to signals, it is called convolution 1-D; to images, it is called as convolution 2-D; and to videos, convolution 3-D.
Convolution Kernel:
A kernel is a small 2D matrix whose contents are based on the operations to be performed. A kernel maps on the input image by simple matrix multiplication and addition; the output obtained is of the lower dimension and, therefore, easier to work with. The shape of the kernel is mainly dependent on the shape of the input and the architecture of the whole network. Mostly, the size of the kernel is (M x M).
The convolutional kernel is used to extract the features from the image. It reduces the size of the input by multiplying it with the kernel by sliding across the image to give the output. The above image shows two matrices taken from the input, which is multiplied by the kernel matrix, which gives the result of 19. It is calculated as 0 * 0 + 1 * 1 + 3 * 2 + 4 * 3 = 19, and then we slide the kernel by 1, and the second two by two matrix is chosen with the element [ [1, 2], [ 4, 5]]. It is multiplied by the kernel matrix as 0 * 1 + 2 * 1 + 4 * 2 + 5 * 3 = 25 and similarly all the output as calculated.
Stride:
Stride tells what step it takes to move the kernel. Filters are one dimension higher than kernels and can be seen as multiple kernels stacked on each other, where each kernel is for a particular channel. The pixels of the image are moved over the input matrix. If the value of stride is 1, then the kernel is moved 1 pixel at a time. If the value of stride is 2, then the kernel will shift 2 pixels and so on. Strids are important because the convolution operation is controlled. Strides regulate the features that could be missed while the image is being flattened.
Padding:
The padding is an important part of building the convolutional neural network. After performing the convolutional operation, the initial size of the image is reduced. So, if there are multiple convolution operations are performed on the image, then the size of the image will be diminished, which we don't want.
The second problem is that the kernel passes more time through the middle layer than the edge layer in the original image, which causes the problem of overlap.
To solve this problem, the concept of padding came into the picture. The padding is the layer that is added to the borders of an image to keep the size of the original picture the same.
Pooling:
Repeatedly, it reduces the spatial size of the representation to reduce the number of parameters and reduce the computations in the network to control the problem of overfitting. Spatial Pooling is also referred to as down-sampling, which is used to reduce the dimensionality of each map. An activation function called ReLU is applied to each value in the feature map.
There are two types of Pooling:
- Max Pooling: Max Pooling take the highest values form the feature. Max Pooling is a sample-based process that transfers continuous functions into discrete counterparts. Max Pooling reduces the dimensionality and makes assumptions about the features that are contained in the sub-region.
- Average Pooling: Average Pooling finds the average of the elements in the feature map that is covered by the filter. It gives the average of features in a patch.
Flatten Layer:
After performing the pooling operation, the next layer in CNN is flattening; its function is to convert the pooling layer into a 1-D matrix by putting the values together in a column one by one.
Implementation of Image Recognition algorithm in C++:
Code:
// CPP program to detect face in a video
// Include required header files from the OpenCV directory
#include "/usr/local/include/opencv2/objdetect.hpp"
#include "/usr/local/include/opencv2/highgui.hpp"
#include "/usr/local/include/opencv2/imgproc.hpp"
#include <iostream>
using namespace std;
using namespace cv;
// Function for Face Detection
void detectAndDraw( Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade, double scale );
string cascadeName, nestedCascadeName;
int main( int argc, const char** argv )
{
// VideoCapture class for playing video for which faces to be detected
VideoCapture capture;
Mat frame, image;
// PreDefined trained XML classifiers with facial features
CascadeClassifier cascade, nestedCascade;
double scale=1;
// Load classifiers from "opencv/data/haarcascades" directory
nestedCascade.load( "../../haarcascade_eye_tree_eyeglasses.xml" ) ;
// Change path before execution
cascade.load( "../../haarcascade_frontalcatface.xml" ) ;
// Start Video..1) 0 for WebCam 2) "Path to Video" for a Local Video
capture.open(0);
if( capture.isOpened() )
{
// Capture frames from video and detect faces
cout << "Face Detection Started...." << endl;
while(1)
{
capture >> frame;
if( frame.empty() )
break;
Mat frame1 = frame.clone();
detectAndDraw( frame1, cascade, nestedCascade, scale );
char c = (char)waitKey(10);
// Press q to exit from window
if( c == 27 || c == 'q' || c == 'Q' )
break;
}
}
else
cout<<"Could not Open Camera";
return 0;
}
void detectAndDraw( Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade,
double scale)
{
vector<Rect> faces, faces2;
Mat gray, smallImg;
cvtColor( img, gray, COLOR_BGR2GRAY ); // Convert to Gray Scale
double fx = 1 / scale;
// Resize the Grayscale Image
resize( gray, smallImg, Size(), fx, fx, INTER_LINEAR );
equalizeHist( smallImg, smallImg );
// Detect faces of different sizes using cascade classifier
cascade.detectMultiScale( smallImg, faces, 1.1,
2, 0|CASCADE_SCALE_IMAGE, Size(30, 30) );
// Draw circles around the faces
for ( size_t i = 0; i < faces.size(); i++ )
{
Rect r = faces[i];
Mat smallImgROI;
vector<Rect> nestedObjects;
Point center;
Scalar color = Scalar(255, 0, 0); // Color for Drawing tool
int radius;
double aspect_ratio = (double)r.width/r.height;
if( 0.75 < aspect_ratio && aspect_ratio < 1.3 )
{
center.x = cvRound((r.x + r.width*0.5)*scale);
center.y = cvRound((r.y + r.height*0.5)*scale);
radius = cvRound((r.width + r.height)*0.25*scale);
circle( img, center, radius, color, 3, 8, 0 );
}
else
rectangle( img, cvPoint(cvRound(r.x*scale), cvRound(r.y*scale)),
cvPoint(cvRound((r.x + r.width-1)*scale),
cvRound((r.y + r.height-1)*scale)), color, 3, 8, 0);
if( nestedCascade.empty() )
continue;
smallImgROI = smallImg( r );
// Detection of eyes in the input image
nestedCascade.detectMultiScale( smallImgROI, nestedObjects, 1.1, 2,
0|CASCADE_SCALE_IMAGE, Size(30, 30) );
// Draw circles around eyes
for ( size_t j = 0; j < nestedObjects.size(); j++ )
{
Rect nr = nestedObjects[j];
center.x = cvRound((r.x + nr.x + nr.width*0.5)*scale);
center.y = cvRound((r.y + nr.y + nr.height*0.5)*scale);
radius = cvRound((nr.width + nr.height)*0.25*scale);
circle( img, center, radius, color, 3, 8, 0 );
}
}
// Show Processed Image with detected faces
imshow( "Face Detection", img );
}
Output:
When you run the above program with an input image then you will see the output the objects are marked with circle, any object in the image is recognized.
Explanation:
The above code recognizes images with the help of a webcam. The Haar cascades are used to recognize the images, which is a machine learning object detection image, to identify the images. First of all, the required libraries are imported, and the name declaration is done. A function is made detectAndDraw, which detects the images and marks the object. We declare some variables globally. In the main function frame, images are declared with a scale of 1. When loading the Haar cascade file, the camera is opened. A loop is run, and we detect the object frame by frame and mark the frame.