Project 3. Painting in the air!

If you've followed all my tutorials till now, this project will seem very easy.
The only thing I've done is combine and interlink different concepts.
Object Detection, Circle Detection, Creating an Image Matrix, Video Rendering and Drawing Circles are included in this project.
With this, you've got quite a good exposure to Computer Vision and Image Processing.
You can play around with it and create your own applications.
Keep posting your doubts!

#include<iostream>
#include<highgui.h>
#include<cv.h>
using namespace std;
using namespace cv;

int main()
{
 VideoCapture cap(0);
 Mat frame;
 Mat img;

 namedWindow("video");
 namedWindow("shapes");
 namedWindow("paintbox");
 vector<Vec3f> circles;

 double dWidth = cap.get(CV_CAP_PROP_FRAME_WIDTH);
 double dHeight = cap.get(CV_CAP_PROP_FRAME_HEIGHT);
 Mat paintbox=Mat(dHeight,dWidth,CV_8UC3,Scalar(255,255,255));
imshow("paintbox",paintbox);
 Mat pb_flipped;
 while(char(waitKey(1))!='q'){

  cap>>frame;

  cvtColor(frame,img,CV_RGB2GRAY);
  imshow("shapes",img);

  HoughCircles(img,circles,CV_HOUGH_GRADIENT,1,10,100,95,5);

  for(int i=0;i<circles.size();i++){

   Point center(cvRound(circles[i][0]),cvRound(circles[i][1]));
   int radius=cvRound(circles[i][2]);


   circle(frame,center,3,Scalar(0,0,255),-1);
   circle(frame,center,radius,Scalar(0,0,255),3,8,0);

 //  Mat img_local=paintbox.clone();
   circle(paintbox,center,3,Scalar(0,0,255),-1);
   flip(paintbox,pb_flipped,1);
   imshow("paintbox",pb_flipped);
  }
  imshow("video",frame);
 }
 return 0;
}

#12a. Hough Transform:Detecting lines

#include<iostream>
#include<cv.h>
#include<highgui.h>
using namespace cv;
using namespace std;
Mat img;
int thresh=100;

void on_trackbar(int, void *){
 Mat edges;
 Canny(img,edges,50,100);
 vector<Vec2f> lines;
 HoughLines(edges,lines,1,CV_PI/180.F,thresh);

 Mat img_show=img.clone();
 for(int i=0;i<lines.size();i++){
  float rho=lines[i][0];
  float theta=lines[i][1];
  double a=cos(theta), b=sin(theta);
  double x0=a*rho, y0=b*rho;
  Point pt1(cvRound(x0+1000*(-b)),cvRound(y0+1000*(a)));
  Point pt2(cvRound(x0-1000*(-b)),cvRound(y0-1000*(a)));
  line(img_show,pt1,pt2,Scalar(0,0,255));
 }
 imshow("shapes",img_show);
}
int main(){
 img=imread("images/linescircles.jpg");
 namedWindow("shapes");
 createTrackbar("Acc. thresh","shapes",&thresh,300,on_trackbar);
 on_trackbar(0,0);
 while(char(waitKey(0))!='q'){}

 return 0;
}

If you've gone through the previous posts(#11x.), this would'nt need much explanation.

HoughLines(edges,lines,1,CV_PI/180.F,thresh);

Format & Explanation:

 HoughLines(InputArray image, OutputArray lines, double rho, double theta, int threshold, double srn=0, double stn=0 )¶

Parameters:

image – 8-bit, single-channel binary source image. The image may be modified by the function. --> In our case: 'edges' lines – Output vector of lines. Each line is represented by a two-element vector  .  is the distance from the coordinate origin (top-left corner of the image).  is the line rotation angle in radians (  ). -->The above point is the gist of the program rho – Distance resolution of the accumulator in pixels. theta – Angle resolution of the accumulator in radians. threshold – Accumulator threshold parameter. Only those lines are returned that get enough votes (  ). The following may not be relevant to this tutorial. srn – For the multi-scale Hough transform, it is a divisor for the distance resolution rho . The coarse accumulator distance resolution isrho and the accurate accumulator resolution is rho/srn . If both srn=0 and stn=0 , the classical Hough transform is used. Otherwise, both these parameters should be positive. stn – For the multi-scale Hough transform, it is a divisor for the distance resolution theta. method – One of the following Hough transform variants: CV_HOUGH_STANDARD classical or standard Hough transform. Every line is represented by two floating-point numbers  , where is a distance between (0,0) point and the line, and  is the angle between x-axis and the normal to the line. Thus, the matrix must be (the created sequence will be) of CV_32FC2 type CV_HOUGH_PROBABILISTIC probabilistic Hough transform (more efficient in case if the picture contains a few long linear segments). It returns line segments rather than the whole line. Each segment is represented by starting and ending points, and the matrix must be (the created sequence will be) of the CV_32SC4 type. CV_HOUGH_MULTI_SCALE multi-scale variant of the classical Hough transform. The lines are encoded the same way asCV_HOUGH_STANDARD. param1 – First method-dependent parameter: For the classical Hough transform, it is not used (0). For the probabilistic Hough transform, it is the minimum line length. For the multi-scale Hough transform, it is srn. param2 – Second method-dependent parameter: For the classical Hough transform, it is not used (0). For the probabilistic Hough transform, it is the maximum gap between line segments lying on the same line to treat them as a single line segment (that is, to join them). For the multi-scale Hough transform, it is stn.

You may go through the Documentation HERE.

circle() function

Draws a circle.

C++: void circle(Mat& img, Point center, int radius, const Scalar& color, int thickness=1, int lineType=8, int shift=0)

Parameters:

img – Image where the circle is drawn. center – Center of the circle. radius – Radius of the circle. color – Circle color. thickness – Thickness of the circle outline, if positive. Negative thickness means that a filled circle is to be drawn. lineType – Type of the circle boundary. See the line() description. shift – Number of fractional bits in the coordinates of the center and in the radius value.

The function circle draws a simple or filled circle with a given center and radius.

 circle(img_show,center,3,Scalar(0,0,255),-1);

radius: 3

thickness: -1, filled circle




 

 circle(img_show,center,radius,Scalar(0,0,255),3,8,0);

radius: as detected

thickness: 3

line type: 8 

• 8 (or omitted) - 8-connected line.
• 4 - 4-connected line.

    shift: 0

#11d. Application: Detecting number of circles in an image

This is actually a redundant post.

There was a problem which I came through in one of the OpenCV Hackathons.

It asked to detect number of circles in the given image.

This is very easily done by Hough Transform.

#include<iostream>
#include<highgui.h>
#include<cv.h>
using namespace std;
using namespace cv;
Mat img;

int thresh=100; 

void on_trackbar(int, void *){
    Mat img_gray;
    cvtColor(img,img_gray,CV_RGB2GRAY);
    vector<Vec3f> circles;
    HoughCircles(img_gray,circles,CV_HOUGH_GRADIENT,1,10,100,thresh,5);
    Mat img_show=img.clone();

    //the following line has to be added to count the number of circles 
    cout<<"Number of circles: "<<circles.size()<<endl;

    for(int i=0;i<circles.size();i++){
        Point center(cvRound(circles[i][0]),cvRound(circles[i][1]));
        int radius=cvRound(circles[i][2]);

        circle(img_show,center,3,Scalar(0,0,255),-1);

        circle(img_show,center,radius,Scalar(0,0,255),3,8,0);

    }
    imshow("Shapes",img_show);

}
int main()
{
    img =imread("images/linescircles.jpg");
    namedWindow("Shapes");
    imshow("Shapes",img);
    createTrackbar("Acc. threshold","Shapes",&thresh,300,on_trackbar);
    on_trackbar(0,0);
    while(char(waitKey(0))!='q'){}
    return 0;
}

cout<<"Number of circles: "<<circles.size()<<endl; 
prints out the number of circles.

#11c. Varying Parameters in Hough Transform

Lets try varying the parameters in HT by trackbar and see what happens.

#include<iostream>
#include<highgui.h>
#include<cv.h>
using namespace std;
using namespace cv;
Mat img;

int thresh=100;
int a=1,b=10,c=100,d=5;
int a_min=1,a_max=20;
int b_min=1,b_max=100;
int c_min=1,c_max=200;
int d_min=1,d_max=10;

void on_trackbar_a(int pos,void *){
 a=pos;
}
void on_trackbar_b(int pos,void *){
 b=pos;
}
void on_trackbar_c(int pos,void *){
 c=pos;
}
void on_trackbar_d(int pos,void *){
 d=pos;
}
void on_trackbar(int, void *){
    Mat img_gray;

    cvtColor(img,img_gray,CV_RGB2GRAY);


    vector<Vec3f> circles;


    HoughCircles(img_gray,circles,CV_HOUGH_GRADIENT,a,b,c,thresh,d);

    Mat img_show=img.clone();

    for(int i=0;i<circles.size();i++){
        Point center(cvRound(circles[i][0]),cvRound(circles[i][1]));
        int radius=cvRound(circles[i][2]);


        circle(img_show,center,3,Scalar(0,0,255),-1);


        circle(img_show,center,radius,Scalar(0,0,255),3,8,0);
    }
    imshow("Shapes",img_show);
}
int main()
{

    img =imread("images/linescircles.jpg");
    namedWindow("Shapes");
    imshow("Shapes",img);

    createTrackbar("Acc. threshold","Shapes",&thresh,300,on_trackbar);
    on_trackbar(0,0);
    createTrackbar("A","Shapes",&a_min,a_max,on_trackbar_a);
    createTrackbar("B","Shapes",&b_min,b_max,on_trackbar_b);
    createTrackbar("C","Shapes",&c_min,c_max,on_trackbar_c);
    createTrackbar("D","Shapes",&d_min,d_max,on_trackbar_d);

    setTrackbarPos("A","Shapes",a);
    setTrackbarPos("B","Shapes",b);
    setTrackbarPos("C","Shapes",c);
    setTrackbarPos("D","Shapes",d);

    on_trackbar(0,0);

    while(char(waitKey(0))!='q'){}
    return 0;
}

Project 2.0. Detecting Circles from a Video Input

I hope you are familiar with Hough Transform.
In this post, I will show you how to detect circle from a video input using Hough Transform.

We know how to detect circles (from previous post), and if have gone through my previous tutorials, you would have a basic idea of how video, in form of frames, is inputted from the webcam and is outputted to the window.

Let's combine these two concepts and build a simple application.

#include<iostream>
#include<highgui.h>
#include<cv.h>
using namespace std;
using namespace cv;

int main()
{
 VideoCapture cap(0);
 Mat frame; // to capture from the webcam
 Mat img; // to process the 'frame'
 namedWindow("video"); //video output
 namedWindow("shapes"); //grayscale output
 vector<Vec3f> circles; //vector to store data from Hough Transfrom

 while(char(waitKey(1))!='q'){

  cap>>frame;
  //convert frame to grayscale
  cvtColor(frame,img,CV_RGB2GRAY);
  imshow("shapes",img);

  //apply HT to grayscale image 'img'
  HoughCircles(img,circles,CV_HOUGH_GRADIENT,1,10,100,50,5);
  // Here you can vary the second last argument(i.e. 50), according
  //to the threshold of your video.

  for(int i=0;i<circles.size();i++){

  //get 'center' and 'radius' of the detected circles
   Point center(cvRound(circles[i][0]),cvRound(circles[i][1]));
   int radius=cvRound(circles[i][2]);

   //draw center of the detected circle
   circle(frame,center,3,Scalar(0,0,255),-1);

   //draw the outline of circle
   circle(frame,center,radius,Scalar(0,0,255),3,8,0);
   }
  imshow("video",frame);
 }
 return 0;
}

Result:

This would seem to be easy, if you are thorough with most of my previous posts.

Everything until now was a child's play.
Real OpenCV begins from now.
Now, we will be focusing on algorithms and applying it in real life problems.

#11b. Hough Transform: Detecting Circles

#include<iostream>
#include<highgui.h>
#include<cv.h>
using namespace std;
using namespace cv;
Mat img;

int thresh=100; //accumulator threshold;

void on_trackbar(int, void *){
 Mat img_gray;
 //convert image to grayscale
 cvtColor(img,img_gray,CV_RGB2GRAY);

 //create a 3 element floating vector
 //this vector stores 3 floating values:
 //[0]:x-value of center of to-be-detected circle
 //[1]:y-value of center of to-be-detected circle
 //[2]:radius of to-be-detected circle
 vector<Vec3f> circles;

 //see text for explanation of following line
 HoughCircles(img_gray,circles,CV_HOUGH_GRADIENT,1,10,100,thresh,5);

 Mat img_show=img.clone();

 for(int i=0;i<circles.size();i++){
  Point center(cvRound(circles[i][0]),cvRound(circles[i][1]));
  int radius=cvRound(circles[i][2]);

  //draw center of the detected circle
  circle(img_show,center,3,Scalar(0,0,255),-1);

  //draw the outline of circle
  circle(img_show,center,radius,Scalar(0,0,255),3,8,0);
 }
 imshow("Shapes",img_show);
}
int main()
{
 //read image
 img =imread("images/linescircles.jpg");
 namedWindow("Shapes");
 imshow("Shapes",img);

 //create trackbar for threshold
 //threshold defines the minimum circle radius to be detected
 //see text for further explanation
 createTrackbar("Acc. threshold","Shapes",&thresh,300,on_trackbar);

 //initialize the window
 on_trackbar(0,0);

 while(char(waitKey(0))!='q'){}
 return 0;
}

HoughCircles(img_gray,circles,CV_HOUGH_GRADIENT,1,10,100,thresh,5);

Format:
HoughCircles(InputArray image, OutputArray circles, int method, double dp, double minDist, double param1=100, double param2=100, intminRadius=0, int maxRadius=0 )

Parameters:

image – 8-bit, single-channel, grayscale input image. -->Here, our input image is 'img_gray' circles – Output vector of found circles. Each vector is encoded as a 3-element floating-point vector . -->vector<Vec3f> circles; -->This vector stores data of circles detected circle_storage – In C function this is a memory storage that will contain the output sequence of found circles. -->Not relevant here. method – Detection method to use. Currently, the only implemented method is CV_HOUGH_GRADIENT , which is basically 21HT , described in[Yuen90]. -->Detection Method dp – Inverse ratio of the accumulator resolution to the image resolution. For example, ifdp=1 , the accumulator has the same resolution as the input image. If dp=2 , the accumulator has half as big width and height. -->In our case, dp=1; minDist – Minimum distance between the centers of the detected circles. If the parameter is too small, multiple neighbor circles may be falsely detected in addition to a true one. If it is too large, some circles may be missed. -->In our case, minDist=10; param1 – First method-specific parameter. In case of CV_HOUGH_GRADIENT , it is the higher threshold of the two passed to the Canny() edge detector (the lower one is twice smaller). -->In our case, param1=100; param2 – Second method-specific parameter. In case of CV_HOUGH_GRADIENT , it is the accumulator threshold for the circle centers at the detection stage. The smaller it is, the more false circles may be detected. Circles, corresponding to the larger accumulator values, will be returned first. -->In our case, param2=thresh; which is what we are varying from trackbar.  minRadius – Minimum circle radius. maxRadius – Maximum circle radius. -->Not relevant here. You may go through the documentation HERE.

One more point:
cvRound() Rounds floating-point number to the nearest integer.