Thursday, September 17, 2009

Eyes region tracking in Common Lisp

I was going to implement eyes region recognition in Lisp, but seems I'm loosing interest in that area. So I've decided to publish what I already has and forget about it.

I've used CL-V4L2 to capture images from two USB cameras and CL-GTK2 with GtkGlExt extension for rasterization. The main idea of how to detect eyes region was taken from [1]. I can also mention [2] and [3].

The basic idea is to apply edge detection filter to gray scale image, this will stress regions with lots of details. More details, more edges, more visibility in filtered image. Region around eyes has a lot of details, btw. Next step is to compute vertical and horizontal derivatives of edges and find extremums at each axises. Ideally, this will give us one horizontal extremum at eye's level and two vertical extremums at the head borders. Or, if a very strong median filter was used, it may give you one vertical extremum near the center of the face.

However, I didn't succeeded in finding of good filter. First filter I've used was Sobel's operator [4]. Basically it works well for edges detection, but... Too much well for my task... It also finds edges in background parts, like shelf,  window, wall corner, etc. All this edges were false positives for my purpose. I've started to play with different similar filters, like Scharr's operator, Robert's cross [5] and also my own modifications of Sobel. The intention was to stress horizontal derivatives of edges during energy computation at X axis and, contrary, vertical derivatives at Y axis. I found Sobel/Scharr filter works better if G (gradient magnitude) was computed separately for X and Y energies the way described below:

Here's matrix for Sobel:

and for Scharr:

I've combined Gx from Scharr and Gy from Sobel to compute G for X energies and Gx/Sobel, Gy/Scharr for Y energies.

So, filters output was slightly improved, but still not enough to reduce unneeded edges. Much better result was brought by simplest Robert's cross. It doesn't filter edges well, but edges itselves are not what we definitely want. We want to stress eyes region, and Robert's cross does it better than Sobel or Scharr. However, image processing became more sensitive to background light. The only good time during the day was at evening when artificial daylight lamps were turned on. Noise reduction filters (a bunch of different modifications of Gaussian filter) didn't help at any degree. Finally, all smart computational logic was swept away leaving plain Robert's cross alone, which gave me the best result. Most of problems were caused by my beard and mustache, but I'm not gonna sacrifice my True Siberian Beard(tm) for the glory of science :)

In my best attempt, I got this:

I really enjoyed to play with it in Lisp, because Lisp gives you very powerful constructions like LOOP macro, which, for example, allows you to implement digital filter very quickly with only small amount of syntactical garbage. I can't forget to mention about automatic memory management, it saved me a ton of time. Nevertheless, with all these high-level whistles and bells, my non-optimized program still worked quite fast. I'm happy.

Source is available here:


No comments: