Merler CSE 190A Project

2006-07-10T10:43:00.000-07:00

From now on, this blog will be dedicated to a new project I'm working on, called GROZI.

2006-04-05T22:23:00.000-07:00

Here are the links to my final presentation and report.

2006-03-06T15:49:00.000-08:00

All the thresholds and values presented were adjusted considering the statistics of 20 pictures from the databases. It must be pointed out that the database images have different sizes: Vision Caltech 1999 are 896x592, Vision Caltech 2001 are 360x240, while the Computer Vision Laboratory at UCSD are 640x480.

In Figure here is the result of the license plate detector, for a window of size 125x61

Licence Plate detection

2006-03-01T15:53:00.000-08:00

Licence Plate detection will be performed by applying a filter to a region restricted by the backlights position, focusing on the X-derivative of the grayscale converted image.

Backlights Detection

2006-03-01T15:39:00.000-08:00

Backlight detection is performed through the following steps:

- Cr thresholding: the image is converted into YCbCr and a threshold of 160 is applied. Some output results are presented in the Figure below. Such simple operation turned out to be quite effective: in fact the large majority of backlights are detected, with some issues. The first concerns red or orange cars, which are totally highlighted. This is not a problem, on the contrary, helps the system dteecting a whole car. The second is the presence of multiple regions, which must be analyzed to distinguish between backlight and non-backlights.

- Distance and area comparison: each labeled connected component in the thresholded image is compared with others of roughly the same size and Y-coordinate, but sufficient different X-coordinate

- Filtering : at this point, when two regions are compared, one is considered as a filter, and therefore flipped and compared to the other one. The features computed are: ratio between Major and minor axes of an ellipse of 2nd order moment equal to the one of the region, ratio between area of the convex hull and area of the region.

If two regions are likely to be backlights for the system, it proceeds to the next step: licence plate detection.

Block Diagram

2006-02-28T20:30:00.000-08:00

A block diagram of the system. The car detection task, not being the main concern of the project, is intended to be simple.

Segmentation improvement

2006-02-15T15:49:00.000-08:00

So what I miss to do right now is to apply the NN algorithm to segment the images and see how it performs in comparison with the grid one. Not being able to compile the program until yesterday, I could not really work on that. I would like to try to find some simple way of getting the SUV colorspace to work. In theory it is very promising and i would like to compare its performances with the other two.

Kriegman, Mallick, Zickler, and Belhumeur; CVPR 2005

2006-02-15T15:31:00.000-08:00

In "Beyond Lambert: Reconstructing Specular Surfaces Using Color" (by Mallick, Zickler, Kriegman, and Belhumeur; CVPR 2005) pdf , a particular color space is presented: SUV. As many others it separates luminance (S) from chrominance (UV), but the most interesting part is that, by considering a dichromatic model of reflectance, they consider the reflectance as decomposed into two additive components:

- the interface (specular) reflectance
- the body (diffuse) reflectance

So getting rid of the specular reflectance one will be able to get the pure color of the objetct! That is done mapping the image into onother colorspace, where one axis(S) is perfectly alligned with the source color S. Here comes the problem: how to determine S?

In the example is presented the strength of the method, but in such case S was know a priori, which is not the case of our problem.

I tried to average 4 UV images obtained with 4 random generated source colors, but the results, as shown down here, are quite poor.

Segmentation 2

2006-02-15T15:24:00.000-08:00

So the first sample was an easy one. Let's consider something more challenging (and interesting). Image #13 presents an evident specularity on the left side. Here the algorithm fails badly to connect side and back of the car. I tried to play with k a little bit, but of course it is not helpful, since the problem does not come from the area of the regions, but from their colors!

So is specular reflectance a problem? For sure! So I thought that maybe it affected the statistics on the color distributions I did too. A possible solution could come from CSE 252A...

Segmentation

2006-02-15T15:01:00.000-08:00

Here are the first results of the segmentation algorithm. I chose a simple image as starting point, just to see how sigma, k and min actually work. The test image is #104 from my car colors dataset.

As expected, k is the key parameter to select, since it deeply influences the merge criteria of the algorithm (big values of k force the algorithm to merge greedily small regions while preserving big ones). Sigma is the variance of the Gaussian filter applied at the beginning to smooth the image. Such preprocessing step could maybe be improved, I will post about it later. min i a postprocessing step that erases from the world regions smaller than a certain area.

In the images below I put some statistics: first are the number of components detected by the system as k or minArea increase, with the other two parameters fixed (sigma = 0.8). Clearly they both decrease, both it must be noticed that the k curve is much sharper, that is, k affects the algorithm in an heavier way.

Here under are the same statistics, but visualized with the images. We can note that for k = 900 and min >= 70 the algorithm get the car pretty well. Should we infer a generalization on that? Not at all!!!!

Felzenszwalb Alleluia!!!!!

2006-02-14T23:30:00.000-08:00

Finally I managed to get Felzenszwalb code run!!!
Soon the first results (hopefully better than this one) playing a little bit with the parameters.

Segmentation follow-up

2006-02-06T15:53:00.000-08:00

The next step will be an evaluatin of the performances of the algorithm on my dataset, comparing the code as it is with an implementation of a K-nearest neighbor one. I wiil read through the ANN paper too, to see what advantages it gives, compared to the normal one.

Felzenswalb Segmentation

2006-02-06T15:25:00.000-08:00

Felzenswalb segmentation algorithm looks appealing for my project since is fast, quite simple and boasts an example in which reflectance on a car surface does not affect the segmentation in a sigle region for a car. This last aspect is very interesting, considering the color distribution I found for the car colors, which are influenced (among the others) by reflectance factors.

I read Felzenswalb's article to understand what segmentation it applies exactly. It turned out that 2 different methods are applied, both graph based:

- GRID GRAPHS
- NEAREST NEIGHBOR GRAPH

I will describe them in class, the second one being the most appealig for my project.
It claims the use of an Approximate Nearest Neighbor (ANN) algorithm, which is referenced, but not explained (I will read the referenced article)

I took a look at code, downloadable for free from here, to understand which method of the two was implemented.
Ironically, none of them was exactly reproduced in the code. In fact, the program runs the Grid Graphs with an alteration in the graph weights computation.

2006-02-01T15:50:00.000-08:00

Here finally I plotted the distributions of 7 colors (I picked only the Light part of each color, since Matlab does not allow more then 11 different distributions to plot).

- Red, Black, Yellow, Orange, Blue, Green, Silver, Brown

Some colors are distinguishable, some not. No surprise then that rg and CbCr were not able to separate them.

Maybe I should try Lab (Robin kindly gave me a code source for the transformation).

2006-02-01T15:48:00.000-08:00

Here are the RGB 3D scatterplots, confronting the colors distribution in pairs:
- Light and Dark : Red, Blue, Green, Silver, Brown
- Yellow and Orange
- Black and White (represented in violet)

2006-02-01T15:44:00.000-08:00

2006-02-01T15:41:00.000-08:00

In order to generate the scatterplots in RGB, since my samples were too many, I took 4 vertical lines of samples from each color square (as presented in the picture). The vertical lines allow the system to get data from each original sample image, since they were put into the square horizontally.

2006-02-01T15:37:00.000-08:00

I rearranged the color samples in sqaured images, divided in horizontal stripes. Each stripe represent the pixels of one sample image

2006-02-01T14:43:00.000-08:00

2006-01-23T15:56:00.000-08:00

Still many color regions are overlapping, allowing to distinguish only 6-7 colors. A check on the results and a chioce of another chrominance plane are the nexts steps to be done.

2006-01-23T15:37:00.000-08:00

I have extracted the histograms in normalized rg chrominance plane

and in CbCr chrominance plane

and produced a 2D intersection of the histogram, to realize whether they are overlapping.

rg

CbCr

Due to the poor results of CbCr, I computed the transform in a different way, that worked well for another project a couple of years ago:

CbCr

2006-01-23T15:30:00.000-08:00

I manually extracted the color pixels from each of the labeled cars, obtaining samples or "masks" like the following:

2006-01-11T15:34:00.000-08:00

Here are the first statistics I'm doing on five people analyzing a car database of 150 images collected from the web. Such operation will allow me to have a reliable database from which to get car color samples and create the model of each color in the different chrominance planes

These are, as an example, the cars labeled as Light Green. I will extract from here the color samples to train the system

This one tells the percentage of indentification of each color by person 2 among the database samples. E.g. how many yellow cars person 2 found among the 150 samples. The colors are labeled as numbers:
1 = Light Red
2 = Dark Red
3 = Light Blue
4 = Dark Blue
5 = Light Brown
6 = Dark Brown
7 = Light Green
8 = Dark Green
9 = Light Silver
10 = Dark Silver
11 = Orange
12 = Yellow
13 = Black
14 = White

Up here every car sample is labeled with a color if:
- 4 or more people agree on the color
- 3 people agree on a color and the other 2 disagree between them

A sample is defined 'car not labeled' if :
- less then 3 people agreee on the same color
- 3 people agree on one color and the other 2 agree on another one

Papers references

2006-01-11T14:50:00.000-08:00

Here's the list of papers I found until now, with a link to each of them, where possible.

[1]Belongie, S.; Malik, J.; Puzicha, J. “Shape matching and object recognition using
shape contexts”. Pattern Analysis and Machine Intelligence, IEEE Transactions on Volume 24, Issue 4, April 2002 Page(s):509 – 522
pdf

[2] Dlagnekov Louka, “Video-based Car Surveillance: License Plate, Make, and Model
Recognition”, U.C. San Diego (Masters Thesis) , 2005
pdf

[3] Drew M. S., J. Wei, and Z.-N. Li, "Illumination-invariant color object recognition via compressed chromaticity histograms of color-channel-normalized images," in Proc. of ICCV '98, IEEE, 1998.
pdf

[4] Finlayson G., S. Chatterjee, and B. Funt, "Color angular indexing" in Proc. of ECCV96, pp. II:16--27, 1996.
pdf

[5] Gonzalez, Woods, “Digital image processing”, 2nd edition, Prentice Hall, 2002

[6] Hagedoorn M.and R. C. Veltkamp. “Metric pattern spaces”. Technical Report UUCS -
1999-03, Utrecht University, 1999.
pdf

[7] Lee J. Y.and S. I. Yoo. “An elliptical boundary model for skin color detection”.
Proc. of the 2002 international Conference on Imaging Science, Systems, and
Technology.
pdf

[8] Lowe D. “Distinctive image features from scale-invariant keypoints.” IJCV,2(60):91–110, 2004.
pdf

[9] Ma W. Y., Y. Deng, and B. S.Manjunath, "Tools for texture/color based search of
images," Proc. SPIE Conference, Human Vision and Electronic Imaging II, Vol. 3106,
San Jose, CA, pp. 496-507, Feb 1997
pdf

[10] Merler M., “Video processing and analysis techniques for smart environment
application” University of Trento, Italy, 2004

[11] Mindru F., T. Moons and L. Van Gool. “Color-based moment Invariants for Viewpoint and Illumination independent Recognition of planar Color Patterns”, International Conference on Advances in Pattern Recognition, ICAPR'98, 1998, pp. 113--122
pdf

[12] Remco C. Veltkamp and Michiel Hagedoorn. “State-of-the-art in shape matching”.
Technical Report UU-CS-1999-27, Utrecht University, the Netherlands, 1999.
pdf

[13] Rojas, J.C.; Crisman, J.D , “Vehicle detection in color images”; IEEE Conference on Intelligent Transportation System, 12 Nov. 1997. ITSC 97. Page(s):403 – 408
pdf

[14] Rubner Yossi, Jan Puzicha, Carlo Tomasi, and Joachim M. Buhmann, “Empirical Evaluatio of Dissimilarity Measures for Color and Texture”. Proc. ICCV ’99, pages 1165-1172, 1999
pdf

[15] Swain M. J., D. H. Ballard, “Color indexing”. Int. J. Comput. Vision, 7: pages11-32.1991.

Project Proposal

2005-12-13T20:32:00.000-08:00

pdf

Michele Merler Blog for CSE 190A

2005-11-15T01:55:00.000-08:00

This blog has been created by Michele Merler for the Project in Vision and Learning, class CSE 190A, Winter 2006