Computer Vision and Human Skin Colour
Ph.D. Dissertation by
Moritz Störring
Computer Vision & Media Technology Laboratory
Aalborg University, Denmark
mst@cvmt.dk


A crucial processing step in computer vision based face and gesture recognition is robust detection and tracking of faces and hands. This is often done by combining complementary cues, e.g., motion, shape, and colour. Skin colour is used because it is invariant to orientation and size, gives an extra dimension compared to grey tone methods, and is fast to process. The main problems with the robustness of skin colour detection are: dependence on the illumination colour, variations between individuals, and many everyday-life objects are skin colour like, i.e., skin colour is not unique.
The objective of this study is to open for an improved skin colour cue, and the focus is to investigate the image formation process theoretically and experimentally – in particular with respect to human skin colours under changing and mixed illumination.  Physics-based approaches are used to model the reflections of skin and the image formation process when registered by a camera.
It is shown that skin colour “perception” as viewed by a state-of-the-art colour video camera can be modelled sufficiently accurate with a physics-based approach given the illumination spectra, the reflectance of skin, and the camera characteristics. This modelling may provide the basis for applications such as adaptive skin segmentation. For adaptive segmentation it would be useful to estimate the illumination colour and to track skin areas through changing illumination conditions. Such methods are suggested and tested showing accuracies sufficient to improve adaptive skin segmentation. Finally, the reflectance characteristics of skin in the near infrared (NIR) spectrum are explored. A combination of standard RGB bands with three narrow NIR bands is suggested to robustly detect skin under changing illumination and distinguish it from other skin colour-like objects.
The results of this work may contribute to an adaptive skin colour cue that in combination with other cues will enable robust face and hand detection in unconstrained environments. The features of the skin colour cue, which combines the methods developed, are outlined at the end of this work