We recently had the chance to sit down and have a short chat with Jiang Li, Researcher/Project Leader of Microsoft Research discussing Microsoft Portrait. Here is an excerpt:

ActiveWin.com: What was the most difficult aspect of the development process?

Jiang Li: It's how to reach low bitrate target at the beginning of development process. The motivation of the project came from the observation that current video communication software is still not suitable to dialup users who have only about 56 Kbps bandwidth (the actual available bandwidth is bout 80% of it) and exist in most areas of the world. So we want to develop a codec that works at 10-20 Kbps, therefore can provide two-way video communication for dialup users. Since it seems that there are no more rooms in the improvement of conventional DCT based coding, we consider using line drawings in video communication in which expressions are the most important and scenes are relatively fixed.

We tried edge detection algorithms in order to extracting the outlines of face, eyes, eyebrows, mouth, etc., but the results are not very robust. In addition, the visual effects are also not satisfactory since if you only write the outlines with black lines, you will see that the hair areas are white - the color of the background. In order to avoid this situation, we consider combining the outline image with a binary image that is converted from a gray scale image by a threshold. In this case, hair areas are always black and the visual quality improved significantly. Although the visual quality becomes better, the compression ratio cannot be higher due to the existence of lines and dots from the outlines. We considered what would happen if we just used the binary image that was converted from a gray scale image by a threshold and did not use any outline information. It is surprising that the visual effects are even better. This is exactly what the current black/white video form is.