How to capture the smile? The secrets of “smile-detect” technology.

On September 21st, Sony shipped their “DSC-T200” and “DSC-T70” digital cameras to Japan. Both cameras have a unique function called “Smile Shutter” that captures people’s smiling faces. We interviewed Ryo Ochi, Product Planner at Digital Imaging Business Group of Sony, about the background of development.

Massive data of smiles collected worldwide

―What is the “Smile Shutter”?

Once users set the mode to “Smile Shutter,” the camera spots people’s faces and automatically pops the shutter when they smile. It is able to take a maximum of six pictures in a row. Additionally, to detect the different degrees of smiles by subjects, we can set the Smile Level sensitivity to “high” (to detect a faint smile), “medium” (for a normal smile), or “low” (for a hearty laugh). I think this is very high-tech function that other companies cannot easily imitate.

―How does “Smile Shutter” define the smile?

When people smile, we can see their teeth and both ends of the mouth go up and the outer corners of the eyes go down. “Smile Shutter” detects the smile from these expressions of teeth, mouth and eyes. This function is available not only for Japanese people but also for people all over the world. However, if subjects wear sunglasses or hats over their eyes, the camera cannot find their smile. The most extraordinary feature of the “Smile Shutter” is the speed between “detecting the smile” to the “release of the shutter.” “Smile Shutter” will release the shutter the same time a subject smiles. One’s smile is just for a moment. To take that moment clearly, “Smile Shutter” has an exclusive circuit for face detection. We labored for the rapid action not by using software but by the hardware.

―What point have you started the development?

There are so many secret points, but I will tell you as much as possible. A few years ago, we began gathering pictures of smiles from all over the world. I cannot tell you the number of pictures, but it was extensive. We analyzed these pictures by using super computers and made numerical records for the database of smiles. We made the greatest effort at this point because the quantity of data will influence the quality of the “smile-detect” technology. The definition of smile also varied from the situation of the pictures. It was very difficult to decide whether the faces were smiling or not for each picture.

To take the picture of smiling kids

―Why you did you focus on the smile?

I was using the “DSC-R1,” a high-end camera from Sony. However, one day, I was not able to take the smile of kids. They moved very quickly and it was difficult to focus them, and they also hardly smiled when we tried to take their picture. I thought that it was very difficult to capture their smile even if we had high-performance cameras. Then I started to think about how to take the smile more easily. Finally, I hit upon the idea to make the camera find the smile. At the same time, our technical group told me that it is possible to detect the smile, so we started development.

It is said that to capture the smile is difficult. It is not easy for us to make a natural smile and to take the very moment of that smile. We often make forced smiles when faced with a camera and kids cannot stay in the same place even if it is just for a short period. We are confident that this camera can be a hit product because there is potential demand.

The recent camera market has been influencing consumers to buy based on specifications such as pixels, zoom or ISO. However, it becomes difficult to differentiate products by specifications. Therefore, we made another “pillar” of sales promotion that is the “Smile Shutter” function, which makes the camera a more enjoyable device to use.

Make people smile, make the world brighter

―In what situations can we enjoy the “Smile Shutter”?

I think people want to take treasured shots such as the full smile of a joyous moment. For example, we can take the smile of a baby while we cradle them, or we can practice smiling with “Smile Shutter.” We can also take the picture by smiling so we don’t need to release the shutter button, which makes it easy to take the two-shot picture of a couple. I think “Smile Shutter” is a function that makes people smile and makes the world brighter. It is still uncommon to install this function into other products, but I hope the “Smile Shutter” becomes the standard function for every digital camera.

―Do you have any ideas for the next step of “Smile Shutter”?

I will continue to explore the other “pillar” that makes people happy. We cannot find new ideas only from data or research results. I want to make new functions based on my personal experience with potential demands like “Smile Shutter.” How about a function that can re-create the scene with the tone or atmosphere we felt? It is also fun to think about a function that can take steady pictures from inside of a moving train, or can take fine pictures no matter how dark a place is. I want to develop a camera that can take the precious moments easily and clearly. (Interviewed by Akihiro Tsue, BCN)