Sketch2Photo

October 14, 2009
Stacey Mason

If you’re like me, you’ve occasionally wished your MS Paint doodles were a bit more high-tech or that you could produce something that looks less like your three-year-old niece’s crayon art. Well, the future is here!

A group of students in China have created a program called Sketch2Photo , which takes a hand-drawn outline sketch, searches the web for photos that match the sketch, and brings it all together in a composite picture. The results, in press at ACM Transactions on Graphics, are actually pretty incredible:

Sure, the new image doesn’t have exactly the right lighting and shadows, but it definitely comes out looking less photoshopped than most tabloid photos.

“We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found by searching the Internet. Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images. We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful; we also evaluate our system using the results from two user studies.”

One issue that immediately comes to mind is the issue of photo ownership and copyright infringement: here we have the music sampling issue appearing in a new context. . At the very least, the software promises interesting things for the future of image editing.

Sketch2Photo: Internet Image Montage from Tao Chen on Vimeo.

.