Search by image: a review

A little over two weeks ago my favourite search engine (favourite because I’m too lazy to try the others) rolled out voice search and search by image. I tried them out so that, if you’ve not yet got a chance to use them, you can raise or lower your expectations accordingly 🙂

The idea for search by image is simple: drag and drop an image into the search box. I used my own photos which had the default name and no extra meta information (such as geotagging, etc) that would make the search engine’s job easier.

Architecture

It got these two right:

Search for St Paul's Outside the Walls
Search by image using a photo of St Paul's Outside the Walls
Search for Villa Aldobrandini in Frascati, Italy
Search by image using a photo of Villa Aldobrandini in Frascati, Italy

It was able to recognise the Spanish Steps even with the people, scaffolding and temporary decorative flowers:

Search for the Spanish Steps
Search by image using a photo of the Spanish Steps

It got stuck on the 21st century Ara Pacis museum:

Search for the Ara Pacis Museum in Rome
Search by image using a photo of the Ara Pacis Museum in Rome

Landscapes

It got the idea of sky and water:

Photo of Lake Nemi, Italy
Search by image using a photo of Lake Nemi, Italy

And that of sky and (mountainous) land:

Search by image using a photo taken in Antrodoco, Italy
Search by image using a photo taken in Antrodoco, Italy

Objects

If it can’t make out what’s in the photo, it goes for matching colours:

Search by image using a photo of a fruit stand
Search by image using a photo of a fruit stand
Search by image using a photo of a red flower
Search by image using a photo of a red flower

Faces

A human face is recognised as such, but I’d hope for more similarity (Picasa’s face recognition spoiled me):

Search by image using a photo of a person
Search by image using a photo of a person

Not wishing to use pics of my friends and acquaintances, I used my cats. Bolla looks like food (which is kind of funny considering Bolla eats A LOT):

Search by image using a photo of a cat
Search by image using a photo of a cat

Missing and presumed dead Tempesta is inscrutable, but at least there’s one cat in the suggestions:

Search by image using a photo of a cat
Search by image using a photo of a cat

A black and white shot of Wiki brings up lots of human faces, as well as an unexpected symbol:

Search by image using a photo of a cat
Search by image using a photo of a cat

Conclusion

I’d hoped that it would work better than it did, especially with faces. I guess that will improve with time.

Next: Should people with non-American accents bother with voice search?