Blippar now recognizes cats, other items without AR codes

Augmented reality app Blippar is hardly new (it’s been around since 2012, almost as long the lab!), but it’s back in the bright lights after curiously shifting its focus to features associated with machine learning.

Until the latest update, the U.K.-based Blippar was mainly focused on advertising and content for brands. In a nutshell, Blippar allowed customers to “blipp” (or scan) a logo on its partners’ products to get more information. So, kinda like a QR code, with gusto: Blipps can generate a variety of actions on the partaker’s mobile device, like launching audio or video clips, dialing a phone number, or connecting to a social media page.

Blippar gained quite a bit of traction with its initial concept, partnering with several large brands and media companies including Conde Nast, TIME, and Unilever. But over the past year and a half, and for reasons unknown to us, founder Ambarish Mitra shifted Blippar’s focus towards machine learning, even going so far as to move his team from the U.K. to the Silicon Valley.

With the most recent update, Blippar gained the ability to recognize objects that don’t include a logo – such as dogs, apples, and cars. And as Mitra puts it, in an interview with Re/code,, the technology is still in the early stages, like the brain of a six-year-old. It can recognize “car,” but not “Prius;” an item of clothing, but not the label.

So naturally, we took it for a spin to see for ourselves how the technology works.

When I aimed my phone at Raya the cat, sitting in the windowsill, a flurry of words swirled around my screen like a tag cloud. And initially, the app seemed to have some difficulty distinguishing between species. It got “mammal” right away, but seemed to think my cat was a dog (to its credit, she does play fetch and growl at the mailman.) Finally it caught on and started bringing up cat-related words like “curiosity,” “kitten,” and “tabby.”

An icon of a cat appeared on the screen and when I tapped it, more icons appeared with different actions – giving me links to buy pet food, watch cat videos, and donate to the Humane Society.

But, in contrast to most apps that tie into e-commerce, Blippar wasn’t able to read barcodes when we tested it (however, it did spout a bunch of words like “savings” and “finance” when I aimed my phone at a barcode.)

Next, I stepped outside to go see what Blippar made of the chickens. As soon as the flock appeared in view, Blippar blurted another jumble of words, like “farm,” “food” and “nature.”

Impressively, the app even seemed able to distinguish between the hens (which it called “beautiful,” and we have to agree) and the rooster, zeroing in on Caesar’s tail with the word “cockerel”:

 

Back inside the house, it thought it saw a “squash racket” in the kitchen and initially mistook a pair of scissors for a table knife, but then quickly got it right. When I pointed my iPhone at an orange, it immediately displayed an icon of an orange followed by a bunch of juicy descriptors:

In general, the app did a remarkably good job of recognizing anything I put in front of my iPhone lens. Until I got back to my desk, that is. When I aimed my phone at the glass of water sitting next to my computer, Blippar declared it happy hour.

“Vodka!” “Cocktails!” Even whisky and wine – it must have chimed in with every type of alcohol (and juice) — but didn’t once mention water.

Thinking the Blippar app might be confused by the fact that I was drinking water out of a pint glass with the words “hard cider” on it, I filled a plain water glass instead. But as I focused on the glass of water with my work still in the background, it did the same thing.

Then a glass appeared on screen with a bunch of options, and I thought Blippar might have some sort of crowdsourcing feature that could help it learn the difference between water and whisky. Alas, no, but it did give me the opportunity to shop for water glasses online, learn about how to safely dispose of a broken glass, and watch a video of how to pour the perfect beer.

 

 

Aside from the fact that I felt a bit like the app was peer-pressuring me to pour a cocktail at the beginning of a workday, the revamped Blippar is surprisingly entertaining – particularly when you take it outside for a walk – and we’re interested to see how its machine learning evolves over time, and what can be done with it.

We can think of plenty of possible uses down the road (for example – describing the view through the lens to visually-impaired people). But in its current state, Blippar doesn’t provide much more than entertainment and a few links to buy things online. Is Blippar just a blip on the radar, or will we see it expand into something more useful? We’d love to hear your thoughts in the comments.