Windows Phone 8 Tip: Find Out What You're Looking At

The Bing experience in Windows Phone 8 includes an integrated feature called Bing Vision that helps you find out more about what you’re looking at while you’re out in the world. Bing Vision understands QR codes, Microsoft Tags and bar codes; and CD, DVD, video game, and book covers. It can even scan and translate text.

Note: This tip is from my new (and free!) Windows Phone 8 book, which is currently in progress. You can follow along at WindowsPhoneBook.com.

Because your Internet-connected smart phone is with you at all times when you’re out in the world, it makes sense that you would expect to use this versatile tool to find out more about what you’re looking at, at any given time. And while it’s possible to use visual clues to manually conduct an Internet search to discover more about the world around you, Windows Phone’s Bing experience includes an integrated feature called Bing Vision that makes doing so even easier.

Bing Vision is the ultimate in integration, combining the device’s camera with software-based visualization capabilities and the sweeping set of Bing online services to allow you to find out more about what you’re looking at. Bing Vision specifically understands QR codes, Microsoft Tags and bar codes, and it can scan text from virtually anywhere to drive a search query. But it also understands far more than that, including CD, DVD, video game, and book covers. You can use it to discover whether a product is available near you, or online, and then compare prices.

While I’ll highlight some key usage scenarios in more detail below, the basics are the same no matter what you’re trying to find out more about. Simply launch the Bing experience on your handset, tap the Vision app bar button (it looks like an eye) to change to a view through the camera, and then point the phone at the item in question.

Depending on the item you can see through the screen-sized viewfinder, Bing Vision will return standard Bing search results or display a Quick card for the item.

Here’s what you can learn more about with Bing Vision.

Barcodes, QR codes, and Microsoft Tags. Bing Vision can interact with Barcodes, QR (“Quick Response”) codes, and Microsoft Tags, which are two-dimensional, machine-readable codes that provide data about the object on which they can be found. To scan any of these code types, enable Bing Vision and point the viewfinder towards the code. Bing Vision will identify the code and provide a link.

If it’s a bar code for a retail product, tap this link to see Bing search results. But other code types can act differently. For example, oftentimes when you scan a QR code in an advertisement—perhaps in a magazine, or in a subway—you will navigate to the web site related to the advertised product, using Internet Explorer.

Scan and translate text. You can use Bing Vision to scan text and then copy it into the phone’s Clipboard. From there, you can paste it into an app like OneNote or Word and save it for posterity. To scan text, enable Bing Vision, point your phone’s camera at the text, and tap the Scan Text button. After a second, Bing will display a photo of the text with the blocks of text it understands highlighted.

Bing Vision can also translate text in a foreign language to your own language. To do so, engage Bing Vision and tap the Scan Text button as before. But this time, select the Translate button. Bing will display the translated text—it’s not always very accurate—onscreen.

You can also trigger Bing searches from scanned text—including text you’ve translated. Just tap individual text blocks in the Bing Vision scan view after a scan or translation and they’ll be added to a search box that appears at the top of the screen, in sequence. Tap the Search button to search on that term.

CDs, DVDs, video games, and books. Bing Vision can also be used to find out more about specific physical products, including music CDs, movie DVDs, video games, and books. To find out more about an item this way, enable Bing Vision, point your phone’s camera at the item, and then hold still for a second. Bing will fill in a list of potential search results.

Tap the one you want and a standard Bing Quick card will appear so you can find out more and potentially buy it online.

Don’t forget: You can find out more about Bing Vision, the Bing experience in Windows Phone 8, and Windows Phone 8 in general in my in-progress book, Windows Phone Book.