Google: Smart Home Technology of the Future Will Anticipate User Wishes

BURLINGAME, Calif. — The concept of using computing and home automation in the home has been around for several decades. The 1967 short film “1999 A.D.,” for instance, presents a vision of central home computer that is “secretary, librarian, banker, teacher, medical technician, bridge partner and all-around servant in this house of tomorrow.” In this Ford-backed production, the computer keeps track of pertinent information about a family’s preferences and needs and makes recommendations related to everything from education to exercise routines to diet.

“We have been looking at the connected home — the smart home — for a very long time in terms of the zeitgeist,” said Ben Brown, product lead, Google Home and Nest apps and services at Google at the Smart Home Summit in Burlingame, Calif. But until fairly recently, smart home technologies were niche and tended to have a DIY slant. Rather than fulfilling the smart home prophecies from past decades, they tended to be aspirational and often expensive. The aforementioned vision of the smart home technology of the future, where user interfaces become frictionless and electronic gadgets work in tandem, leveraging artificial intelligence to support the household is beginning to crystallize, Brown said.

First Steps Toward a Truly Smart Home

Two of the first smart home technologies to gain widespread acceptance were connected thermostats and light bulbs. But while those technologies excelled at creating new types of experiences, “the problem we started to see with these things is they really didn’t work together,” Brown said. They also tended to require users to download specific smartphone apps, requiring users to use their phone as a remote control.

The rise of virtual assistants — most notably the Amazon Echo and Google Home speakers — enabled a wave of integration with many smart home devices. Instead of needing to have a smartphone nearby, “you could just seamlessly and frictionlessly ask for it and it would just happen,” Brown said. Smart speakers also gave users an easy way to perform a range of tasks such as check the time or the weather forecast, listen to the radio or a particular song. And more enthusiastic smart home users can use voice for adjusting multiple lights at once, opening shades, controlling connected sprinkler systems, ceiling fans and so forth. The model is changing from smart home devices to connected households.

The increase in functionality and integration has propelled the smart home market’s growth. The market research company IDC expects the smart home niche to grow by 31 percent in 2018 over last year and points out that smart speakers are the quickest-growing smart home category with a five-year growth rate of 39 percent.

Making the Smart Home Technology of the Future Real

While the recent explosive growth of the smart home market is noteworthy, “smart homes just aren’t that smart yet,” Brown said. The industry has done “an amazing job” using natural language processing in the home to transform basic interactions. “But we still need to spend that energy thinking of the context of the home,” he added. In the future, smart home devices shouldn’t just respond to commands; they should anticipate user needs and make suggestions.

“When we think about all the devices that we have in our home, they are actually starting to net together [to enable] what could be a pretty compelling experience,” Brown said. But delivering this promise will require improved integration of disparate smart home devices. In theory, devices ranging from robotic vacuum cleaners to smart light switches and connected security systems should be working together and learning from each other.

Anticipating User’s Needs

When Google’s search engine launched in 1998, it took a one-size-fits-all approach to locate content on the web. In recent years, the search engine — and other Google services — have increasingly focused on context and user intent. Now, if you search for “Mexican food” on the site in the late 1990s, you turn up not just relevant cookbooks and historical information about the cuisine, but local restaurants with user reviews.   

Google has also extended the functionality of Gmail to suggest default email replies and, more recently, context-based phrases with its Smart Compose feature. In a demo of the feature, a user writing an email to a person named Jacqueline Bruzek with the subject line “Taco Tuesday” sees the recommendations “Hey Jacqueline” pop up for the beginning, while also helping populate suggestions about “guacamole” and “chips and salsa.” In this demo, Smart Compose also offers to automatically type the user’s address.

“These things are very natural for automated intelligence and machine learning,” Brown said. As the smart home market matures, expect similar functionality to debut where smart home devices work to anticipate user needs.  

The recently launched Google Home Hub includes a screen that can allow users to share recent images with friends and family. With a feature known as “Live Albums,” a user can share not just a specific image but a memory that updates over time. “What we saw with [the photo sharing and storage service] Google Photos was that the experience was all about memories,” Brown said. “It wasn’t about photos.”

“For me, I wanted to be able to capture memories of what my baby looked like six months ago and just be able to see my baby grow up over time,” Brown said. “I didn’t want to have to search through my whole photo catalog.” Instead, the system automatically identifies relevant pictures and can share them with others. “I set this up for my mom. She, with her Google Home Hub, can automatically see pictures of my baby and have those show up and automatically populate over time,” he added.

After making significant progress with voice, both Google and Amazon are refining users’ ability to interact with their smart home devices in different ways. “There are certain things that [are] hard to do purely through a voice medium,” Brown said. The Google Home Hub offers support for voice commands as well as a touch-screen interface for controlling lights, thermostats and video cameras. The screen (as well as a TV with Chromecast) can also provide a live view of a person who has just rung a supported smart doorbell outside.

As the smart home technology of the future becomes a reality, expect smart home devices to not just interact with conversations initiated by users, but also increasingly initiate that type of conversation interaction. “We can be less confident of the outcome and still facilitate a natural interaction for the user,” Brown said. “You can suggest an interaction.” For instance, if a user wants to relax in the living room, a virtual assistant could ask if it should dim the lights. Or the assistant could notify the household that the oven is preheated to 425 degrees. “It will let you know you can do the next task,” Brown added.

When users announce they are leaving the house, the virtual assistant could automate turning off the lights and set the thermostat to an energy efficient mode. It will be able to “understand user intent and user desire without having them preprogram all of these things,” Brown said. Smart home technology of the future can anticipate intent and make recommendations based on context. “This will enable more thoughtful experiences.”