Will you let ChatGPT control your smart home?

​Josh.ai demonstrates a proof-of-concept ChatGPT integration using its voice assistant. Josh systems are currently only available through professional installers. Image source: Josh.ai

Nowadays, smart home users can ask Alexa to turn on the lights or Siri to tell you the temperature in their bedroom, and sometimes it will be accurate. Or you might hear, "You have 15 devices named lights; which one do you want to control?" or "The current temperature at XXX is 35 degrees Celsius." But what if your voice assistant is not only always accurate? You can also respond to vague comments, such as "I'm having a bad day; what's a good way to relax?" Have an "intelligent" response? For example, by lowering the curtains, dimming the lights, adjusting the thermostat, and buying some goodies on an online platform? Alex Capecelatro, co-founder of the Josh.ai home automation system, says this is the potential of voice assistants powered by new AI language models. Josh.ai has begun prototype integration using OpenAI’s ChatGPT. This proof-of-concept video shows Capecelatro asking Josh's assistant to open the curtains, turn off the music, and tell him the weather (controlling three things simultaneously is something Josh already has the ability to do). He then moved on to more natural voice commands for the smart home, like "I'm taking a video; it's a little dark in here," to which the voice assistant responded—albeit a little clunkyly—by turning on the room's lights.

The possibility of improving smart home control by using AI language models to parse natural language is tantalizing. Capecelatro thinks this is the future. "We're trying to figure out how we can control your environment in a more natural and intuitive way," he said.

Today’s voice assistants often require precise language and often confuse basic smart home commands with requests for information, which results in frustrating and sometimes useless responses. This was the problem Josh.ai set out to solve when Capecelatro and Tim Gill (founder of Quark) founded the company in 2015. Its eponymous voice assistant is designed to control connected gadgets brilliantly, no matter how the request is expressed.

"If we don't adopt ChatGPT type technology, businesses like mine will cease to exist within a year. This is critical to the future of anyone with voice control in their home."

Using an extensive knowledge graph model, Josh can parse and do the appropriate thing when he hears "satellite" instead of "turn on the lights." "Open the curtains" sounds like "get some grapes," but Josh is smart enough to know that you don't live in a vineyard. "We spend a lot of time working under the hood to fix mishearing, deal with different accents, understand imperfect sentences and more, so even if you say 'turn on the damn lights,' we know what you mean," Capecelatro said .

Currently, Josh is only used as a voice control layer in custom smart home installations powered by companies like Crestron and Control4. In that more protected environment, systems are set up and controlled by professional installers and use proprietary hardware from Josh.ai, a voice assistant known for being more reliable and private—albeit with a higher cost of entry. ( While Josh has a cloud component, most requests are handled locally on Josh Core, and identifiable information is removed when using the cloud-based API, Capecelatro said.)

Josh.ai The system includes a hub and two models of smart speakers as well as an app and integration with the Ava smart remote. Image source: Josh.ai

The company, which recently announced a partnership with Amazon, is now betting on a new generation of large language models (LLMs) used by ChatGPT and other chatbots. Capecelatro believes these systems will make today's voice assistants more useful. "A year from now, no one is going to want to tolerate the old way Alexa, Google, Siri or even Josh works. It's not enough," Capecelatro said. "If we don't adopt ChatGPT type technology, businesses like mine will cease to exist within a year. This is critical to the future of anyone with voice control in their home."

For Josh.ai, which doesn’t have the same general depth of knowledge as its competitors, the ChatGPT integration added to the voice assistant’s knowledge base is a huge leap. “We always wanted to make Josh as smart as possible, but we’re a small team,” Capecelatro said.

But for smart homes in general, the hope lies in combining the conversational capabilities of AI language models with the context that home automation systems can provide. For example, by understanding details about what smart devices you have in your home and how you use them, Josh can parse natural language commands into actions in your home. Say, "Hey Josh, it's almost time for the kids to get home and it's getting dark. Are you sure everything is ready?" For example, the voice assistant could turn on the porch light, start preheating the oven, draw the curtains, Turn on the kitchen light.

Josh is also working on using ChatGPT for media discovery in the smart home. The hitherto missing link. “If you don’t know what you want, voice control isn’t ideal,” Capecelatro said. "We built an integration with the Ava Remote that you can use to browse for content you want to watch. By adding ChatGPT to the mix, you can say, 'What are some really good shows on Netflix, maybe romcoms and Featured (this) cast. "ChatGPT can compile a list and display it on the remote control's screen." That's family movie night.

Josh’s AI upgrade isn’t live yet, and Capecelatro said the company is paying close attention to other companies’ emerging technologies in the field in case they offer better models. In addition to the fact that ChatGPT is currently very slow (the video has been edited to speed it up), there is a very real AI generation problem. (In fact, the dataset on which ChatGPT was trained ended in mid-2021. It’s worth noting that in the demo video, when Josh is asked “What shows are available to watch on Netflix?” the most recent show it lists is Debuting in 2019.) But Capecelatro says some form of generative AI voice assistant will make its way into the smart home.

Caution is absolutely necessary. No business wants a racist, homophobic, homicidal voice assistant spreading its "opinions" through their hardware into people's homes, which is a distinct possibility since generating AI is essentially regurgitating the cud without a filter content. "We were being very cautious. We could have used the ChatGPT integration immediately," Capecelatro said. “We didn’t do that. Because we didn’t want to give people really bad data. We didn’t want to lie.” Crucial. "I think Microsoft and Google went a little too far [in their search ChatBot model] and they're now seeing the consequences," Capecelatro said of recent high-profile releases that have quickly gone sideways.

It wouldn’t be a leap to assume that Google, Apple, and Amazon are all working on how to incorporate new AI language models into their voice assistants, but smart home enthusiasts have already figured out how to use Siri shortcuts for ChatGPT into their smart home. Talking to a smart speaker is much easier than typing into a web browser.

But do we really want this kind of artificial intelligence in our homes? Is our desire for a voice assistant that “just works” so great that we would be happy to have one that can try to teach my 8-year-old about quantum physics? Personally, I think a reliable, voice-controlled smart home system that knows what I mean when I say "turn off the damn lights" would be enough, rather than an omniscient intelligence managing my home.

While the promise of a naturally capable, highly intuitive voice assistant serving as the perfect household butler is very appealing, I worry that the reality may be more like Space Odyssey than Downton Abbey. But let's see if I'm proven wrong.

The above is the detailed content of Will you let ChatGPT control your smart home?. For more information, please follow other related articles on the PHP Chinese website!