Meta’s newest attempt at live, multimodal calling flopped a few times during onstage demonstrations, often aggressively. A virtual assistant embedded in a pair of smart glasses for cooking misunderstood what it was asked to say and wrongly reported what was visible in the room. A neural wristband, which would replace Android’s single-finger wheel with subtle muscle signals in the arm, never managed to pick up the call.
Whatever the option, it was excruciating to see what real-time AI can be like when it has to do rather than just vision or discourse—and do it in front of people during a live keynote.
In one demonstration, Meta’s latest smart glasses posed a Korean-inspired sauce to a Korean cook while Meta’s latest Instagram connection hovered overhead. The VFX correctly discerned two bowls and dinner table elements. However, it got bored and began to speak on its behalf.
Despite consistent questioning, the presenter did not silence the machine by putting ‘How do I start up’ at the tip of his tongue, and the machine continued to advance while incorrectly admitting they had transferred steps. The reps stumbled around for a few minutes before the portion was broken off due to poor Wi‑Fi.
Network latency and cloud reliance undermined live AI
Blaming the network is more than a trope. Live, vision-powered assistants tend to combine on-device vision with cloud models for reasoning and language, making them highly susceptible to latency and packet loss. Even a few hundred milliseconds of delay can potentially disrupt voice barge-in or desynchronize the “live AI” perception of what’s happening in the world from the AI “thought” of reality based on what’s being said.
That is why OpenAI, Apple, and Amazon have all emphasized lower-latency voice stacks and local processing for their most recent assistants. Meta’s own warning that the “Live AI” mode would only be available in short stints indicated the system’s compute and battery limitations are still quite prominent.
Neural wristband failed to answer calls during the demo
The second stumble was Meta’s neural instrumentality: a wristband that uses electromagnetic signals from the forearm to decipher micro-gestures into input. This was derived from Meta’s 2019 procurement of CTRL‑Labs, broadly detailed at a cost of between $500 million and $1 billion, and proclaimed as a “huge scientific step” in the path to imperceptibly integrated computing.
The band operated during the presentation, transforming air-writing into text. However, when it was required to answer a WhatsApp call on the glasses with a single-finger motion, the interaction fluttered—more than once. The CEO attempted again, looked upset, and eventually changed the subject. Minutes later, another gesture was used to activate a Spotify interaction, underlining the exasperating erratic behavior that affects first-generation neural interfaces: they may function effectively in a regulated series of movements and still fail in time-sensitive instructions.
Why real-time multimodal assistants are so fragile
Researchers developing EMG at organizations like Meta Reality Labs and universities have long noticed that signal quality degrades due to differences in electrode placement, skin conductivity, user movement, and even the user’s stress level—all of which spike on a keynote stage. Indeed, real-time classification requires careful calibration; a gesture easily decodable in a quiet lab can become ambiguous in a noisy RF environment filled with actively transmitting phones and cameras.
Live multimodal AI is difficult because putting an assistant in your glasses seems simple: see what I see, hear what I say, help me do the thing. In practice, it’s a high-wire act. The system must track synchronized context across audio, vision, and user intent; manage conversational interruptions; and render step-by-step guidance without data-augmenting hallucinations, and, of course, run efficiently on-device to minimize latency while also not draining the battery or overheating. The instant any link—Wi‑Fi, camera capture, speech recognition, model inference—hiccups, the illusion of fluency evaporates.
We’ve witnessed big tech demos crash previously. Google’s debut Bard stumble has become legendary, while Amazon’s more talkative Alexa required a carefully choreographed routine. However, a live fizzle in a real, physical-workflow cooking, Zoom calls—feels different, because the off-note mistake is apparent to everyone in the following audience. The assistant doesn’t just sound wrong; it appears incorrect.
Smart glasses remain niche despite ambitious roadmaps
Meta made a similar observation. Momentum is critical for the Ray‑Ban partnership, which is a fundamental element of its long-term “ambient computing” vision; the business has been continuously piling on new technologies, from hands-free photographs to multimodal AI. According to industry studies from research firms IDC and CCS Insight, “XR,” which includes smart glasses and has consolidated almost all technologies from virtual reality to augmented reality, is still a tiny niche compared to the smartphone industry.
However, the upside is tremendous if the most popular usage cases, such as messaging, navigation, translation, and coaching, appear as they might imagine. It’s a blunder since the firm isn’t on the frontier of science—it’s basic home furnishings. Barge-in for mainstream usage necessitates that systems comprehend conversation naturally. Basic instruction must link up to the camera image those systems are constructing, and neural input must succeed the very first time without fail. All are solvable science mysteries, but they’re difficult to implement in one tiny machine that you can wear all day without being conscious of it.
The benefit is that these kinds of errors frequently result in more superior design. This will entail:
- On-device treatment to minimize cloud roundtrips
- Improved power and thermal management
- EMG calibration that adapts to the user in a matter of seconds rather than a matter of minutes
The demonstrations will seem nearly magical once these three core components are in place. The “live” evaluation will be challenging AI’s promise—and its limitations.