If interacting with Siri has been the extent of your experience “talking” to AI, you’re in for a rude awakening. While Apple’s voice assistant often fumbles basic commands and can take an unnaturally long time to process a request, the latest and greatest AI models are moving closer and closer to natural human speech.
Case in point: Dessa, the brainchild of Jack Dorsey’s Block, is allegedly the world’s first full-duplex speech model, meaning it can listen to you and speak simultaneously, much like a real human does in the flow of normal conversation. In fact, that’s Dessa’s draw: it was trained entirely on human speech to make it as natural-sounding as possible.
This Tweet is currently unavailable. It might be loading or has been removed.
But these claims have to be heard to be appreciated, and Block’s press release announcing Dessa shows just how far AI models have come to approximating the rhythm of natural human conversation.
In the second sample audio clip, for example, a real-life human and Dessa have an extended conversation about the movies of Christopher Nolan, and though the model does flex its knowledge at certain points, supplying a movie title, for example, it does so in a natural rather than robotic way, mirroring the tone of his human interlocutor while keeping up what, to human ears, sounds like a perfectly ordinary back-and-forth about Nolan’s cinematic world.
In the third audio sample, Dessa spontaneously reacts to an unexpected prompt from the human (“Are you excited for our anniversary?”) with genuine surprise and enthusiasm, complete with the conversation markers you would expect from a human being: audible gasps, a joyful enthusiasm that verges on tears, and even a difficulty finding the right words.
The final clip nixes human involvement entirely, featuring a conversation between ChatGPT and Dessa, and the contrast between the robotic, unnatural speech patterns of the ChatGPT agent and Dessa’s prompts, interruptions, and puzzled questions perfectly illustrates how far Block has pushed its AI speech models.
Perhaps most impressive of all, Dessa seems to understand when a human speaking to it is being sarcastic, which requires a sophisticated understanding of both tone and its relationship to the content of speech that adult humans process effortlessly but that has historically stumped AI models (don’t even think of speaking to Siri in a sarcastic tone).
According to the same press release, Block’s motivation for developing Dessa is simple:
“Speech is the most natural interface to any intelligence, super or otherwise, and today’s voice AI wastes it. It flattens what we say into text and can’t keep up with human timing. AI keeps gaining power and independence, and it’s still nearly deaf to the emotion in our voices. Imagine being in serious pain while a voice AI has no idea and just keeps going. That’s the safety problem. Dessa is a step toward fixing it.”
Leave aside the question of whether recognizing a tone of urgency, fear, or pain in a voice pattern is the same as sympathizing with the emotions behind those markers. But one should take a second to appreciate the complexity of training agentic AI to pick up on them in the first place.
Wherever this leads us, Dessa is undoubtedly a major advancement in artificial intelligence and, more importantly, in bridging the gap between human communication and AI understanding.
Source: https://mashable.com/tech/block-unveils-new-ai-model-dessa-trained-entirely-on-human-speech
