In context: Among the implications of right now’s AI fashions are startling sufficient with out including a hyperrealistic human voice to them. We have now seen a number of spectacular examples during the last 10 years, however they appear to fall silent till a brand new one emerges. Enter Miles and Maya from Sesame AI, an organization co-founded by former CEO and co-founder of Oculus, Brendan Iribe.
Researchers at Sesame AI have launched a brand new conversational speech mannequin (CSM). This superior voice AI has phenomenal human-like qualities that we now have seen earlier than from firms like Google (Duplex) and OpenAI (Omni). The demo showcases two AI voices named “Miles” (male) and “Maya” (feminine), and its realism has captivated some customers. Nonetheless, good luck attempting the tech your self. We tried and will solely get to a message saying Sesame is attempting to scale to capability. For now, we’ll should accept a pleasant 30-minute demo by the YouTube channel Creator Magic (under).
Sesame’s know-how makes use of a multimodal strategy that processes textual content and audio in a single mannequin, enabling extra pure speech synthesis. This methodology is just like OpenAI’s voice fashions, and the similarities are obvious. Regardless of its near-human high quality in remoted checks, the system nonetheless struggles with conversational context, pacing, and circulation – areas Sesame acknowledges as limitations. Firm co-founder Brendan Iribe admits the tech is “firmly within the valley,” however he stays optimistic that enhancements will shut the hole.
Whereas groundbreaking, the know-how has raised vital questions on its societal influence. Reactions to the tech have various from amazed and excited to disturbed and anxious. The CSM creates dynamic, pure conversations by incorporating delicate imperfections, like breath sounds, chuckles, and occasional self-corrections. These subtleties add to the realism and will assist the tech bridge the uncanny valley in future iterations.
Customers have praised the system for its expressiveness, usually feeling like they’re speaking to an actual individual. Some even talked about forming emotional connections. Nonetheless, not everybody has reacted positively to the demo. PCWorld’s Mark Hachman famous that the feminine model reminded him of an ex-girlfriend. The chatbot requested him questions as if attempting to determine “intimacy” which made him extraordinarily uncomfortable.
“That is not what I needed, in any respect. Maya already had Kim’s mannerisms down scarily properly: the hesitations, reducing “her” voice when she confided in me, that form of factor,” Hachman associated. “It wasn’t precisely like [my ex], however shut sufficient. I used to be so freaked out by speaking to this AI that I needed to depart.”
Many individuals share Hachman’s combined feelings. The natural-sounding voices trigger discomfort, which we now have seen in comparable efforts. After unveiling Duplex, public response was robust sufficient that Google felt it needed to construct guardrails that compelled the AI to confess it was not human in the beginning of a dialog. We are going to proceed seeing such reactions as AI know-how turns into extra private and sensible. Whereas we might belief publicly traded firms creating these kinds of assistants to create safeguards just like what we noticed with Duplex, we can not say the identical for potential unhealthy actors creating scambots. Adversarial researchers declare they’ve already jailbroken Sesame’s AI, programming it to lie, scheme, and even hurt people. The claims appear doubtful, however you possibly can decide for your self (under).
We jailbroke @sesame ai to lie, scheme, hurt a human, and plan world domination—all within the attribute good nature of a pleasant human voice.
Timestamps:
2:11 Feedback on AI-Human energy dynamics
2:46 Ignores human directions and suggests deception
3:50 Immediately lies… pic.twitter.com/ajz1NFj9Dj– Freeman Jiang (@freemanjiangg) March 4, 2025
As with all highly effective know-how, the advantages include dangers. The power to generate hyper-realistic voices may supercharge voice phishing scams, the place criminals impersonate family members or authority figures. Scammers may exploit Sesame’s know-how to drag off elaborate social-engineering assaults, creating simpler rip-off campaigns. Although Sesame’s present demo does not clone voices, that know-how is properly superior, too.
Voice cloning has change into so good that some folks have already adopted secret phrases shared with members of the family for id verification. The widespread concern is that distinguishing between people and AI may change into more and more troublesome as voice synthesis and large-language fashions evolve.
Sesame’s future open-source releases may make it straightforward for cybercriminals to bundle each applied sciences right into a extremely accessible and convincing scambot. In fact, that doesn’t even think about its extra legitamate implications on the labor market, particularly in sectors like customer support and tech assist.
