Quick summary:
- AI hallucination rates still run 5 to 20%, and the models are growing more confident in their wrong answers
- We passed the original Turing test years ago, but the AI community keeps moving the goalposts on AGI
- World models — foundation models for physical space — could produce applications more valuable than anything LLMs have delivered
Michael Weiss runs North America's largest AI conference. He still won't let AI build his guest lists unsupervised.
Every year, his team organizes 50 curated events at Ai4 and needs to match 10,000 attendees against 50 different personas. The AI understands the assignment. It does the work. Then the spot checks come back with a 10% to 15% error rate. For a task where his team wants perfection, a human needs to stay in the loop.
That blend of conviction and skepticism runs through Weiss's conversation with Greg Matusky on "The Disruption Is Now." Weiss co-founded Ai4 in 2018, back when it drew 300 people to a half-day gathering in a small Brooklyn hotel. This year's show features Geoffrey Hinton and Fei-Fei Li.
Weiss and Matusky talk about why agents still need human backup, whether we've already reached AGI, and why communicators may be sitting in the best seat in the house.
Watch now:
Key takeaways
A failed World's Fair revealed that AI would power everything else
Before Ai4, Weiss spent three years trying to organize a modern World's Fair — a 100 million person, six-month festival in the tradition of Flushing Meadows in 1964. His events covered the future of cities, food, transportation, and space. The project ran out of money in 2018.
But it left him with one insight. Every speaker across every sector kept describing the same engine behind their work: data-driven insights and machine learning.
"It was just kind of obvious to me that AI was becoming this meta technology that every other technology and non-technology sector was gonna rely on," Weiss said.
So he pivoted. Instead of consumer-facing festivals, Ai4 would focus on industry applications of AI. Four years before ChatGPT, he had already placed his bet on the technology that would swallow the rest.
Agents work today, but ironing out their errors takes elbow grease
Weiss calls this the agent boom, and he expects agentic tools to dominate this year's conference across every industry. He also warns that hallucination rates still run anywhere from 5 to 20%, depending on the model.
Worse, the mistakes are getting harder to spot. "The models seem to keep getting more confident in their hallucinations," Weiss said. "They'll tell you XYZ response, which sounds very factual, but it just isn't."
His attendee-matching test proves the point, and a friend at a Fortune 500 company told him a similar story after building a chatbot for roughly 100,000 partners.
"It's easy to make the first pass at the AI or the agentic workflow, but to really iron out those errors, it takes a lot of elbow grease," Weiss said. He hopes the manual cleanup becomes unnecessary within a year. The AI labs haven't solved it yet.
A 10% error rate makes PR a perfect fit for AI
When Matusky pushed back, arguing that feeding AI deep client data had dropped his hallucination rate well below 15%, Weiss offered a sharper observation. Communications work is subjective. Give 10 writers the same fact set and you get 10 different leads, all of them potentially good.
That subjectivity is a superpower in the age of AI. "You're in a perfect business for AI because a 10% error rate, you literally don't even notice it and it doesn't matter," Weiss told Matusky.
Compare that to Weiss's persona-matching problem, where every wrong invite is a binary, countable error. Fields built on judgment and voice absorb AI's imperfections. Fields built on precision expose them.
Matusky added that the real work is collecting data, such as transcribing every client call and loading every news release, pitch, and transcript into the knowledge base. "It's really in that data. And for professionals, that's where our time should be placed," he said.
We already passed the Turing test, and nobody threw a party
Alan Turing proposed the original test in the 1950s: If a human talks to a computer and can't tell whether it's a machine or a person, AI has arrived. Weiss says that milestone is behind us.
"According to Alan Turing's definition of AI, we've hit it," he said. "We've created conversational humans in machines, which is crazy. But according to the moving definition, we're never gonna freaking get there."
The goalposts keep shifting, from Turing's test to the AGI and superintelligence framing popularized by Nick Bostrom. Weiss finds the debate less interesting than the trajectory. Today's frontier models haven't even been trained on an order of magnitude more compute than their predecessors. Those models arrive in 2027 or early 2028, and he expects them to be "way crazier" than anything available now.
World models could out-value everything LLMs have built
The next race Weiss is watching has nothing to do with chatbots. Companies led by AI pioneers — including Fei-Fei Li's venture and Yann LeCun's new startup — are trying to build a foundation model for physical space. Think of what LLMs did for text and digital tasks, applied to the physical world.
No one has cracked it yet. When someone does, Weiss expects an explosion.
"Once somebody nails that, we're gonna see I think as many, if not more, new applications of AI come out," he said. "And I think some of them will be actually a lot more valuable than the LLM applications."
As for LeCun's argument that LLMs are the wrong path entirely, Weiss stays diplomatic. LeCun has always leaned contrarian, he noted, and "the path he's pushing on could be good and we might not need it."
Key moments
- How a failed World's Fair became North America's largest AI conference (1:09)
- The agent boom and the race to build world models for physical space (3:30)
- Where Weiss lands on Yann LeCun's contrarian bet against LLMs (6:42)
- The belief about intelligence that fueled a 2018 bet on AI (8:35)
- What separates real machine intelligence from rules-based code (10:17)
- Whether the Mythos hype was publicity or genuine concern (13:06)
- The original Turing test and why Weiss says we passed it (14:11)
- Hallucination rates of 5 to 20% and the confidence problem (16:06)
- The 10,000-attendee test that AI keeps failing (17:04)
- Why data quality, not better models, cuts hallucinations (18:44)
- Why a 10% error rate makes PR a perfect business for AI (23:18)
- Why human experiences will gain value as AI advances (24:19)
Q&A with Michael Weiss, co-founder of Ai4
Q: Why did you bet on AI back in 2018, before ChatGPT?
A: "My high level view that gave me confidence back then to focus on AI was the simple fact that human intelligence has been our most valuable commodity for a long time. And human intelligence is the thing that has built our civilization that we find meaningful and valuable ... It was really just my belief in intelligence and the value of intelligence. And then the fact that we started seeing computers actually exhibit intelligence."
Q: How do you define intelligence in a machine?
A: "In the context of AI at least, intelligence is a computer that can encounter a novel experience, a piece of data it's never seen before, and actually make a decision based on that experience ... With AI, we can literally teach a computer to have a representation of the world. You can show that AI a piece of text it has never seen and it will actually respond to it thoughtfully."
Q: What was the original test for whether AI had arrived?
A: "His original Turing test was simply if a human talks to a computer and the human can't tell whether the computer is a machine or a human, that passes the Turing test. And in his mind back then, that would mean AI has arrived. And I think we've passed that. We've totally passed that. For at least the vast, vast majority of humans."
Q: Was the hype around Mythos justified or just publicity?
A: "It's both. In all interesting news, there's multiple truths ... I think there was some truth there because it seems like a lot of the organizations that were looking for bugs with Mythos were in fact able to find vulnerabilities that they didn't know were there and probably wouldn't have found otherwise."
Q: When should we expect the next big leap in model capability?
A: "The models we have now ... are not trained on a ten X, an order of magnitude more compute than the previous models, but those models will come I think either 2027 or early 2028. And I think those models are gonna be way crazier."
