The AGI Debate – Are We Five Years from Singularity or Chasing the Wrong Definition?

Summary

This episode of Turing Post’s “Attention” series brings together founders, researchers, and security experts from companies including OpenAI, Microsoft, Replit, Encord, Rubrik, Axiom Math, Predibase, Vectara, and others to debate the true definition of AGI and when we might achieve it. The discussion reveals a striking lack of consensus on what AGI actually means, with definitions ranging from “a super intelligent remote worker” to “a system better than any human at any topic” to the original AI research definition of “an agent that can be dropped into any unfamiliar environment and figure things out on its own.”

The experts present competing timelines and perspectives. One notable prediction suggests AGI is just 5 years away, based on the theory that once AI masters coding (expected within 2 years), it can begin improving itself, leading to rapid advancement across all domains. However, others argue that complete AGI may be impossible without understanding consciousness and the human mind, and that “practical specialized intelligence” is the more realistic near-term trajectory. The “Plate Theory” offers a visual framework: easy problems like basic math sit in the center, while hard problems like curing cancer, solving the Riemann hypothesis, and writing Nobel-worthy literature sit at the jagged edges.

The discussion also addresses significant practical concerns including workforce disruption, cybersecurity implications (AGI as an “accelerant” for both attackers and defenders), and the emerging distinction between “cheap intelligence” (raw knowledge, pattern recognition) and “expensive intelligence” (human agency, creativity, original insight, and taste). Throughout, most experts converge on a human-centric view: AGI will augment and partner with humans rather than replace them, and direction and perspective will remain fundamentally human.

Highlights

”We need to understand consciousness to get the final breakthrough”

yt-dlp --download-sections "*2:25-3:15" "https://www.youtube.com/watch?v=spWtumumh34" --force-keyframes-at-cuts --merge-output-format mp4 -o "spWtumumh34-2m25s.mp4"

“I think ultimately we need to understand the nature of the mind. They needed to understand consciousness and we need to understand the brain a lot better to be able to get the final breakthrough I think in simulating a mind and machines.” — Amjad Masad, 2:25

”Coding systems will reach that point within 2 years”

yt-dlp --download-sections "*9:49-10:40" "https://www.youtube.com/watch?v=spWtumumh34" --force-keyframes-at-cuts --merge-output-format mp4 -o "spWtumumh34-9m49s.mp4"

“We’re expecting that the coding systems will reach that point within the next 2 years. And as soon as they reach that, yeah, I would say a maximum of 3 years of them working non-stop, day and night, they will be able to figure this out across all the domains.” — Amr Awadallah, 10:13

”Cheap vs Expensive Intelligence”

yt-dlp --download-sections "*10:47-11:40" "https://www.youtube.com/watch?v=spWtumumh34" --force-keyframes-at-cuts --merge-output-format mp4 -o "spWtumumh34-10m47s.mp4"

“There will be a kind of like bifurcation of like expensive and cheap intelligence. Like expensive types of intelligence are things like human agency, creativity, original insight, taste and vision and the stuff that like feels like it’s like genius or original because it can’t be reverse engineered from the data.” — Dev Rishi, 10:47

”The Plate Theory for Superintelligence”

yt-dlp --download-sections "*15:31-16:30" "https://www.youtube.com/watch?v=spWtumumh34" --force-keyframes-at-cuts --merge-output-format mp4 -o "spWtumumh34-15m31s.mp4"

“Think about like you know the world of problems that’s like a plate and then in the middle you have the easy problems like 1 + 1 equals two print hello world right and then in the edge you have many different points maybe the north point is cure cancer okay and the west point is solve the Riemann hypothesis.” — Ulrik Hansen, 15:31

”It’s very hard for this super intelligent AI mathematician to fold laundry”

yt-dlp --download-sections "*15:04-15:35" "https://www.youtube.com/watch?v=spWtumumh34" --force-keyframes-at-cuts --merge-output-format mp4 -o "spWtumumh34-15m04s.mp4"

“I like the word super intelligence better always better than AGI cuz I don’t know really the difference they are so similar in a sense. I think super intelligence can be defined that’s domain specific. So I can be a super intelligent AI mathematician. Right? It’s very hard for this super intelligent AI mathematician to fold laundry the best.” — Ulrik Hansen, 15:04

Key Points

• True AGI Definition (0:56) - Original AI researchers defined AGI as dropping an agent into any environment where it can successfully plan, learn skills efficiently on the fly, just like humans do when learning basketball.

• Current AI Limitations (1:33) - Today’s AIs cannot generalize beyond their training; when dropped in totally new environments, they tend to be unable to do anything.

• Remote Worker vs True AGI (1:46) - Modern labs have redefined AGI as a “remote worker” at human level in certain domains, but this isn’t true AGI since it lacks generalization and efficient learning.

• Need for Consciousness Understanding (2:25) - Achieving true AGI may require understanding the nature of the mind, consciousness, and the brain much better than we currently do.

• No Skynet Risk (2:38) - One expert dismisses the Terminator-style single AI entity taking over the world as implausible, preferring to prepare for continuously improving AI tools.

• Practical Specialized Intelligence (3:42) - Rather than AGI, the near-term focus should be on practical specialized intelligence for enterprise business processes.

• Turing Test Already Passed (3:56) - If AGI is defined as passing the Turing test, we may already be in that ballpark.

• Complete AGI May Be Impossible (5:13) - Some believe complete AGI doesn’t exist because there will always be need for ground truth from human wisdom.

• Job Disruption Is Real Today (6:28) - Concerns about AGI and job displacement are real concerns with current tools, echoing patterns from the industrial revolution and PC invention.

• Developers First to Adapt (7:05) - Programmers will be among the first to grapple with changing roles; fluency with AI tools becomes more important than memorizing APIs.

• 2-5 Year Agent Adoption Timeline (7:52) - Experts estimate 2-5 years before people frequently use agents in a large portion of their professional life.

• AGI as Partner, Not Replacement (8:08) - AGI will be an accelerant and partner to humans; AI mimics human behaviors but won’t replace human creativity and dynamism.

• 5-Year ASI Prediction (9:49) - One expert predicts ASI (superintelligence better than any human at any topic) in 5 years, driven by AI mastering coding within 2 years then self-improving.

• Cheap vs Expensive Intelligence (10:47) - Intelligence bifurcates into “cheap” (raw knowledge, pattern recognition, boilerplate) and “expensive” (human agency, creativity, original insight, taste, vision).

• AGI Accelerates Cyber Attacks (12:15) - From a security perspective, AGI will accelerate both attackers (social engineering, rapid compromise) and defenders.

• Reversibility and Guardrails (12:52) - Key security challenge is understanding what AI agents did, having guardrails, and being able to reverse changes.

• National Security Implications (14:07) - AGI could allow giving very broad outcomes (vs specific outcomes for current AI), enabling new attack vectors in national security contexts.

• The Plate Theory (15:33) - Visualize problems as a plate: easy problems (1+1=2, hello world) in center; hard problems (cure cancer, Riemann hypothesis, Nobel-winning novel) at edges. AGI means enlarging the plate to all edges.

• AI for Math as Foundation (16:53) - AI for math will be the algorithmic pillar and reasoning platform for AI for science, representing the next frontier.

• Massive Productivity Gains (17:37) - Tasks taking an hour of information retrieval drudgery could be done in minutes or seconds with massively parallel AI systems.

Mentions

Companies

• Replit (0:56) - Featured company, Amjad Masad discusses AGI definition and coding
• Microsoft (4:14) - Eric Boyd discusses enterprise AI implications
• OpenAI (17:04) - Ben Goodger discusses future of AI and productivity
• Vectara (10:24) - Amr Awadallah discusses AGI as continual improvement
• Predibase (10:47) - Dev Rishi discusses cheap vs expensive intelligence
• Rubrik (12:08) - Anneka Gupta discusses cybersecurity implications
• Encord (15:04) - Ulrik Hansen discusses plate theory for superintelligence
• Axiom Math (15:30) - Carina Hong discusses AI for math as reasoning platform
• Webflow (8:08) - Linda Tong discusses AGI as partner to humans
• Block (7:02) - Alex Hancock discusses changing developer roles
• Toloka (4:53) - Olga Megorskaya discusses hybrid human-AI systems
• Marsh McLennan (4:18) - Mentioned as example of Fortune 200 company affected by AI

Products & Technologies

• AGI (Artificial General Intelligence) (0:00) - Central topic; variously defined throughout the video
• ASI (Artificial Superintelligence) (3:16) - Intelligence beyond human level, leading to singularity
• AI Agents (7:52) - Autonomous AI systems expected to be widely adopted in 2-5 years
• Turing Test (3:56) - Historical benchmark for AI; potentially already passed
• AI for Math (16:53) - Predicted to be the algorithmic pillar for AI for science

People

• Amjad Masad (0:56) - Replit CEO, discusses true AGI definition and consciousness requirements
• Eric Boyd (3:36) - Microsoft, focuses on practical specialized intelligence
• Olga Megorskaya (4:53) - Toloka, believes complete AGI requires human wisdom
• Linda Tong (8:08) - Webflow, views AGI as accelerant and partner to humans
• Amr Awadallah (8:57) - Vectara, predicts AGI/ASI in 5 years based on coding breakthrough
• Dev Rishi (10:47) - Predibase, introduces cheap vs expensive intelligence framework
• Anneka Gupta (12:04) - Rubrik, discusses AGI cybersecurity implications
• Ulrik Hansen (15:02) - Encord, prefers “superintelligence” term, introduces plate analogy
• Carina Hong (15:30) - Axiom Math, bets on AI for math as reasoning platform
• Ben Goodger (17:05) - OpenAI, excited about massive productivity gains
• Alex Hancock (7:02) - Block, discusses developer role changes

Surprising Quotes

“I think ultimately we need to understand the nature of the mind. They needed to understand consciousness and we need to understand the brain a lot better to be able to get the final breakthrough I think in simulating a mind and machines.” — 2:25

“We’re expecting that the coding systems will reach that point within the next 2 years. And as soon as they reach that, yeah, I would say a maximum of 3 years of them working non-stop, day and night, they will be able to figure this out across all the domains.” — 10:13

“It’s very hard for this super intelligent AI mathematician to fold laundry the best.” — 15:20

“There’s some companies that are interviewing for developers and trying to prevent them from using AI in the interviews and I was talking to another company was like that’s a totally wrong approach. They should be using the tool.” — 7:23

“I personally don’t think that there is such a thing as complete AGI. There will always be some need in the ground truth which you cannot get from anywhere apart from the human wisdom.” — 5:13

Transcript

0:00 What is true definition of AGI? Is it a super intelligent remote worker that can code better than any human? Or is it something deeper like an agent that you can drop into an unfamiliar domain and it will be able to figure out everything by itself? The rules, the tools, the goals without any step-by-step manual.

0:21 In this episode, we explore the spectrum of artificial general intelligence delivered to you by actual builders. We have experts debating if we are 5 years away from the singularity or if complete AGI is actually impossible without human wisdom and human supervision. We talk about the massive shift to the workforce. The risks of cyber security and if we’re building a tool or replacement. If you want to know where the future is heading, you need to listen to this. Welcome to Attention.

0:56 The true definition of AGI that like the original researchers of AI have talked about is being able to drop an agent in any environment, an agent to be able to be successful, plan and be successful, learn skills efficiently on the fly. That’s how humans work. Obviously, we have different degrees of abilities, but if you drop me in a basketball game and I don’t know anything about basketball, I might observe people a little bit and then deduce, you know, how I’m going to be able to do things or someone can teach me for an hour and then I can acquire the skill. I’m not going to be great, but I can start learning it.

1:33 Whereas AIs right now, when you drop them in a totally new environment, they tend to not be able to do anything. And so because they’re function of their training, we haven’t seen AIs that really generalize beyond their training. Now like modern startups, modern labs change the definition of AI. They call it like a remote worker. We’re going to create a remote worker that is like human level at certain domains.

1:57 Well, okay, you know, that’s of course that’s going to happen. I mean, I’m talking about it happening in coding. And then if you get data for most knowledge work out there, yes, you’re going to be able to create a general purpose remote worker, is that AGI? I don’t think so because it’s not going to be generalizing and efficiently learning and those are the big breakthroughs we need to get to AGI. And I feel like yes, we’re going to get more breakthroughs and we’re going to get closer and closer, but I think ultimately we need to understand the nature of the mind. They needed to understand consciousness and we need to understand the brain a lot better to be able to get the final breakthrough I think in simulating a mind and machines.

2:38 Yeah. I haven’t noticed any doomism in your interviews or anything like this about AGI. No, I think there’s big risk in AI like any other technology, but I don’t think there’s a Skynet type risk like from that movie Terminator where you know there’s a single AI entity that’s taking over the world. I don’t think that’s plausible. So if you achieve AGI it becomes so good so intelligent that it can construct its own tools and wouldn’t need us.

3:07 And so part of the reason why I don’t like to sit down and try to predict AGI because there are certain outcomes that we don’t have control over. So if we’re actually going to achieve AGI and then ASI which is super intelligence then it doesn’t matter what we predict because then the world is going to rapidly change. We’re going to reach the point that’s called the singularity. So, I’d rather prepare for what I think the more likely world is, is AI that is continuously improving and becoming more autonomous, but it’s generally still a tool.

3:38 I think AGI is something that’s far from the world that I see. In honesty, the world that I see tends to be like rather than artificial general intelligence, it’s like practical specialized intelligence. And so, I think that AGI often times think about in the research labs, you know, folks that will come up with good definitions. If we take a definition of AGI is like can a model pass a Turing test, I would suggest that we’re probably, you know, in that ballpark already.

4:03 But what’s the practical implication of that? I don’t spend too much of my time thinking about the Terminator style scenarios, but I do spend a lot of my time thinking about what does having this generalized intelligence look like when you actually have businesses processes like a company Fortune 200 like Marsh McLennan or you know another organization that I mentioned earlier like Checker or you know any of these other companies that have a lot of productivity that they’ve unlocked via business practices over the past previous decades and that productivity is about to take a step function change and increase to me that’s really kind of like the interesting area for where this is going to go.

4:37 There’s probably some deeper philosophical questions about what will happen like over you know 5 10 20 year period. We found it pretty hard to even predict what’s going to happen 18 months from now in AI. And so I think that’s really where a lot of my focus has been is what’s going to be the practical implication on both enterprise and consumer.

4:53 I prefer to be more down to earth looking at the what we can do from the engineering point of view. I definitely think that the co-agency and hybrid systems where humans and AI are collaborating are the next step. Whether it is the last step or not, I honestly don’t know. I have no idea. I personally don’t think that there is such a thing as complete AGI. There will always be some need in the ground truth which you cannot get from anywhere apart from the human wisdom.

5:22 That’s what I love talking to practitioners because basically every time I ask a question about AGI, people say well I try to look more practical on that and you create this whole thing you really know like how it works practically. So I think it’s very helpful to have this narrative out. I think this is maybe both good and bad because well we are developing step by step and every step seems quite small and maybe within those small steps there is kind of a risk to the bigger picture.

5:55 So AGI, you know, is a topic people sort of like it’s such a great science fiction topic and ASI for superhuman intelligence and all those things and it becomes almost a philosophical conversation with lots of people of like you know what is how do we know that we’ve actually reached it over a nice glass of wine and indulge in a nice conversation about that. I tend to be just much more pragmatically focused of like we have this AI that’s you know is it AGI? I don’t know. I don’t think so. It’s pretty helpful though. It’s very useful tool. I really like being able to use it and so I very much focus on that.

6:28 There will be, you know, the things that people worry about with AGI in terms of the way that people’s jobs change and potential disruption to jobs that come with that. Those are very real concerns, but those are real concerns with the tools that we have today. And they’ve been very real concerns with, you know, back to the industrial revolution, with the invention of the PC, and like the number of jobs that have changed. They change. And that’s hard for societies to sort of work through, but they do work through them. When I think about AGI, I hear the debate really being about around, hey, the world as we know it is going to work differently. What does that mean for me? That’s the important question for everyone to work through.

7:02 Are we close to understanding that or just more questions? It’s interesting how developers, programmers are going to be one of the first places to grapple with this that the role is going to change. And so you’re going to need to, you know, the ability to sort of remember some arcane API or quickly search for it is less important. The ability to use the AI tool is suddenly much more important.

7:23 It’s funny the calculator analogy like there’s some companies that are interviewing for developers and trying to prevent them from using you know AI in the interviews and I was talking to another company was like that’s a totally wrong approach. They should be using the tool. I want them maybe using the most advanced tools and showing that they’re comfortable and fluent in it cuz that’s where the developer world is moving and they need to be good at that. They’re going to have to work through like how does the developer job change but being comfortable and fluent with the tools is going to be a key part of that.

7:52 How long will it take do you think to be for everyone to be more productive more knowledgeable about agents and for agents to work properly like from two to five years to get to a place where people are very frequently using agents in a large portion of their professional life. I think that’s probably the right timeline. Yeah.

8:08 My personal belief is that with AGI it is ultimately an accelerant and a partner to humans. The thing that makes us unique as a species is we are inherently creative and opinionated and we are so dynamic and that we constantly change. You know, the way that AI has been built right now is that it is meant to reflect human intelligence and synthesize it at levels of capacity well beyond what we’ve ever seen before. And with AGI, it’s the ability to iterate off of that and actually generate and create. And I think it’ll mimic some of those human behaviors, but I don’t fundamentally believe that it will replace them.

8:42 And so if I then apply that back down at a software level, AGI will fundamentally change the experience of like how we would need to interact with software to accomplish a task and it could potentially accomplish the majority of that task. But I believe that it will still need some amount of direction or perspective from a human lens to drive it.

8:59 We all as a community have been struggling now to define AGI and the definition itself has been changing ever since it came to the world. In my opinion, AGI is achieving a system that can be better than any human at any topic. That is AGI, which is a very very hard goal to achieve. We don’t have that today. We have these systems that are better than us in some domains, but only better than the average of us. Not better like even in coding for example, we still are not better than the best human coder. We’re still behind them. I think AI is now like 89th or 90th something like that across the world when you rank them with amazing coders that we have.

9:38 So imagine now doing that for coding, for legal and for medicine and for chemistry and for quantum physics and for architecture and for poetry and for art and for dancing. Now that is ASI in my opinion and I still think that’s 5 years out. Just 5 years. Okay.

9:54 The reason why just 5 years is because of coding. So everybody right now the big labs they’re very focused on the coding problem to get the coding systems to be as good or better than the best human at coding. And once that is achieved, then now the AI system can start working on itself. And once the AI system can start working on itself, then that will speed up timeline significantly. We’re expecting that the coding systems will reach that point within the next 2 years. And as soon as they reach that, yeah, I would say a maximum of 3 years of them working non-stop, day and night, they will be able to figure this out across all domains.

10:24 AGI discussion is an interesting one. And I think I forget who said this, but I liked the description of it that I heard that it’s like a process that’s going to be continual improvement and it’s not going to be like a before and after, right? These things just continuing to get more and more and more capable and doing more for us. And at a certain point, you’re going to look at them and you’re going to like we already do today realize that they’re incredibly powerful and incredibly intelligent.

10:47 At the end of the day, like treating intelligence as a single construct is difficult because there’s like different levels of intelligence, right? There’s like the cheap types of intelligence which is just like raw knowledge and facts where like the models can like spit out infinite versions of something at like near zero cost. There’s like pattern recognition or like common things like writing boilerplate code, summarizing documents and like generating standard designs. All that stuff is like quite not easy to do. It’s been a lot of time development.

11:17 But I think there will be a kind of like bifurcation of like expensive and cheap intelligence. Like expensive types of intelligence are things like human agency, creativity, original insight, taste and vision and the stuff that like feels like it’s like genius or original because it can’t be reverse engineered from the data. And so I think the kind of going back to the analogy around having AI systems that evolve in like kind of like doing both of those things like the ones where like we can do it for cheap that’ll be less need for like a human in the loop and factchecking like the ones that the things that are like will be more difficult or require like more expensive types of intelligence like that’s really where like human agency tastes and all these things that you can’t really explain necessarily like in data will become like more important. So like the main constraint now really is to like embed those models and like kind of wrapping them into products that can like solve real problems.

12:08 What do I feel about AGI? My sense is it’s an inevitability. It’s a matter of when, not if it’s going to happen. From a security landscape, it’s going to be yet another tool that attackers are using already. Attackers use AI to have highly personalized and realistic social engineering that’s leading to more compromised identities. Certainly AGI will help that, but I think it’s going to completely change even once attackers break in how quickly they’re able to compromise an organization and their data. And so just all of this is going to mean that the timeline for what happens and how quickly people get breached and the impact of that is much higher.

12:49 Even whether it’s a cyber attack or unwanted AI agent actions, how do we think about reversibility? So I think like reversibility, it only makes the problem more acute. The solution to the problem is still probably not that different which is being able to really I talked earlier about how do you understand this whole timeline of what an agent actually did having those guard rails in place if you understand those you have the guard rails yes you’re going to miss stuff some stuff in the guardrails but you can understand you can look at all the actions and rank them essentially by their risk and AGI will help with that to make understanding of that risk like a lot better a lot fewer false positives and false negatives and really elevating the things that are the highest risk actions and then it’s about how do you have the integrations into all the different technologies that allow you to quickly understand what changed and go into that system and reverse that change.

13:39 Do you think this system AGI system will act on itself whatever decision it is or it still will be hackers who decides? Well, I mean I think there’s it’s a good question. I don’t know the answer to that. I think it’s a great question. I mean I think AGI some version of it will be used by humans that are giving a very like very broad outcome whereas today with AI agents you have to give a very specific outcome right with AGI you could give a very broad outcome so you could have from a national security perspective you could have a foreign government that’s trying to wage war on another country give very broad outcomes for what it’s looking for and AGI will just go figure out how to execute on those outcomes.

14:21 I do think that it’s like again it’s an accelerant of what’s already happening. There’ll probably be new vectors that AGI discovers of hey this is like an interesting vector of an attack or on the defense side as well. Here’s like an interesting vector of things that we can defend against both proactively and reactively. It’s just accelerating the cat and mouse game that already exists. Will we have AGI where it’s like not even human directed? Now you’re starting to talk about total science fiction world which like is certainly possible but then we might have a lot bigger problems on our hands than what we’re talking about here today. It’s hard to predict that kind of future which would probably mean a fair amount of social upheaval and a lot of different things that monumentally affect our day-to-day life.

15:04 I like the word super intelligence better always better than AGI cuz I don’t know really the difference they are so similar in a sense. I think super intelligence can be defined that’s domain specific. So I can be a super intelligent AI mathematician. Right? It’s very hard for this super intelligent AI mathematician to fold laundry the best. I think it’s not quite general in a way. The thing with AGI is I don’t quite know what it means.

15:31 And I think what it means the difference could be that if you think about like you know the world of problems that’s like a plate and then in the middle you have the easy problems like 1 + 1 equals two print hello world right and then in the edge you have many different points maybe the north point is cure cancer okay and the west point is solve the Riemann hypothesis the south point can be write a novel that’s going to win the Nobel prize literature or you know be the generational novel fiction and then you know you can have the east one as something even better than that right like so something that can like I don’t know like the sort of like the ultimate question of physical intelligence.

16:21 The concept seems to be that from the center you enlarge the radius like slowly right and like you’re betting on the system that does really well in writing the Shakespeare equivalent work that can also solve the Riemann hypothesis and cure cancer. That to me feels like AGI, right? Like the plate just keep being enlarged to the edge. I think for me super intelligent is hey I pick a point in the middle right and then I pick the goal and I just go there like I just go there one line. I bet on the specific model. I also bet on AI for math being the algorithmic pillar the reasoning platform of AI for science and that’s going to be the future. I think that’s going to be the next frontier in AI. AI for math, AI for science.

17:04 I don’t want to sort of get out ahead or differ from like other perspectives at OpenAI on this, but what I’d say I’m excited to work towards is something where this technology can take really meaningful, valuable action for you and help you sort of sift through all of the challenges and opportunities in your life and get stuff done for you in a really meaningful way. So, that’s the kind of product that I’m interested in building.

17:27 I think in the future we will see the computing platform that supports all of this evolve in massive ways where you know the these technologies can do things way faster than you can and what that means will be that you can do much more. I’m very bullish on like productivity increasing massively as a result. You know there’s certain things that I do by hand today where I might spend an hour or more doing some like information retrieval drudgery whereas I can imagine a system that is like massively parallel doing that in minutes or seconds even. And so I’m excited about that. Whatever definition that fits, I’m not sure, but that’s something that I’m excited to see happen and I’m excited to build.

18:03 My personal take on AGI is simple. It will augment us and will work alongside us as a partner, but direction and perspective will remain fundamentally human. Thank you for watching again and please leave your comments and your definitions in the comment section. Like the video, repost it and subscribe.