What AI Legends Talked About 15 Years Ago | 3 Wow and 1 Promise

Summary

This historical episode of “3 Wow and 1 Promise” takes viewers back 15 years to explore three pivotal moments that shaped AI as we know it today. The host examines archival footage of Jensen Huang, Fei-Fei Li, and Demis Hassabis when they were “much more available” and Jensen “did not yet have his leather jacket.”

The first wow features Jensen Huang at Nvidia’s very first GPU Technology Conference (GTC) in 2009, where he introduced the revolutionary idea of using GPUs for general-purpose computing. Jensen demonstrated “CEO math” showing how parallel computing could speed up AI tasks by 200x, and unveiled the Fermi GPU with 3 billion transistors. The second wow covers Fei-Fei Li at Stanford pioneering ImageNet - a massive labeled image database that nobody believed in when she proposed it in 2006. By 2012, ImageNet powered AlexNet’s breakthrough, igniting the deep learning revolution. The third wow shows Demis Hassabis at the 2010 Singularity Summit discussing AGI five years before OpenAI even existed, advocating for combining machine learning with systems neuroscience.

The “promise” segment delivers a twist: Jensen Huang’s 2009 commencement speech where he shares that by all conventional wisdom, Nvidia “shouldn’t even be here today” - they were “dead last among many” PC graphics companies. Their success came from being “young, naive, idealistic” and challenging conventional wisdom with “the same innocence.”

Highlights

”Never in history has there been a GPU developers conference”

yt-dlp --download-sections "*2:25-2:55" "https://www.youtube.com/watch?v=BQMiT4fcdM4" --force-keyframes-at-cuts --merge-output-format mp4 -o "BQMiT4fcdM4-2m25s.mp4"

“This is the first conference for GPUs. Never in the history of the computer industry has there been a developers conference for programming GPUs. And look how large it is.” — Jensen Huang, 2009, 2:25

”200 times speedup and it did no harm”

yt-dlp --download-sections "*4:33-5:10" "https://www.youtube.com/watch?v=BQMiT4fcdM4" --force-keyframes-at-cuts --merge-output-format mp4 -o "BQMiT4fcdM4-4m33s.mp4"

“We saw a speedup for parallel applications of nearly 200 times - between 100 to 200 times. An enormous feat. And yet on mostly sequential programs, it did no harm. It did no harm. And so the most important thing in creating a new architecture is to make sure number one it does no harm.” — Jensen Huang, 2009, 4:33

”Even Google doesn’t have it”

yt-dlp --download-sections "*9:15-9:35" "https://www.youtube.com/watch?v=BQMiT4fcdM4" --force-keyframes-at-cuts --merge-output-format mp4 -o "BQMiT4fcdM4-9m15s.mp4"

“I’m very proud to report that at Stanford Vision Lab, we are the first one to have ever built a visual recognition algorithm that recognizes more than 20,000 object classes. Even Google doesn’t have it.” — Fei-Fei Li, 2012, 9:15

”5 years before OpenAI was even thought of”

yt-dlp --download-sections "*11:25-12:00" "https://www.youtube.com/watch?v=BQMiT4fcdM4" --force-keyframes-at-cuts --merge-output-format mp4 -o "BQMiT4fcdM4-11m25s.mp4"

“He talks about AGI. He calls it AGI. It was 5 years before OpenAI was even thought of. Isn’t that crazy? Deep Mind’s early work laid the foundation for later triumphs like AlphaGo, which beat the world’s best Go player in 2016 and blew everyone’s mind.” — Ksenia Se, 11:25

”We didn’t know that it was impossible”

yt-dlp --download-sections "*13:57-14:35" "https://www.youtube.com/watch?v=BQMiT4fcdM4" --force-keyframes-at-cuts --merge-output-format mp4 -o "BQMiT4fcdM4-13m57s.mp4"

“We were young, naive, idealistic. We didn’t know what we didn’t know about building the industry and about building our company. We didn’t know that it was impossible. We just imagined how great it would be if computers would do what we imagined. Our fresh and innocent perspective, unencumbered by conventional wisdom, allowed us to see what 30, 40, 50 other companies could not and did not.” — Jensen Huang, 2009, 13:57

Key Points

• Jensen without leather jacket (1:10) - Looking at AI leaders when they were more available, before they became rock stars
• Same ideas decades ago (1:30) - When people say AGI is coming next year, these clips show leaders talked about same ideas decades ago
• 50 years still unsolved (1:44) - Even after 50 years, so much is still unsolved - we have a long road ahead
• GTC 2009 (2:00) - Nvidia’s very first GPU technology conference, first ever developers conference for GPUs
• GPUs for general computing (2:20) - Jensen’s gamechanging idea: using GPUs not just for gaming but for general purpose computing
• CEO math (2:50) - Jensen’s simple explanation of parallel computing speedup potential
• 200x speedup (4:33) - Parallel applications saw 100-200x speedup without harming sequential programs
• Fermi GPU (5:32) - “A supercomputer with the soul of a GPU” - 3 billion transistors
• 1,500 surprise developers (5:48) - Nvidia had to close registration 2 weeks early; didn’t expect the turnout
• Fei-Fei Li and ImageNet (6:52) - Teaching computers to see at Stanford; proposed idea in 2006 when no one believed it
• AlexNet 2012 (8:52) - GPU-trained neural network crushed ImageNet competition, igniting deep learning
• 20,000 object classes (9:15) - Stanford built first algorithm to recognize 20,000+ object classes - more than Google
• Demis at Singularity Summit 2010 (10:13) - Discussing AGI and hybrid neuroscience-ML approach when founding DeepMind
• AGI before OpenAI (11:25) - Demis talked about AGI 5 years before OpenAI was even conceived
• AlphaGo foundation (11:40) - DeepMind’s early work laid foundation for beating world’s best Go player in 2016
• Nvidia should be dead (13:00) - Jensen admits all conventional wisdom said Nvidia shouldn’t exist - they were last among many
• Naive idealism (13:57) - They didn’t know it was impossible; saw what 30-50 other companies couldn’t see

Mentions

Companies

• Nvidia (2:00) - Hosted first GTC in 2009, now backbone of AI revolution
• DeepMind (10:03) - Founded by Demis Hassabis in 2010, later became Google DeepMind
• Stanford Vision Lab (9:15) - Where Fei-Fei Li pioneered ImageNet

Products & Technologies

• CUDA (2:42) - Nvidia’s platform for GPU parallel computing
• Fermi GPU (5:32) - 3 billion transistor GPU designed for graphics and AI workloads
• ImageNet (7:58) - Massive labeled image database that fueled deep learning
• AlexNet (8:52) - GPU-trained neural network that won ImageNet 2012 competition
• AlphaGo (11:40) - DeepMind system that beat world Go champion in 2016

People

• Jensen Huang (1:55) - Nvidia CEO, introduced GPU revolution for AI
• Fei-Fei Li (6:52) - “Godmother of AI”, created ImageNet at Stanford
• Demis Hassabis (9:52) - DeepMind founder, was chasing AGI 5 years before OpenAI existed

Surprising Quotes

“Suppose over time I gain access to 10 times the number of transistors. I’m going to use those 10 times transistors in parallel cores - 500 cores where each of these little tiny parallel cores is equivalent to one very very well-designed CPU.” — Jensen Huang, 2009, 3:00

“If you were a scientist designing Wall-E, you’re likely to be an artificial intelligence researcher and you’ll be empowering Wall-E with all kinds of functionalities.” — Fei-Fei Li, 2012, 7:42

“Where we know how to build a component for an AGI system, let’s use the state-of-the-art algorithms. Where we don’t know how to build a component, we should look at systems neuroscience for ideas for potential solutions. In parallel, not one or the other.” — Demis Hassabis, 2010, 10:28

“To this day, we challenge conventional wisdom with the same innocence. I wish you my third wish - to see the world like a child.” — Jensen Huang, 2009, 14:26

Transcript

0:00 Hello everyone and welcome to Three Wow and One Promise in the AI World. Today we’re going to make a thrilling journey to a time when artificial intelligence was on the cusp of the revolution. It was just a glimpse of the scale that became possible 15 years later.

0:25 We will explore three pivotal moments that shaped AI as we know it today, featuring visionaries that I’ve been following for years: Jensen Huang, Fei-Fei Li, and Demis Hassabis.

0:40 You know, it’s really, really hard to get interviews with top AI CEOs and researchers these days, especially the famous ones. Some of them travel like rock stars with conference agents and speaker tour rides and whole agencies behind that. And they ask for a lot of money to show up, some of them. Crazy times.

1:00 But today I’m going to show you the other times. The times when the current rock stars were much more available. The times when Jensen Huang did not yet have his leather jacket.

1:14 My goal here is simple. First, to look at these brilliant young excited people - excited about what they were building 15 years ago and they still are excited about now. But also I want to show you this long relentless continuity of their work.

1:30 When people say “next year it all will be solved, super intelligence will be here, AGI will be here” - just watch these clips. These folks were talking about the same core ideas decades ago. There’s a lot of energy in the field. There’s a lot of vision. And even after 50 years, so much is still unsolved. We have a long and exciting road ahead of us.

1:54 And to the first Wow. Jensen Huang and the GPU. Our first Wow moment takes us to 2009 where Jensen Huang, CEO of Nvidia, was igniting a revolution in computing. At Nvidia’s very first GPU technology conference, now famous as GTC, Jensen introduced a gamechanging idea - using GPUs, graphics processing units, not just for gaming, but for general purpose computing. That was huge.

2:25 “This is the first conference for GPUs. Never in the history of the computer industry has there been a developers conference for programming GPUs. And look how large it is.”

2:40 Jensen’s vision was bold. He explained how GPUs powered by Nvidia’s CUDA platform could handle massive parallel computations perfect for tasks like machine learning and AI. In his talk, he breaks it down with what he jokingly calls “CEO math.”

3:00 “Suppose over time I gain access to 10 times the number of transistors. That’s my first problem with one CPU. And I’m going to use those 10 times transistors in parallel cores - 500 cores where each of these little tiny parallel cores is equivalent to one very, very well-designed CPU.”

3:25 “Now that little tiny parallel core has a deficiency. It runs serial code badly. So we make it five times more badly. And so that 1 second becomes five. But for parallel code, it’s very easy for the parallel processor to accelerate. And let’s say that I simply accelerate that 500 times.”

3:50 “What you’ll discover is that the parallel application is in fact 40 times faster.”

4:15 This shift to parallel computing was a gamechanger. Jensen showed that combining one powerful CPU with hundreds of GPU cores could speed up parallel tasks like AI algorithms by nearly 200 times at that moment without slowing down traditional programs.

4:33 “We saw a speedup. What’s amazing is we saw a speedup here for parallel applications of nearly 200 times - between 100 to 200 times. An enormous feat. And yet on mostly sequential programs it did no harm. It did no harm.”

4:58 “And so the most important thing in creating a new architecture is to make sure number one it does no harm. That everything that you used to run runs better. But for new applications it has an enormous opportunity to take you to a new space.”

5:15 “And now over time more and more people will realize this architecture exists and they will adapt their applications to take advantage of all the resources inside the computer, eliminating more and more of the sequential dependencies inside the program. As a result, speeds will continue to enhance. That is the essence of GPU computing.”

5:32 And then the big reveal - the Fermi GPU, a supercomputer with the soul of a GPU. At that moment everything was just so new. This whole conference was solely for developers who were working on GPUs and it gained enormous success. Nvidia did not expect 1,500 developers to show up. They had to close the registration 2 weeks before the event. That was crazy.

5:57 “Fermi is an absolute, absolute powerhouse and we call it a supercomputer with the soul of a GPU. Instead of a GPU that has been extended for general purpose computing, we took a massive investment to start from the ground up - a brand new architecture which is designed to be a computer first, however to treat computer graphics and parallel computing as equal faces.”

6:33 Fermi with its 3 billion transistors was designed for both graphics and AI workloads, setting the stage for the deep learning boom. This was the moment GPUs became the backbone of modern AI and Jensen Huang saw it coming so many years ago.

6:52 The second wow is about the godmother of AI - Fei-Fei Li and the ImageNet moment. So let’s zoom to Stanford University where Dr. Fei-Fei Li was tackling a different challenge: teaching computers to see.

7:10 In the mid-2000s, Fei-Fei was pioneering computer vision, a field she described as making computers understand images like humans do.

7:20 “I’m going to share with you some of the exciting research and the field of computer vision in general. So let me ask you a question. How many of you think you know what the word artificial intelligence means? Raise your hand. Great. Don’t worry if you don’t. Here’s a crash course for you. I hope you’ve seen the movie Wall-E.”

7:48 “If you were a scientist designing Wall-E, you’re likely to be an artificial intelligence researcher and you’ll be empowering Wall-E with all kinds of functionalities.”

7:58 Her big wow moment - ImageNet, a massive database of labeled images that became the fuel for training AI to recognize objects. She came up with the idea in 2006. At that moment that was a completely crazy idea. No one believed in it.

8:15 But her team showed computers thousands of photos teaching them to identify everything from starfish to cats. “The computer knows what the event is - it’s a rowing event. Where it’s taking place - the computer tells you it’s a lake. And who are the objects involved in this picture? The computer would label: trees, athletes, rowing boat and water. So this is automatically generated by our computer vision algorithm.”

8:52 By 2012, ImageNet powered a breakthrough when a GPU-trained neural network, AlexNet, crushed the ImageNet competition, recognizing objects with unprecedented accuracy. This was the spark that ignited deep learning and it wouldn’t have happened without Fei-Fei’s vision and those GPUs Jensen Huang was championing.

9:15 “I’m very proud to report that at Stanford Vision Lab, we are the first one to have ever built a visual recognition algorithm that recognizes more than 20,000 object classes. Even Google doesn’t have it.”

9:33 Fei-Fei’s work showed that with enough data and computing power, computers could start to see the world like we do, paving the way for self-driving cars, facial recognition, robots talking to you and seeing you, and other artificial intelligence things.

9:52 Our final wow moment takes us to London where a young Demis Hassabis was dreaming big - huge, enormously huge. It was 2010 when he founded DeepMind, a company focused on artificial general intelligence, AGI - what he described as machines that can think like humans.

10:13 At the Singularity Summit, Demis shared his unique approach combining neuroscience with AI.

10:18 “Although I’m making the case for systems neuroscience, what I’m really advocating is a hybrid approach - which is that we want to combine the best of machine learning has to offer and systems neuroscience.”

10:33 “So I think I can make this most clear by saying where we know how to build a component for an AGI system, let’s use the state-of-the-art algorithms, let’s just take those - the best of breed. Whether that’s reinforcement learning or hierarchical neural networks.”

10:50 “Where we don’t know how to build a component, then we should continue to push machine learning algorithms as far as we can but also it makes sense in parallel to look at systems neuroscience for ideas for potential solutions. So to do this in parallel, not one or the other.”

11:05 Demis argued that to build AGI we should look to the human brain. He highlighted how neuroscience could inspire new algorithms like those mimicking the brain’s visual system and navigation cells in 2010.

11:20 Just think about it - when others focused on narrow AI, weak AI, Demis was already chasing the holy grail of AGI. He talks about AGI. He calls it AGI. It was 5 years before OpenAI was even thought of. Isn’t that crazy?

11:40 DeepMind’s early work laid the foundation for later triumphs like AlphaGo, which beat the world’s best Go player in 2016 and blew everyone’s mind. Demis’s vision from 2010 - and probably he thought about it much earlier - shows that AGI was never a sci-fi to him. It was a real possibility.

12:05 And if we think who will be able to achieve AGI, my bet would be on Demis Hassabis.

12:13 There you have it. Three wow moments that define AI in the early 2010s. Jensen Huang gave us the GPU horsepower. Dr. Fei-Fei taught computers to see with ImageNet which sparked AlexNet and the whole deep learning movement. And Demis Hassabis set his sights on AGI with DeepMind that later became Google DeepMind.

12:38 Together they laid the foundation for the AI revolution we’re living in today. I’m excited. I am inspired by these people. They’ve been relentlessly working on their vision for decades and they made it happen for all of us.

12:57 But the AI world would not be possible without many, many, many promising categories and companies and concepts that failed. And today in the category “One Promise” is Nvidia that completely failed.

13:10 “If up to all the customers, industry analysts, other companies in the computer industry, Nvidia shouldn’t even be here today. We were already dead last among many. The world surely didn’t need yet another PC graphics company, and the space was littered with tens of competitors leapfrogging each other every month.”

13:38 “All of the conventional wisdom was true historically, but we believed completely irrelevant to the future that we saw. We believe that computer graphics was to become a massive opportunity as computing devices and displays proliferate everywhere and surround us.”

13:57 “We were young, naive, idealistic. We didn’t know what we didn’t know about building the industry and about building our company. We didn’t know that it was impossible. We just imagined how great it would be if computers would do what we imagined.”

14:18 “Our fresh and innocent perspective, unencumbered by conventional wisdom, allowed us to see what 30, 40, 50 other companies could not and did not. To this day, we challenge conventional wisdom with the same innocence. I wish you my third wish - to see the world like a child.”

14:38 Thanks for joining us today for this historical episode of Three Wow and One Promise. Please share this episode with your friends and everyone else and subscribe to this channel. I will be very grateful. It’s a very unconventional and very human way to look at artificial intelligence.