WEBVTT 00:00.560 --> 00:04.000 You know, I'm actually very excited about the future of databases. 00:04.960 --> 00:08.200 I know you're going to ask me another question. Let's share one thing. I feel 00:08.700 --> 00:12.840 like right now the world is going through a massive data center build 00:13.340 --> 00:17.120 out with a lot of AI training and inference data centers being built. 00:18.000 --> 00:21.680 I suspect something that a lot of the world does not yet appreciate is that 00:21.840 --> 00:25.520 in the future a lot of core infrastructure, be it 00:26.020 --> 00:29.480 computers, storage, storage will be used not just by humans, but by agents. 00:29.880 --> 00:33.160 And that agentic usage of all this infrastructure 00:33.660 --> 00:38.840 will drive massive demand. And so in addition to the AI 00:39.340 --> 00:43.120 data center buildup for training and inference, I think a 00:43.620 --> 00:47.040 lot of databases have been built for human usage today and we can only look 00:47.540 --> 00:51.160 at or query so much data. But with agentic usage, I think the demand for 00:51.320 --> 00:55.160 core infrastructure for computer will, will continue to skyrocket. 00:55.660 --> 00:58.360 So actually very bullish on this category as a whole. Yeah, it's a, it's an 00:58.860 --> 01:02.320 amazing space. I mean we feel very fortunate to be in the thick of it 01:02.480 --> 01:05.920 and actually we, we had a partner day yesterday where we invited a lot of 01:06.420 --> 01:10.080 people who work with builders or developers and basically everyone's saying, 01:10.580 --> 01:13.840 yeah, we're, we're running out of compute. We need like the amount of builders pouring 01:14.340 --> 01:17.920 into the world is just going crazy right now. Yeah, yeah. So actually 01:18.420 --> 01:22.570 I'm going to start you're. Because you have one of the most impressive backgrounds 01:23.070 --> 01:25.730 that, and I couldn't memorize it all, so I'm just going to. I know you're 01:26.230 --> 01:29.370 very humble. I'm going to read out for anyone who doesn't know, you're the founder 01:29.870 --> 01:33.250 of Deep Learning AI. You're a general partner of an 01:33.750 --> 01:36.410 AI fund, managing partner of AI Aspire, 01:36.810 --> 01:40.730 Chairman and co founder of Coursera professor at Stanford University. 01:41.610 --> 01:44.730 Previously you were the founding lead of Google Brain, 01:45.160 --> 01:48.680 VP and chief scientist at Baidu, Director, 01:49.240 --> 01:53.360 formerly director of the Stanford AI Lab, and you 01:53.860 --> 01:57.080 are part of the TIME 100 AI list for the most 01:57.580 --> 02:00.640 influential AI persons in the world. So first of all, 02:01.140 --> 02:01.320 that's crazy. 02:07.160 --> 02:10.880 What a life actually. How do you have 02:11.380 --> 02:14.820 the energy to kind of keep reinventing yourself and keep 02:15.320 --> 02:16.820 doing or do all these things? 02:18.100 --> 02:22.180 I feel like the answer I have is the obvious one, 02:22.500 --> 02:26.740 which is I've been really so blessed to work with so many great people consistently 02:27.240 --> 02:30.260 across so many organizations. I feel like, 02:30.900 --> 02:34.900 but for the people I get to work with, I would hardly manage to accomplish 02:35.400 --> 02:38.420 anything myself. So I've just been really blessed. That's cool. That's cool. 02:39.110 --> 02:42.710 Okay, so I was going to talk about 02:43.110 --> 02:46.910 skills Map. I was going to try explain it, but you are first and 02:47.410 --> 02:50.990 foremost a teacher, so maybe you can tell us a little bit about skills, 02:51.490 --> 02:54.710 map what you're doing at the moment and then we can dive deep. 02:54.870 --> 02:58.590 Yeah, I think the most exciting profession to be 02:59.090 --> 03:02.870 that's very rapidly at this moment in time is AI engineering. 03:03.270 --> 03:06.840 And by, by that I don't mean people that take on an AI 03:07.340 --> 03:10.600 engineering job title, but rather people in all sorts of job roles, 03:11.100 --> 03:14.920 including software engineering, front end, back end, DevOps mobile that 03:15.420 --> 03:18.000 will benefit from having AI engineering skills. 03:18.560 --> 03:21.800 So my team at D1AI did a lot of work over several 03:22.300 --> 03:26.240 months to try to map out what are the skills that are needed to 03:26.740 --> 03:30.590 do AI engineering well. And so we wound up categorizing, 03:31.090 --> 03:35.070 we said we looked at over 10,000 job descriptions, job postings, 03:35.310 --> 03:39.710 did dozens of structured interviews, had ran a bunch of our own surveys and 03:40.210 --> 03:43.950 synthesized views, talking to partners as well. And we wound up 03:44.450 --> 03:48.110 figuring out that there are four key skills needed to do AI engineering well. 03:48.510 --> 03:52.350 One is the ability to build and deploy AI applications 03:52.510 --> 03:56.030 and the difference between AI and non AI applications is they're uncertain. 03:56.910 --> 04:00.430 And so how do you manage that? The second is software engineering fundamentals. 04:00.590 --> 04:03.790 It turns out that even with AI writing a lot of code for us, 04:04.110 --> 04:07.950 people that understand software engineering fundamentals, understand what's the 04:08.450 --> 04:10.990 read write ratio, what's a transaction, you know, 04:11.550 --> 04:14.430 people can steer coding agents much better that way. 04:14.750 --> 04:18.910 Third is the ability to use coding agents. And then lastly is 04:19.410 --> 04:23.490 the ability to is a skill of shaping the bill because developers 04:23.990 --> 04:27.090 are no longer playing a role where someone tells them what to do and they 04:27.590 --> 04:31.130 just implement it, but instead are participating in deciding 04:31.630 --> 04:35.130 what to build and also influencing the process by which something gets built. 04:35.610 --> 04:39.610 So what I'm seeing in the job market today is developers, 04:39.850 --> 04:43.530 either people with an AI engineering title or people with other 04:44.030 --> 04:47.650 title that's gained some AI engineering skills are in great 04:48.150 --> 04:51.960 demand and we just can't find enough people with these skills. 04:52.200 --> 04:55.960 So deeplendai is excited to work upon this 04:56.460 --> 05:00.200 to help more people become skilled at AI engineering. Cool. 05:01.240 --> 05:06.280 And the skills map itself actually 05:06.780 --> 05:09.960 first of all, starting with skills, why are you focused on skills in particular? 05:10.440 --> 05:14.080 I think you have a view on skills that they're one of 05:14.580 --> 05:18.010 the more important parts. Do you think that they're going to be critical as we 05:18.510 --> 05:21.730 go along or can the agents just become so good that we don't need to 05:22.230 --> 05:25.770 focus so much on providing our own custom skills? How do you see this part 05:26.270 --> 05:30.610 evolving? Yeah, so you know, I think about there's 05:31.110 --> 05:35.250 some Things we do today that I hope an agent will take over for 05:35.750 --> 05:39.330 me. For example, you may remember three years ago, 05:40.130 --> 05:43.810 there's a lot of hype about the prom engineering job and how much money 05:44.310 --> 05:47.680 someone could learn make by being a prom engineer. And I think at that time 05:48.180 --> 05:51.600 it was already clear to actually give Chris Manning credit 05:52.100 --> 05:55.480 for saying this very clearly even back then, which was this idea that we 05:55.980 --> 05:59.200 need humans have arcane incantations to prompt an ELM to 05:59.700 --> 06:03.040 do something that's going to go away because an ELM was going to learn how 06:03.540 --> 06:07.440 to make those same arcane incantations. And so even today I 06:07.940 --> 06:11.640 see some things that, like how 06:12.140 --> 06:15.620 do you manage the sword form of agents? Some things humans will be needed 06:16.120 --> 06:18.900 to do. Some things is more of a one size fits all that. I think 06:19.400 --> 06:22.660 AI agents will manage other AI agents better and better in the coming years. 06:23.140 --> 06:27.220 And one of the key things that distinguishes whether 06:27.720 --> 06:31.100 AI will take this over or not is whether or not humans have a 06:31.600 --> 06:35.380 fundamental context advantage. It turns out when all of us are building applications, 06:35.700 --> 06:39.140 we know a ton of stuff that AI agents do not know. You may 06:39.640 --> 06:43.460 remember a conversation with a customer where you said something and 06:43.960 --> 06:47.540 there was a funny expression on their face, or you may remember something 06:48.040 --> 06:51.460 that you learned five years ago reading some market report saying people 06:51.960 --> 06:55.620 want this and not that. I find that humans have a bewildering vast 06:56.120 --> 06:59.660 range of contexts and remarkably good at pulling out this context 07:00.160 --> 07:03.780 and make key decisions. And so for that set of decisions where humans 07:03.860 --> 07:07.670 have a fundamental context advantage of, until someday 07:08.170 --> 07:11.910 where I don't know, like slap a camera on us and monitor the facial expressions 07:12.410 --> 07:15.590 of everyone we talk to and have, that's a very difficult path. So I 07:16.090 --> 07:19.990 think there's a lot of the task with humans with a fundamental context advantage 07:20.070 --> 07:23.910 that we want humans to do for a long time and then giving 07:24.410 --> 07:28.310 people the skills to do that valuable work to complement the wonderful 07:28.810 --> 07:32.030 work the coding agent is doing, that seems to be the path 07:32.530 --> 07:35.670 that we'll be on from a long time to come. And this is why with, 07:36.170 --> 07:39.590 I think you've probably seen with coding agents being the wonderful things they are, 07:40.230 --> 07:43.590 the number of job postings, the software engineering is up because as 07:44.090 --> 07:47.190 software engineers and AI engineers become more productive, we want a lot more of 07:47.690 --> 07:51.030 them. And businesses, I know, even my teams, we just can't seem to find enough 07:51.530 --> 07:55.150 skilled AI engineers. And I think this will continue. Yeah. 07:55.650 --> 07:59.760 So let's dig into that. Let's imagine two developers 08:00.260 --> 08:03.960 who are, who are too strong developers, but one gets better results than 08:04.460 --> 08:07.880 the other. What would the stronger Developer be doing that the 08:08.380 --> 08:11.640 weak one isn't doing today. You know, this is a great 08:12.140 --> 08:14.960 question. When we're mapping out the AI engineering skills map, 08:15.600 --> 08:19.160 that was one of the questions we repeatedly asked ourselves, which is we actually 08:19.660 --> 08:23.120 interviewed a lot of people and one of the standard questions you asked them was 08:23.360 --> 08:26.240 think of a great developer, think of a less good developer. 08:26.640 --> 08:29.800 What is it that a really great developer does compared to the less 08:30.300 --> 08:34.600 good one? And a lot of the answers were what we 08:35.100 --> 08:38.080 wound up distilling into the AI engineering skills map. 08:38.160 --> 08:41.440 And it's actually more than one thing, it's actually a lot of very different things. 08:41.760 --> 08:45.280 But just to give another example, the first of four 08:45.780 --> 08:49.000 key skills we highlighted was the ability to build and deploy 08:49.500 --> 08:53.690 AI applications. And it turns out that the key differentiator 08:54.190 --> 08:58.610 for AI applications compared to traditional software applications is they are uncertain. 08:58.930 --> 09:03.010 Right. Whether you prompt an LLM or train a supervised learning algorithm, 09:03.510 --> 09:06.770 you don't know in advance what outcome you get. And so skill 09:07.270 --> 09:10.890 AI developers have come up with or are familiar with a 09:11.390 --> 09:15.250 set of techniques to monitor the performance of these systems, often in a statistical 09:15.750 --> 09:19.710 way, and to drive a disk error analysis and evals 09:20.210 --> 09:24.230 loop to make progress in a systematic way. And so it was by asking 09:24.730 --> 09:28.510 questions like that and reflecting on our own experiences that we then tried to systematically 09:29.010 --> 09:32.230 map out the, not one or two, but frankly the very large number of gaps 09:32.470 --> 09:36.510 between skill developers and novice level developers. And we 09:37.010 --> 09:40.790 tried to distill that into what we call the AI engineering schools map 09:41.290 --> 09:44.850 to then try to work systematic to help people go from 09:45.350 --> 09:48.610 your less skilled expert level at all of these really valuable skills. 09:49.010 --> 09:52.050 Is the skills map itself for anyone to use, 09:52.370 --> 09:55.410 mostly for software engineers, people with background, 09:56.050 --> 09:59.290 how should people think about the skills map itself? Yeah, 09:59.790 --> 10:03.090 so we published Deep Learned AI. We published 10:03.590 --> 10:07.210 a weekly newsletter called the Batch. So for five weeks in a row I wrote 10:07.710 --> 10:11.340 about nothing but the AI aging skills map and actually know other 10:11.840 --> 10:15.060 companies that are now looking to structure curricula 10:15.560 --> 10:17.740 around REI English Skills, which is actually pretty cool. 10:18.620 --> 10:23.020 But I feel like for individuals there's a way to understand what 10:23.100 --> 10:28.060 being excellent, which many of you already are, but where you're 10:28.560 --> 10:32.580 already excellent in other places where you want to become more excellent. And then one 10:33.080 --> 10:36.730 thing deep learning AI is working hard on is to then create those paths 10:37.230 --> 10:40.330 for people to become skilled at AI engineering broadly. 10:40.810 --> 10:44.570 But I think the educational space, both deep learned AI 10:45.070 --> 10:48.410 and other companies still have work to do to help people develop 10:48.490 --> 10:52.290 clearer paths to become skilled broadly at all of these AI 10:52.790 --> 10:56.890 engineering schools. I'm curious, what are you Seeing in terms of 10:57.390 --> 11:01.420 people who are pouring in, is it experienced developers? Basically there 11:01.920 --> 11:05.300 probably is some sort of curve where there's these experienced developers who don't want to 11:05.800 --> 11:09.780 let go. I mean, myself, I loved the getting in the zone of 11:10.420 --> 11:13.780 coding and it became a long time. It took me a long time 11:13.860 --> 11:17.060 before I was like, okay, now I just let the agent do it. 11:17.380 --> 11:20.540 Are you seeing this shift slowly happen where people are like, 11:21.040 --> 11:24.260 okay, I'm giving up and I'm shifting all of it to 11:24.500 --> 11:27.900 skills first. Yeah, it's interesting actually. 11:28.400 --> 11:31.920 You know, um, I, I don't know, it's funny, I actually don't 11:32.420 --> 11:34.520 miss coding by hand. Yeah, 11:35.160 --> 11:37.640 yeah, yes, sir. 11:39.080 --> 11:42.440 But actually, do you, do you miss the. 11:42.920 --> 11:46.200 I don't know, do you get like in the zone? You know, when you just 11:46.700 --> 11:49.240 kind of lock in and it's problem. You had to go to stack overflow and 11:49.740 --> 11:53.880 copy and paste 50,000 things. Maybe you didn't. I did. And then, 11:54.520 --> 11:57.480 you know, you spend three hours on a problem and finally it clicks. 11:58.020 --> 12:01.300 I don't kind of get that vibe, I guess anymore. But yeah, I also. 12:01.800 --> 12:05.100 Yeah, yeah, I bet you, I bet lovie have 12:05.600 --> 12:09.060 this problem too. Remember before AI, we used to have a list of projects we 12:09.560 --> 12:13.140 wanted to work on but couldn't finish. Now my list, I'm going to 12:13.640 --> 12:17.500 guess yours too has gotten way longer now that AI has supposedly made us more 12:18.000 --> 12:21.740 productive. So I find that for myself, there's so many projects 12:22.240 --> 12:25.780 doesn't work on and don't have time to work on. And even as our capabilities 12:25.860 --> 12:29.390 grew up, certainly my personal ambitions grew maybe 12:29.890 --> 12:32.390 even faster than our capabilities. And so, 12:33.270 --> 12:36.430 yeah, it's a different type of vibe. I think starting a coding 12:36.930 --> 12:39.590 agent before you go to bed to see what it comes up with the next 12:40.090 --> 12:43.990 morning is a different way of getting into the zone, I guess, than debugging 12:44.490 --> 12:47.750 some weird JavaScript syntax. But I have to admit, 12:47.910 --> 12:51.430 I seem to like the new vibe more than he owes one. 12:51.930 --> 12:55.290 Yeah, yeah, we're definitely getting more done, 12:55.790 --> 12:59.330 that's for sure. Actually, like, I'm curious 12:59.830 --> 13:03.890 how you design a curriculum in such a rapidly changing 13:03.970 --> 13:08.210 market. Do you foresee that you're going to have to change this curriculum 13:08.710 --> 13:11.970 every single year, every month? How are you iterating on it? 13:12.450 --> 13:16.170 You know, it's interesting. I feel like the hype is that AI advances so fast, 13:16.670 --> 13:20.180 everything changes every three months. Um, of the AI 13:20.680 --> 13:24.140 engineering skills map, the one piece that does change the fastest 13:24.640 --> 13:28.220 is coding agents. And the way we code now is very different 13:28.720 --> 13:31.740 than three months ago, very different than six months ago. So the use of coding 13:32.240 --> 13:35.940 agents is changing very rapidly. But when I look at the other parts of the 13:36.440 --> 13:40.700 skills that I find really valuable, the way we build and deploy AI applications, 13:41.200 --> 13:44.660 there's a lot of fundamental knowledge about how to steer these systems, how to deploy 13:45.160 --> 13:48.190 them, how to monitor them, how to update them, how to that is actually changing 13:48.270 --> 13:51.710 much slower than the PR hype says. One of the 13:52.210 --> 13:55.950 funny things, at least before Astra, I think Astra 13:56.450 --> 13:59.710 may have changed, I'm not sure. It turns out most AI systems, 14:00.210 --> 14:03.510 they're not very good data scientists and not very good AI engineers. You kind 14:04.010 --> 14:07.230 of ask it, what should I do in a project? And for the most part, 14:07.790 --> 14:11.190 frankly, data science, right. I now don't do 14:11.690 --> 14:15.210 my data science work by hand. No more coding up pandas, 14:15.710 --> 14:18.930 data frames, whatever I'm still using on the backend, but I now use AI agents 14:19.430 --> 14:23.250 to do a lot of my data science work for me. And then frankly it 14:23.750 --> 14:26.330 just makes really weird decisions, comes to really weird conclusions. 14:27.210 --> 14:30.410 I hope that frontier models will get better, but somehow 14:30.569 --> 14:34.330 there's a lot of understanding the fundamentals to make trade offs. 14:34.570 --> 14:38.810 And I see that a lot of engineering is making trade offs, 14:39.310 --> 14:43.690 right? Cost, latency and then speed, 14:44.190 --> 14:47.730 reliability. And the challenge 14:48.230 --> 14:51.610 is until you understand the fundamentals, you often don't even 14:52.110 --> 14:55.770 know what trade offs exist to be made. Which is why when someone vibe codes 14:56.270 --> 14:59.610 without understanding, very often the coding agent will make a bunch of 15:00.110 --> 15:03.730 trade offs that you didn't even know it made a trade off and just bites 15:04.230 --> 15:07.850 you later. So I find that people that are good engineers 15:07.930 --> 15:11.800 understand the fundamentals in many, many parts of the skills 15:12.300 --> 15:15.600 needed just make much better decisions. But I'm also encouraged at 15:16.100 --> 15:19.520 what's now possible to do. You know, when you write code without understanding. But I 15:20.020 --> 15:23.400 think really good engineers just do such vastly better 15:23.900 --> 15:27.000 work than, you know, inexperienced engineers. 15:27.240 --> 15:31.040 Yeah, yeah. And actually on that one, like as 15:31.540 --> 15:34.040 part of the skills map you talk about shaping the build. I found this kind 15:34.540 --> 15:37.980 of an interesting concept. Maybe you can explain what you mean by shaping the 15:38.480 --> 15:41.700 build. Yeah, so as building has become 15:42.200 --> 15:46.020 easier, the bottleneck is less building, but deciding 15:46.520 --> 15:50.300 what to build. And so previously there were used to be a lot 15:50.800 --> 15:54.380 of, let's say front end engineers, but now front end engineers with 15:54.880 --> 15:57.220 AI assistance can also do a lot more backend than was possible. 15:57.860 --> 16:02.690 I'm proceed much stronger on backend and front end, but now frankly 16:02.770 --> 16:06.450 I'm maybe like a bad front end developer with AI 16:06.950 --> 16:10.330 systems. But so a lot of developers have gone from a more narrow role to 16:10.830 --> 16:13.650 being a full stack developer. But then even beyond that, 16:14.130 --> 16:17.330 it turns out that if a developer builds something and 16:17.830 --> 16:22.370 needs to wait for a product manager to get customer feedback and steer 16:22.870 --> 16:26.690 what's built next, that's much slower than the developer themselves 16:27.010 --> 16:30.170 has a little bit of product instinct, a little bit of product gut. Okay, 16:30.670 --> 16:34.400 talk to a couple customers or users and get feedback and keep driving the process 16:34.900 --> 16:39.600 forward. So in development, what I'm seeing is a lot of people are 16:40.100 --> 16:44.000 expanding their roles from just implementation to deciding 16:44.160 --> 16:47.800 what to build. And it does not mean engineers 16:48.300 --> 16:52.240 should become product managers, but it does mean that engineers, they can 16:52.740 --> 16:56.400 make some product decisions or more architecture decisions are much 16:56.900 --> 17:00.320 more productive than engineers that played the more 17:00.820 --> 17:04.040 narrow side of role from a few years ago. And so we call 17:04.540 --> 17:07.840 this shaping the build where engineers that can 17:08.000 --> 17:11.400 influence product decisions have some product guts also 17:11.900 --> 17:14.600 help to think through the build process. Like is this a good time to build 17:15.100 --> 17:18.920 an mvp? Or is it a good time to invest in building really robust architecture 17:19.420 --> 17:22.160 and thinking about scale? Or is this a good time to go out to customers 17:22.660 --> 17:25.400 to get feedback? Or this is a good time to do some research to better 17:25.900 --> 17:30.240 understand what backend technology stack you use. So developers that can steer 17:30.480 --> 17:34.480 the building process end up contributing much more to the overall 17:34.880 --> 17:38.240 comfy or project. How much then should they 17:38.320 --> 17:41.760 spend time kind of designing the system? One of my favorite skills, 17:42.260 --> 17:45.080 I don't know if you know, it's this like grill with docs creates these like 17:45.580 --> 17:49.400 architectural decisions and it spends an hour grilling you exactly what 17:49.900 --> 17:52.960 you want to build. Like how much time do you think people should be focused 17:53.460 --> 17:57.360 on upfront, what they want to build, how they want to build it versus 17:57.860 --> 18:00.880 like just go build your MVP and then fix it later on. 18:01.200 --> 18:04.560 Iterating? Yeah, I think, yeah, it's a great question. 18:04.640 --> 18:08.360 The challenge answering that question is that the answer is stage 18:08.860 --> 18:12.120 dependent. And it depends a lot on what Jeff Bezos calls 18:12.620 --> 18:15.880 one way door versus two way doors. But directionally, in the early 18:16.380 --> 18:20.360 stages of a project, you know, frankly, whatever idea you have right, 18:20.600 --> 18:24.080 type it into coding agent, build it, see what you 18:24.580 --> 18:28.520 get. Soles can do it in a safe way and don't leak sensitive data or 18:28.680 --> 18:31.880 just don't solas their protection for bad outcomes. But I find that a 18:32.380 --> 18:36.360 lot of projects, one of the challenges about AI engineering is the right decision 18:36.440 --> 18:39.880 is very stage dependent. I find that for early stage projects, 18:40.120 --> 18:43.360 solids, you can do it in a responsible, safe, safe way. 18:43.760 --> 18:47.640 Build a quick and dirty prototype, see what happens, throw it away. Building a 18:48.140 --> 18:51.720 quick and nervous product, see what happens and drive rapid learnings and then there's also 18:52.220 --> 18:55.120 a time and place for thinking carefully about architecture, 18:55.440 --> 18:59.240 figuring out what's your rewrite ratio, what are the customer requirements, what is sensitive 18:59.740 --> 19:03.360 data, thinking through your security infrastructure and then doing that. 19:03.600 --> 19:07.400 So one thing I've seen is I've known a lot of engineers that 19:07.900 --> 19:11.360 are really good at the zero to one startup phase. I've also known a lot 19:11.860 --> 19:14.800 of good engineers good at the mature systematic phase. 19:15.200 --> 19:18.720 But sometimes, depending on the stage of project 19:18.800 --> 19:22.680 you're on, the skill and the speed and how to think 19:23.180 --> 19:26.480 about velocity versus protecting from downside, risk shifts 19:26.880 --> 19:30.480 and even the user coding agents you see, a lot of people talk 19:30.980 --> 19:34.880 about how, oh, you should always plan it out, write a long spec and 19:35.380 --> 19:38.410 there is a place for that. And then sometimes there's a place if you know 19:38.910 --> 19:41.650 you're going to go to bed in 20 minutes, just bang out some stuff, 19:41.730 --> 19:44.330 have a coding agent work for a few hours and see what you get the 19:44.830 --> 19:48.330 next morning. And I think that stage dependent answer has been 19:48.830 --> 19:52.570 very tricky for people to learn. Okay, I need 19:53.070 --> 19:56.890 to switch gears and I need to ask you a question because I was chatting 19:57.390 --> 20:00.770 to Swix yesterday from Legendspace. I don't know if you know him, but he's here 20:01.270 --> 20:04.460 covering the sweep and he said something spicy, I might misquote 20:04.960 --> 20:08.340 him, but he said something like, oh, schools are dead. Or the way that we're 20:08.840 --> 20:12.940 changing, the way that we have to learn has to completely 20:13.440 --> 20:16.860 change. I'm sure you get this question a lot. I'm curious on your 20:17.360 --> 20:21.100 take on how will learning change put us forward 10 years? 20:21.420 --> 20:24.740 How are we learning? Surely does it look the same as what 20:25.240 --> 20:28.710 we're doing now in schools? Completely different. So it will change. 20:28.870 --> 20:34.190 So there are two observations. So first I'm 20:34.690 --> 20:38.110 going to say something obvious but maybe sound controversial, but chatbots 20:38.610 --> 20:42.030 are terrible for learning, right? I love OMS, use them 20:42.530 --> 20:45.830 a lot. Kudi Pro AI at the same time, on university 20:45.990 --> 20:49.590 campuses and in K12, we know that 20:49.670 --> 20:53.270 students that use LLMs do better on homeworks 20:53.360 --> 20:57.040 but worse than the final exam. And educators 20:57.540 --> 21:00.880 have known this intuitively. Students have known this intuitively, but the rigorous studies 21:01.380 --> 21:05.360 are now increasingly coming out that show that cognitive offloading to an LLM, 21:05.600 --> 21:08.760 meaning if a student lets an LLM think for you, you get 21:09.260 --> 21:11.760 more work done, but you learn less. 21:12.800 --> 21:16.240 I think even though the rigorous studies on this are coming out in 21:16.740 --> 21:20.280 K12 and in higher education university in formal educational 21:20.780 --> 21:23.970 context, I think people still underestimate the degree to which this is happening 21:24.130 --> 21:27.690 among working adults. I find that I 21:28.190 --> 21:32.090 love my coding agents, helps me do a ton of work. But the reduced 21:32.590 --> 21:36.170 thinking I have to do, I think there are certain things that. And then to 21:36.670 --> 21:39.970 be clear, there are ways to use arms that do improve 21:40.470 --> 21:44.090 learning. Just like I think there are ways to use TikTok that's great for 21:44.590 --> 21:48.440 learning. But the vast majority of ways that the vast majority of people use social 21:48.940 --> 21:52.400 media does not result in learning. And the vast majority of ways the vast majority 21:52.900 --> 21:56.720 of people use OMS does not result in learning. Now the 21:57.220 --> 22:00.640 flip side of this though is I think we can do better. So the 22:01.140 --> 22:05.400 teams I work with, including Coursera, Udemy, Deep Learned AI, Learn Vector, 22:05.480 --> 22:09.560 are working hard to create new learning experiences that 22:09.720 --> 22:14.210 are not focused on getting stuff done, but rather on developing, 22:14.290 --> 22:16.850 rather on focusing on accelerating human development. 22:17.970 --> 22:21.450 Here in Silicon Valley, we're very excited about our front end AI labs 22:21.950 --> 22:25.410 and accelerating AI development and that's great. Very much in favor of that. 22:25.730 --> 22:29.410 In Learn Vector, which was a recent announced company funded by, 22:29.489 --> 22:33.050 with Coursera's initial investor. I'm actually really excited 22:33.550 --> 22:36.770 about not just accelerating AI development, 22:37.090 --> 22:40.530 but accelerating human development. And the 22:41.030 --> 22:44.330 way that we're operating is building Learn 22:44.830 --> 22:48.730 Vector into a frontier AI, into frontier 22:48.810 --> 22:52.170 human development lab. And as one example, 22:52.970 --> 22:56.370 I feel fortunate that Coursera participated in changing 22:56.870 --> 23:00.010 education 15 years or so ago. But today, 23:00.890 --> 23:04.330 looking back on that experience, which still works, it feels 23:04.830 --> 23:08.490 very much one to many where we create one course and everyone kind of 23:08.990 --> 23:11.610 takes the same or very similar course and it works. Nothing's wrong with that. 23:11.770 --> 23:15.610 But the technology now exists for us to create much more one to 23:16.110 --> 23:20.130 one experiences and it will be much better. So Learn 23:20.630 --> 23:24.610 Vector hasn't yet announced exactly what products we have, but I'm 23:25.110 --> 23:28.370 actually very optimistic from the internal prototypes we have and we 23:28.870 --> 23:32.700 will have more to share in months. That's super exciting. Okay, I want 23:33.200 --> 23:36.940 to leave the crowd with something tangible. So let's say you 23:37.440 --> 23:40.940 could design a way that everyone here could spend 23:41.440 --> 23:46.100 the next four weeks upskilling to become the best AI 23:46.580 --> 23:50.500 developers. How should they spend their time for the next 23:51.000 --> 23:53.140 four weeks? Where should they go? What should they learn? 23:54.020 --> 23:57.060 What tools should they adopt? Anything that comes to mind. 23:57.870 --> 24:01.310 I know. So you know, in the case of AI engineering, 24:02.190 --> 24:06.350 deeplanddey's focus is to help developers become skilled 24:06.850 --> 24:10.190 at AI engineering, or if you're already skilled, to become even more skilled, 24:11.150 --> 24:15.190 working hard on, even improving what Deep Learned AI offers. 24:15.690 --> 24:19.430 But I actually find myself taking a lot of Deep learned AI courses myself to 24:19.930 --> 24:22.390 learn what are the frontier tools? Kind of what's the latest in the voice stack 24:22.890 --> 24:26.870 or in Generative UI or You know, building local memory for agents. So I 24:27.370 --> 24:30.830 find that maybe as an insider, I see a lot of ways I wish deep 24:31.330 --> 24:34.790 learning were even better, but I think that 24:35.350 --> 24:38.870 my own learning, actually, a lot of it has been. Has been with deep, 24:39.370 --> 24:43.350 learned AI. So I hope you all will find that hopeful cool 24:43.510 --> 24:46.870 and keep building. I think. Yeah. I think. I think all of you are builders, 24:47.370 --> 24:51.120 Right? I find myself. I still find myself waking up in 24:51.620 --> 24:53.920 the middle of night too often to check what my coding agents up to. 24:54.720 --> 24:57.280 Do you fire off on your phone as you wake up in the middle of 24:57.780 --> 25:00.760 the night? Yeah, I don't know. I think coding on the phone has been so 25:01.260 --> 25:05.000 wonderful, but also maybe. I don't know. Yeah. Do you text 25:05.500 --> 25:08.720 or do you use voice to chat to your coding 25:09.220 --> 25:13.160 agents? I use text a lot more than voice to coding with 25:13.660 --> 25:17.360 the usual Mac Mini. Okay. Yeah. Okay. 25:17.860 --> 25:21.000 That's cool. There's a bit of inside baseball. I'm very curious about your setup, 25:21.500 --> 25:26.840 actually. Okay. First of all, I know that you had a very strict 25:27.320 --> 25:31.120 schedule, and so we're incredibly grateful that we could fit 25:31.620 --> 25:35.440 you in and you could fit us into your timing. So thank you so 25:35.940 --> 25:39.200 much for coming. Coming to select. It's really a pleasure and an honor to have 25:39.700 --> 25:41.920 you. Thank you so much. And just say, you know, of all the credentials you 25:42.420 --> 25:45.190 rattle off with me just now, the thing that's going through my mind, my one 25:45.690 --> 25:49.070 credential you didn't mention was I'm also proud father of 25:49.570 --> 25:51.030 two children, so. 25:57.590 --> 26:00.190 And so the reason I need to dash is because I actually need to go 26:00.690 --> 26:03.190 pick them up from school. That's super important. 26:03.430 --> 26:06.630 Okay, we'll let you go there. Thanks so much, Javi. Thank you. 26:11.090 --> 26:15.330 Into noise 26:15.830 --> 26:16.610 hardware and a regularity.