A lot of the top AI engineers that I work with on a day to
day basis have a mechanical engineering background. So you can actually be the one to
go into AI and take other people's jobs. There's actually not that much math behind
it. That's very encouraging. It is not too late for me. No, for you it's
too late. Hopefully the problem is that it's interesting,
but I'm so bad at presenting it as being interesting. Damn, I wish I was
wearing shoes. Can you see my feet? No. Okay. Okay, great. All right.
12:30 at night. Jesse, why don't you introduce yourself? I'm Jesse.
I, like Leon was Cornell mechanical engineering.
And I kind of quickly diverged from Leon's scope.
So Leon was more like pure mechanical. And then I went more robotics
programming side. I worked for seven years promotional
on self driving cars. And then recently I joined Boston Dynamics doing humanoid robotics.
You were a control specialist in college, is that right? Undergrad. It was just kind
of vanilla mechanical engineering. I did Cornell autonomous sailboat team,
which kind of got me a flavor in robotics where I started
getting a taste of what I wanted to do was ironically matlab class.
So matlab is this like very mechanical engineering programming language.
I was really enjoying it and I didn't realize that there's a whole world of
programming. I think as a mechanical engineer, it's easy to think we are mechanical engineering,
we are doing machine shop and these things, you know, computer science and programming is
a totally different. But actually robotics is this nice major that
lets you combine mechanical and programming. It's very lucky that
I'm probably the most comfortable in the world with you, Jesse. I'm the least
comfortable with you. You recently left Motional as
like a super principal, hyper principal controls engineer. Just principal.
Just principal. And became a reinforcement learning engineer.
You saw a few years back that machine learning was exciting and I think you've
been low key working on a pivot. And so robotics is this interesting thing that
combines many different majors. So you have computer science, electrical engineers,
mechanical engineers that are all working together. I think a lot of industries it's
easy to silo yourself in like your thing. But robotics, you kind of have to
have a good understanding of each component. You can't just be a mechanical engineer.
You have to be a mechanical engineer who knows how to program, who knows a
little bit of electrical engineering. So I started in kind of this controls route
where I'm like, I don't want to be the one that manufactures or like makes
this part. I want to be the one that controls this thing to do something
useful in the world. So the classical thing that you do as a controlist engineer
is you like start with the pendulum. So like you take this pendulum that's just
swinging on the bottom. The problem you're trying to solve is how do you swing
the pendulum so that it's balancing itself. And so as a controls engineer,
you start with these very simple canonical problems and you build up from that,
like how can you sense the world, how can you use the sensors in the
system that you have and the actuators and the motors to actually do something useful?
In robotics you have these very different industries. So you have humanoid robots or you
have self driving cars. And at face value they seem very different, but actually a
lot of the underlying algorithms are similar. So like controls is this
field where you take sensor input from the outside world and you determine
how to actuate the motors that allow you to get the system
like the pendulum to do what you want. So like swing up and balance itself.
But kind of as you go along this path, you realize that these more complicated
systems, like a self driving car or a humanoid robot, are more
complicated, that are more nuanced. Not just this input and output system. You have like
other people around you, other pedestrians, other agents that are interacting with you.
As you build up these systems, you realize some of these classical algorithms
that you learned in school are too simple, naive, and you actually
have to like add more and more complexity. And at some point you start going
down the path of machine learning and you realize that machine learning and AI is
the way to control these more complicated systems, to look at the world around
you with cameras and perception as the sensors. And now you have the actuator
for self driving cars that could be like the steering wheel and the gas and
brake. And how do you look at the world around you and then actuate the
system to safely interact with the other cars and the other pedestrian.
You might start with these simple control algorithms like pid, go to
more complicated things like model predictive control or trajectory optimization.
And then as you encounter more and more complicated situations,
you realize that kind of the final solution is like machine learning or AI.
The hot take or the sound bite is like mechanical engineers. You might be thinking
like, oh, this AI is going to take our jobs. As a mechanical engineer,
you might be like, okay, that's like computer science, that's AI. That's a different field.
As a mechanical engineer, actually, I'm finding that there's a lot of AI engineers
who have the background of a mechanical engineer. And so you can actually be the
one to go into AI and take other people's jobs.
Mmm, Saucy. Not all the mechis are controls or math
savvy as you, for the meatheads like myself who are more spatial or
keen to design blocks with holes. How would someone like that kind of get
into the controls and robotic stuff if they don't have a knack for dynamics
and hard maths? Yeah, I think it's a misconception that, like, you need math to
go into these fields. Like, a lot of programming and computer science, they don't
even know, like, sine and cosine. It's kind of a wide range of things it
can do in robotics and programming. A lot of it's not theoretical, and actually
a lot of modern robotics is not based on a theoretical understanding
of what works. Like, AI is kind of a classical example now of no one
knows why it works. There's not that much theory behind it. It just kind of
does. There's actually not that much math behind it. You can kind of download a
library like Pytorch or Scikit, learn, and you just throw a bunch of training
data at a problem and get the AI model to learn something. And you
don't really need to know the theory and you can kind of just watch it
work and have it solve problem that you care about. Say someone is a
mechanical engineer in another industry interested in getting into robotics
and wants to do a side project. What is the scope of a meaningful side
project to make their way into the robotics space? Yeah, I mean, one thing that
I like about robotics is that it's kind of easy to iterate and to
work on these side projects, because a lot of it's programming. A good place
to start could be to write your own simulation where you're not needing to actually
build anything physical. You can kind of write a simulation. For instance, like, I started
with, like, an autonomous sailboat. So, like, my project project wasn't building the sailboat,
it was creating a simulation of a sailboat. As far as projects that you can
kind of start, like a mechatronics or robotics project, there's a lot of existing kits
that you can buy. Robotics kits are nice because it kind of walks you through
the basic steps. But I think once you learn the basics, like once you kind
of take a couple introduction classes to robotics, I think it's more meaningful
to jump immediately to, like, your own thing. And so, like,
an example of something that I did was these balancing bots that have just two
wheels, like A segue. And this is going back to the pendulum idea, where like
this pendulum is this very simple system in mechanical engineering. You can have
a pendulum, a pendulum naturally wants to fall over. And a segue, or like a
robot on two wheels, is a system that naturally wants to fall over. You can
create your own motor controller, have your own imu,
your own power system, and develop a robot pretty simply with
maybe only 100 lines of code that reads the IMU to detect the angle,
and then controls the motors to keep the robot upright.
So like, as the robot starts tipping over this way, the motors will kind of
carry the base under it and it's a self stabilizing system, as one example.
But I think you can come up with a lot of interesting, relatively simple projects,
whether it's like purely simulation or involves some simple hardware that
you can kind of work on your own idea and see it to fruition.
How did you transition from controls engineering into
reinforcement learning? Yeah, that's a good question. So there's this famous, this paradox called
Moravec's paradox, that's making the observation that everyone thought that the first
job that would be automated was like the manufacturing job, that a robot
would replace a person in a factory doing these simple pick an object up here
and place it here. And people assumed that the thing that would be hard
to replace would be like an artist or like a poet or something that requires
creativity. Moravec's paradox is this observation that actually
the types of things that are being automated are these creative jobs. And actually
the hardest thing that we're trying to automate is like this simple thing of like,
can I pick up this object here and move it here? Kind of operating in
the physical world. Controls gives you these, this mathematical framework that is
kind of theoretically back. For instance, one of the options here is like trajectory optimization,
where you find like a closed form thing that says like, this is
the optimal solution to move this here and, and put it here.
But it turns out that in any meaningfully useful situation, it becomes
too complicated to like formulate this thing in this classical way
of controls. And that reinforcement learning is simply controls
that's backed by machine learning. So reinforcement learning is this
process of trying to have this agent or this
robot that's operating this world that takes in some observation
of what is around it, you know, the state of this table and some objects
on this table, and is now taking an action, taking its hand and moving this
object here. So that is a controls problem. But for more
complicated systems, which are the systems that we see in like day to day industry
with types of systems that you're trying to automate. It turns out that classical controls
often struggle in these more complicated settings. And reinforcement learning using
AI as the brains of this controller is gaining
popularity as a way to solve these problems. Pick and place and end effectors is
this long standing problem that's been plaguing Amazon and humanoid robotics.
What is the state of the art or the bleeding edge of this problem and
the technologies? Amazon would profit billions and billions of dollars a year if
they could simply figure out how to take an object or a package and
move it from this box to this box. And it seems like such a trivial
problem to solve. We have AI showing up in all these places like ChatGPT,
and yet we can't figure out how to move an object from here to
here reliably enough to make this system profitable. Which I think is an interesting observation.
I think that's bound to change. And that's why I'm in this industry, because I
feel like this is like the next ChatGPT is going to be a system that
can operate in the physical real world and do these simple tasks
that seem to require common sense. So like ChatGPT was learning
common sense on like the digital scale and robotics is, how do we
learn common sense on the physical scale? We need a chatgpt moment. For robotics,
the state of the art is very simplistic. The current state of the art are
these simple robots that have like suction cups that kind of suck up an object
to move it from one place to another. And the reason that they're
using suction cups is simply because you don't have to be very precise.
You just go over to an object, suck on it like an octopus tentacle to
move it, and it doesn't really matter to like detect the exact shape or texture
of that object. So that's more of a crutch than the optimal solution. The optimal
solution is like a human hand, like human hands are incredibly dexterous. We evolved to
be able to manipulate fine grained objects. We underestimate
how difficult of a problem this is to solve. We've evolved for billions
of years to be able to do these very subtle tasks. And our skin is
filled with millions of these very fine grained sensors that are
very hard to get into like a skin of a robot. And so I
think this is an unsolved problem. The state of the art are these like suction
cup like robots. Like even at Boston Dynamics there's this stretch robot which is like
kind of the most industrial friendly robot that we have that is this big crane
filled with suction cups that pick up a box and put it here. Boston Dynamics
is trying to get atlas, which is a humanoid robot, to be able to use
a dexterous hand to do these tasks instead. And I think it's a
matter of time. I think it will be an AI solution and I think that
mechanical engineers will benefit from learning that. It's not
like mechanical engineer versus AI or mechanical engineer
versus software engineer. You can, as a mechanical engineer go down the path.
You can kind of meet in the middle. You have mechanical here, you have software
here. Robotics is in the middle. AI can be in the middle. A lot
of the top AI engineers that I work with on a day to day basis
have a mechanical engineering background. You might think of this as like the thing that
will take your job. You can actually instead be the one that joins this AI
and ML field and be the one to kind of solve this next frontier,
which is like, how do you get robots to. To have common sense and interact
with the world in a useful way? Nice takeaway. When we were in school,
we took that graduate level robotics course and we tried to learn ros, which had
like the steepest, most absurdly painful learning curve ever.
Like, I think we dropped out of it after two weeks. And then you eventually
took it again and passed it, right? Yeah, it was this classical robotics
class where it's like a mix of programmers, computer science majors and
electrical engineers. And then like us poor mechanical engineers.
And the first homework is like, on an Ubuntu machine.
I didn't know what Ubuntu was, I didn't know what Linux was. And it was
like, oh, run this program on this operating system that I've never heard of.
And I think we were both stunned into a sad silence
of like, we have no idea how to even run this program. It's an
entirely new language. Right. When a mechanical engineer tries to learn robotics at first,
how would you guide them past that? Yeah, well, I think now in the age
of large language models, use that to your advantage.
I think people now are using ChatGPT to like write code for them, which I
think is great. But I think you want to make sure that if you're using
ChatGPT to write code because you're not familiar with code, you don't want to replace
the learning opportunity to actually learn how to code. There's still an important
learning process. The end goal should be, I will learn how to code,
and I will learn how to code as A mechanical engineer who wants to get
into robotics. And it's not like I will get ChatGPT to code for me having
no idea how to actually do that myself. So I think using large language models,
using ChatGPT as a learning coach to guide you through these processes.
I think if we had that when we were taking the class, it would have
been way easier. We could be like, where do I even start? And explain like
the basic concepts to me. You had a fairly dedicated path, but you didn't have
the most direct path. Right. If you were to do it all over again,
or if you were a student now starting in 2025, how would you structure your
career to pursue the things that you are most optimistic about? Yeah,
I think I took a very meandering path. I knew like nothing going
into Cornell about like engineering, so I didn't know until like end of senior
year that I was interested in robotics. And I'm kind of jealous when I look
at people around me now who like kind of had like a dead set,
like focus on robotics from like freshman year and like did everything
right and like worked in the top labs and they have a skill set
that I can't really compete with. Meanwhile in sophomore junior
year I was taking dynamics, fluid mechanics, things that don't direct apply to
like what I'm doing now. But I think where I ended up is where I
would want to have ended up anyway. So I got lucky that way. And I
feel like there is a little bit to be gained by kind of taking this
circuitous path because you kind of learn a little bit about these other things
that you might not apply on a day to day basis, but could be useful
especially in a field like robotics where like so many things apply.
So like if I have a little bit more knowledge about fluid mechanics as a
roboticist, well, suddenly if I'm now making a robot that's operating in
the ocean, I can now bring that expertise that I learned from fluid mechanics
to this one application. And so I think that you can always use it to
your advantage. And I think taking this circuitous path is a good way to find
out what you're interested in. Because not everyone's going to be interested in the same
thing and you kind of don't know what sparks your interest until you try
it. So I think actually taking the meandering path can be a good way of
dabbling in many things until something clicks. That's very encouraged. But it's not too late
for me. No, for you it's too late.
What's your take on the robotics industry. I feel like robotics,
it has proven to be harder than anyone thought it would be.
I mean, you go back and you see people like Elon Musk, starting from like
2014, saying that full self driving is going to come the next year.
Elon was like the most vocal optimist with robotics,
but I think that that mindset and mentality was shared by many in the
field. I think part of it was kind of marketing. Like everyone wants to be
like, yes, my robot will be ready next year, but I think a lot of
it is genuine because everyone was surprised by ChatGPT that
it could write poetry all of a sudden. And no one thought it would be
ready at that point. Many people thought useful robots that can do things around the
house, like laundry or fold your bed, would be an easier problem to
solve than generating a painting or things midjourney can do.
And so I think it's surprised people how hard robotics
is. But I think in this, you know, era now of machine learning,
AI and data, we are positioned to potentially see a reawakening
of robotics. I think robotics industries have proved
super hard to be profitable. You have very few companies that
have robots that people want to buy and are useful enough. So like Roomba,
a vacuum cleaner is like one of the few examples of like a robot that
people will actually buy at this point, which is a very telling case of how
difficult robotics is. Everyone now is talking about this flywheel, where in
this era of AI and ML, it's kind of just a matter of getting the
data. ChatGPT works because it was trained on an Internet scale of
data. And so for robotics, a lot of people think now that it's
just a matter of finding that Internet scale or that Internet size of
data for robots. So what might that look like? It might look like simple robots
operating in the real world, like a Roomba or some of these other robots that
are now not just only operating in the world, but collecting data that can
be used to train future iterations of the robot. So you can
imagine this flywheel where you have these relatively simple robots that work well enough
that people will buy them, do relatively simple tasks in the house, but collect
tons of data, and you can use that data to train the next iteration of
robots that are a little bit more useful, a little bit more general, and more
people will buy those. And you now generate this self fulfilling flywheel where
robots are now self improving in this way because the first generation
of robots are making the second generation smarter. And Tesla's kind of doing
this with full self driving now, where they're collecting data of humans driving their
cars kind of for free. Data is inherently hard to generate and it's hard
to get free data unless you're just downloading the Internet, especially for robotics. And so
Tesla's found this really useful niche where they can get people to
collect data for them and use that to improve the next generation of robotics.
So I think robotics is going to become all about data in this
age of AI and I think people who can find ways of getting there in
with maybe an initial simple robot that's just useful enough for
people to buy it and then use that to generate tons and
tons of data that can then be used to make the next generation more useful.
I think is going to be the future. Is this data or are these models,
robot architecture like degree of freedom specific,
or will they be generalizable across different robots,
say like Roomba to humanoid? Yeah, I think that's a big question. And I think
if you look at papers coming out now, there is talk of these foundation models
analogous to LLM, which are like kind of these foundational models.
That just means that these things generalize across many different systems or
environments. And so in robotics they're talking about these foundational robotics
models that generalize or were trained cross embodiment. Cross embodiment
just means you have many different robotic platforms that you collect data with.
And rather than a couple years ago it was all about training
specialized robots, collecting data just for one type of robot.
But now it's becoming more popular to collect data across many different
robotic platforms, many embodiments of robots and use that to train
these generalist models. And the idea is some concept that might generalize across
many different robots could be learned by training with large amounts of data
that span these many different robots. For instance, object permanence, things like this
idea that like objects, though they might go out of your view, still exist
and come back into your view are things that robots have to learn and are
not robot platforms that specific? And you can kind of collect data across many
different robots to learn these kind of common sense principles. Thanks so much for bearing
with me through doing this at 1am for bailing me out. It's three hours
past my bedtime. Yes, but anything for you. Thank you,
Jesse. Anything for robots. I'm just trying to replace your job.
Specifically yours will be the first to go.