Today I resigned from GitHub after spending almost four years there and
growing to senior engineer. I left to help improve AI safety because
there is a dangerous trend in how AI coding agents are being deployed right
now and this is going to affect you personally. I'm going
to break down what I observed, why it made me walk away, and what
I'm doing about this problem. So first of all, I didn't
leave GitHub because I hated my job. It was genuinely the best job
I've had so far. I built the first version of newer respectful AI
support systems with my team, helped other engineers across the org
figure out how to build mature LLM solutions, and those experiences
shaped who I am as an engineer today. I've also experienced the birth
of agentic coding from more traditional AI autocomplete to newer
agentic systems like Claude Code, GitHub, Copilot and others.
Now we as programmers and non programmers experience the benefits of agent
decoding daily and it has been a massive productivity boost.
But over the past year I started noticing an extremely dangerous
trend across the wider industry that I just couldn't ignore
anymore. Companies are not just using AI for code
generation, which I'm a huge fan of, just like many of you. They are now
replacing entire parts of the software development lifecycle with AI
agents. Review testing, deployment decisions
and even architectural choices are being handed
to agents with less and less human oversight. And I'm
not only talking about small startups here that can afford this
risk. The industry pressure has become so high that
companies with millions of users are shipping code that a
human barely touches before it goes into production.
Now, before you misinterpret me, I actually just want to say that at GitHub
I found there to still be a lot of respect for good high quality software
every single day. It's just that the industry trend
overall is causing software quality to drop quickly.
There are already projects and platforms trying to test the limits of
how far we can push coding agents to automate everything.
And the direction here is clear. Some organizations are starting
to point at pull requests as an annoying bottleneck that
slows down their AI generated output. Now think about that for just
a second. The one remaining checkpoint where a human looks
at the code before it reaches your users. And the industry is calling that a
bottleneck. The ultimate step here seems to be to displace
human review altogether. So why does
this problem make me quit my create job at GitHub? Well,
well, it has been one of the best jobs I've had and I will easily
admit this AI that I've used at my job very often writes
better code than me, and it outputs code faster
than me. One of the ways this has excited executives in the tech industry is
that it promises that every developer can be a 10x or even a 100x
developer, because 10x is not enough anymore. Now,
before we get too enthusiastic, think about what that actually means for
a moment. If one developer can generate 100
times the code, they can no longer meaningfully review what
was produced. And I'm noticing this every single day as I
work as an engineer. Instead of the industry seeing this as a problem,
the collective response has been that agents should just take care of reviewing
code as well. The critical issue with this is, is that AI
writes better code often, as I just said, but it does
not always write better code. It will continue to
hallucinate, it will continue to make mistakes, and when tasks are
long and complex enough, it will create incomplete solutions.
But it does always write more code than me. Way,
way more code. In a normal workday in the past,
I might have had days where I wrote a few hundred lines of code.
And now it's easy to write thousands of lines in a single session.
So what happens when you have an imperfect system generating
enormous amounts of almost perfect code?
You get a statistical guarantee that there will be bugs
in the system that you as a human, do not have the capacity to
check for anymore. This is not a gut feeling,
it is a mathematical fact. And the industry's
current response to this problem is to just throw more agents at
it. Oh, let's just evaluate agent output with more agents with just
slightly different system prompts. Let's write automated tests
with agents. Let's let agents test software end to end, stack more
AI calls with slightly different prompts on top of each other, and just
hope for the best. Now, in my brutal opinion, this is the
software industry's equivalent of, of just playing dollhouse and pretending
that there is a solution to this problem. Because we are currently trusting
the same type of technology that's creating the mistakes to
catch its own mistakes. Imperfect systems stacked
on imperfect systems do not converge to perfection.
The industry right now is actively choosing to
prioritize raw code output over everything else. The amount of
code generated can no longer be stopped by humans. And this is
not because of some fictional rogue AI like you might read in some fiction books.
No, this is just because the industry is letting this happen on purpose as
part of a strategic decision. Now, unlike many statements made
in the AI space, which are often based on hype or ulterior motives,
I am confident in stating the following the future
deployment of the fully autonomous AI software development lifecycle will
lead to an absolute increase in broken software.
Now I want to be clear about something because you might be thinking Xen,
do you think that we should be stopping AI coding altogether? No,
I'm not saying that we need to stop the point coding agents because this is
highly unrealistic and I'm quite progressive. And honestly, I never want
to go back to manually writing all my code myself anymore. And to be
honest, some drop in quality is acceptable because some bugs just don't
matter that much. Even in professional settings, not everything needs to be perfect.
No, nobody is going to get hurt if, say, a support portal button
breaks in an edge case condition. In general, being able to ship way
more and break a few small things along the way is a net win
for the majority of software out there. And I also understand that Vibe coding
is awesome. It's very cool to one shot your favorite childhood game and that as
a non programmer you can get into software easier nowadays. It's very nice.
The problem is that the push to use these agents is so pervasive
across every industry that bugs will inevitably appear. And in
software where we cannot afford it, in healthcare systems,
on financial platforms and infrastructure that millions
of people depend on every day. And without fundamentally
new forms of monitoring, AI coding agents will cause
more harm than good in these kinds of critical systems.
So it's a big problem, but what am I doing about it? Because it's
easy to complain from the sidelines, right? Well, that is what made me resign
from GitHub. It is very easy to just complain about problems that AI
is causing, but if an opportunity presents itself to actually contribute
to a possible solution, it is only right to try it, even if it
can be difficult. I happen to find an opportunity at a research
lab to work on exactly this problem. A new role
building monitoring systems specifically designed for AI coding agents.
Now next week I'll share the full details on this new role
on my LinkedIn. So if you want to be the first to know,
connect with me using the link in the description down below.
But I'm also sharing this story because four years ago I was starting
out as a junior and now I get to work on problems that I think
genuinely matter for the wider industry. And this shows that it is
still possible to get a high paid career while making
a positive change or at least trying to do so. And also genuinely loving
your job. And this is also why I push and help others towards their own
ideal AI career as a way to give back. I'll be happy
to answer any questions you have about the path that got me here. Make sure
to leave a comment down below and connect with me on LinkedIn.