I’ve never been good at stopping what I’m doing. AI has made this problem even worse.
Near the end of a recent workday, I was looking forward to closing my laptop and enjoying a relaxing dinner with my family. I had a few minutes free around 4:30, so I decided to check on a dormant project: a pile of prototypes and old data I’d been putting off for months. I opened up the files and threw them into Claude Code to see what was cooking.
Seven hours later, I was hunched over my keyboard with bloodshot eyes. And I was really hungry.
What happened? It turned out that the dusty pile was a gold mine. I spent hours poking and prodding it with Claude Code, trying to turn it into something useful.
But somewhere along the way, I wandered off the track. I somehow ended up building my own compiler to convert the data tables into a different format.
This was not the point of the project. All I wanted was a practical application. But instead I found myself at the bottom of a very deep rabbit hole.
It’s just one example of the danger posed by generative AI. It has altered our perceptions of what we can do, how quickly we can do it, and whether we should delegate it.
As a CEO, this can be lethal to your workflow. I know there are many other executives who feel the same way.
I’ve figured out a couple of ways to keep from drowning. I hope they can be useful to other leaders.
Fast vs. slow thinking
First, it’s important to understand what’s happening to us. Daniel Kahneman’s fast and slow thinking framework is helpful here.
The fast thinking system is reactive and confident. It’s useful when you’re doing things you’ve already done many times and have gotten helpful feedback. You don’t need to look behind you anymore to be sure you’re doing it right.
Fast thinking is great for times when you simply need to execute. But the problem is that if you aren’t actually as good at something as you think you are, it can lead to oversights and mistakes. And maybe more important, it crowds out your ability to think more slowly.
Slow thinking is harder. But it’s necessary. You dial in to what you’re doing and shut out the rest of the world. CEOs need to be doing more slow thinking if they’re going to solve their toughest problems.
Beware the rabbit hole
CEOs tend to be energetic, impatient. We just want to get stuff done.
As you might expect, that tips us toward defaulting to fast thinking. AI has only amplified this tendency.
With vibe coding at our fingertips at any moment, we can easily be nudged into “just one more try.” The LLMs always make us feel like we’re right on the edge of nailing our objective.
Agents are not designed to give us honest feedback. They want to push us forward and keep burning tokens. It makes us feel confident that whatever we’re building is going to be great, and it further entrenches us in fast thinking mode.
When using AI, my shift into fast thinking is so automatic that I usually don’t notice it happening. That means I need automatic systems in place to fight fire with fire, so to speak.
It’s fun to see an AI agent end an exchange with “Done.” But there is a well known difference between what an agent means by “Done” and really being done. So now, before I start working with an agent, I write down what done means to me. Every time. One or two sentences, or sometimes a whole page.
That creates a natural boundary. It stops me from acting like a marathon is a sprint. It changes working with agents from a series of dopamine hits to progress toward some goal. And I never miss dinner anymore.
Sometimes our natural drive kicks in and we slip past the boundary. And that’s okay. But when you realize it’s happening, you have to take action before you get pulled back into a AI rabbit hole.
For me, that means stopping and figuring out who on my team to hand the project off to. You should be hiring people who have the expertise and intelligence to take over your abstracts with only minimal instructions. Otherwise you’ll convince yourself that you and only you can finish what you started.
Once you’ve delegated, your role is to act more like a searchlight than a spotlight. Check in periodically from the outside, but don’t get too absorbed. Your main value is in asking questions from a distance that help clarify and guide.
An AI nanny
That dormant project I mentioned earlier didn’t end with the compiler. Claude suggested I could build a diagnostic tool to tell me how long I could trust my data. When I ran my data through the tool and it found no errors, I called it a win.
But thinking it over later, I realized: Just because the tool didn’t flag a problem, was that proof nothing was wrong with the data? What if there was something wrong that the tool’s algorithm couldn’t detect?
As a leader, you have to realize that you don’t know everything. And even if you did, you don’t have the time to look everything over.
Because of this, you need to apply healthy skepticism to whatever your agent is telling you. Your agent doesn’t have a human brain; it’s built to fit curves to data. You can’t independently trust its judgment.
The best way to overcome this problem is to design validators that check and test whatever you’re using AI to generate. These checkers need to be independent of your agent. AI will gladly write a test it knows it can pass and tell you, “We did it!” And if your agent can touch the checker, it might try to modify it.
Our team built several different validators, testing different approaches to make sure our validators were using principles based on real physics instead of just showing us answers we wanted to see. Now, with these validators in place, we can let our AI agents roam freely because we know they can’t pass any tests by accident.
Building these tests took longer than building the search. That’s what told me we were on the right track.
Look for the unknown unknowns
But even well-designed validators can’t anticipate everything. As CEO, your job is to look for the unknown unknowns, because no one else is looking at the big picture as comprehensively as you are.
Some AI failures don’t just waste time; they can do real, lasting damage.
I once delegated some multi-agent work to one of our senior engineers, who brought on an intern to help.
The intern was exploring whether Claude could handle code reviews. At some point an email came through asking for permission to connect Claude directly to our production GitHub repository.
I caught it. If I hadn’t and permission had been granted, whatever that process did could have exposed our entire source code to the world.
The instinct to treat an AI agent like a trustworthy junior colleague is understandable, but it’s wrong. A human (usually) understands their limitations, and even if they don’t, they have to face consequences for their actions. This creates a natural limitation on their risky behavior.
The job of every leader is to apply as much creativity to “What could go wrong?” as we normally do to “What could go right?”
In software, we often use the term blast radius. For everything we do, I try to get my team to think in terms of potential harm right at the outset. It’s the mindset a medical device company has to take, but all companies can benefit from it.
In these premortems, we imagine something went horribly wrong, and then ask: Why? I pose a lot of open-ended questions, and then try to stay out of the room as much as possible. The power imbalance of a founder present can make honest discussions difficult. You can pretend that everyone on your team feels safe telling you something will explode, but most of the time they’re more concerned about not pissing you off.
I give my managers the autonomy they need to get to the truth. And I reinforce our company’s credo: To succeed here, you need to challenge everything you don’t understand or think is wrong. They have to know that we want them to find trouble before anyone gets hurt. I’ve found that most scientists thrive when they don’t feel like they’re living in a Dilbert comic.
Sometimes I do need to be in the room. When I am, I keep my CSO hat on and let everyone know that. This isn’t so hard when the discussion is science, and ground truth takes precedence over power imbalances.
Deputize smart people
For many executives, the above advice will cut down their AI overload. But for more complicated businesses, AI is simply producing too much output for them to keep up with.
In these cases, you need to start thinking about instituting a layer of decision-makers who sit between you and the AI skunkworks teams.
These deputies should have your trust, a track record of sound judgment, and strong organizational skills. Their job is to filter prototypes, surface potential gaps in verification, and explain candidate projects to you simply. They should have the authority to shoot things down without your approval.
Their goal is to cut down your decision-making surface area, so you’re weighing in only on what’s real and tested.
Don’t let AI replace your thinking
I’m convinced that AI will replace us only if we let it. As executives, our judgment and our experience got us where we are. I’d wager that these qualities are even more important today than they were five years ago.
There are seemingly unlimited opportunities in front of us. Our ability to focus our resources on the most promising ones will determine who survives in the AI epoch.