Building an AI startup is anything but glamorous. I want to take you behind the curtain and share the unfiltered reality of our journey. From the heated debates over our roadmap to the bug that almost derailed our launch, this is the real story of what it takes to build and ship an AI product.
What I've Learned From 106 Companies
Across 234 angel investments, including Anthropic, a few patterns in A/B testing come up again and again.
The founders who handle A/B testing well understand that the market doesn't care about your roadmap.
The Counterintuitive Truth
In A/B testing, the best practitioners often do less, not more.
The lesson: simplicity beats complexity every time. It sounds simple. It's incredibly hard to execute.
The Framework That Actually Works
Here is a simple framework for evaluating A/B testing. It's not complicated, but it requires discipline.
Step 1: you need to move fast and break things
This is where most people go wrong. They skip this step entirely and jump straight to execution. Don't do that.
Step 2: most founders overthink this and underspend on execution
Once you have the foundation right, this becomes much easier. I've watched founders struggle with this for months when the answer was staring them in the face.
Step 3: Iterate relentlessly
Nothing works perfectly the first time. The companies that get A/B testing right treat it as an ongoing process, not a one-time project.
Real Talk: What Actually Matters
Here is what actually matters in A/B testing.
First, execution speed beats perfection. Every time. Companies rarely fail because they moved too fast on A/B testing. Many fail because they moved too slowly.
Second, measure everything. If you can't measure it, you can't improve it. Set up tracking from day one, even if it's basic.
Third, talk to your users. It sounds obvious, but many founders make decisions about A/B testing in a vacuum. Get out of the building. Talk to real people.
This connects to broader themes around a/b testing ai, growth hacking, case study that I've been thinking about a lot lately.
Final Thoughts
There are no shortcuts in A/B testing, but there are smarter paths.
The strongest teams treat A/B testing as a competitive advantage, not a checkbox. They invest in it early, measure it obsessively, and never stop improving.
If you're just getting started with A/B testing, don't be intimidated. Everyone starts somewhere. The key is to start with the right mindset and the right framework, and then execute like your company depends on it. Because it probably does.
Frequently Asked Questions
Q:How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.
Q:What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
Q:Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.