I get pitched on new AI features constantly. At least a dozen times a week, a founder will tell me they’re adding some new AI-powered capability to their product. My first question is always the same: “Why?”
More often than not, the answer is some version of “Because everyone else is doing it.” That’s a terrible reason to do anything in business, but it’s an especially bad reason to get into AI. The AI landscape is a minefield right now. It’s full of hype, misinformation, and snake oil salesmen. If you’re not careful, you can waste a lot of time and money building something that nobody wants.
I’ve seen this movie before. I saw it with the dot-com bubble, the mobile app explosion, and the crypto craze. Every new technology wave brings a gold rush of opportunists looking to make a quick buck. But the companies that last are the ones that focus on solving real problems for real customers. The rest are just noise.
That’s why I’ve developed a simple framework for evaluating new AI features. It consists of three questions that every founder should ask themselves before they write a single line of code. These questions have helped me filter out the noise and focus on the opportunities that have real potential. They’ve also saved me from making some very expensive mistakes.
1. Are you solving a real problem or just chasing the hype?
This is the most important question, and it’s the one that most founders get wrong. They get so caught up in the excitement of a new technology that they forget to ask themselves if they’re actually solving a real problem for their customers. They build a cool piece of tech, and then they go looking for a problem to solve with it. That’s a recipe for disaster.
The best companies start with the customer. They obsess over their customers’ needs, and they work backwards from there to find the right solution. Sometimes that solution involves AI, and sometimes it doesn’t. The technology is a means to an end, not the end itself.
One of my portfolio companies, a vertical SaaS business, is a great example of this. They were feeling the pressure to add AI to their product. Their competitors were all launching AI-powered features, and their investors were asking them what their AI strategy was. They were about to jump on the bandwagon and build a generic AI chatbot, but then they took a step back and asked themselves what their customers actually wanted.
They spent a few weeks talking to their customers, and they discovered that their customers’ biggest pain point was not a lack of information, but a lack of time. They were drowning in data, and they didn’t have time to make sense of it all. So instead of building a chatbot, the company built a tool that used AI to automatically analyze their customers’ data and surface the most important insights. It was a huge hit. It solved a real problem for their customers, and it gave them a huge competitive advantage.
Contrast that with another company I know of that built a “generative AI” feature for their e-commerce platform. It was supposed to automatically write product descriptions. It was a cool piece of tech, but it didn’t solve a real problem. Most of their customers were happy to write their own product descriptions, and the ones who weren’t could just use a cheap freelancer. The feature got a lot of press, but it didn’t move the needle on any of the company’s core metrics. It was a solution in search of a problem.
2. Is your data a moat or a liability?
Data is the lifeblood of AI. The more data you have, the better your models will be. But not all data is created equal. If you’re building an AI product, you need to have a unique, proprietary dataset that you can use to train your models. If you’re just using the same public datasets that everyone else is using, you’re not going to have a sustainable competitive advantage.
I like to use the analogy of a moat. A data moat is a competitive advantage that you have because of the data you’ve collected. It’s something that your competitors can’t easily replicate. For example, Google has a huge data moat because of all the data they’ve collected from their search engine. They use that data to train their models and make their search results better. It’s a virtuous cycle that’s very difficult for anyone else to compete with.
If you don’t have a data moat, you’re going to be in a tough spot. You’ll be competing with everyone else who’s using the same generic models and datasets. It will be a race to the bottom on price, and the only winners will be the big cloud providers.
But data can also be a liability. If you’re collecting sensitive customer data, you have a huge responsibility to protect it. A data breach can be catastrophic for your business. It can destroy your reputation and lead to massive fines. That’s why you need to be very thoughtful about the data you collect and how you store it. You need to have a robust security and privacy plan in place from day one.
I’ve seen too many startups get this wrong. They collect a bunch of data without thinking about the consequences, and then they get hacked. It’s a completely avoidable mistake. If you’re not prepared to be a good steward of your customers’ data, you have no business being in the AI game.
3. How will you measure success?
This is the question that separates the amateurs from the pros. The amateurs get excited about vanity metrics like the number of users or the amount of press they’re getting. The pros are obsessed with the metrics that actually matter: revenue, retention, and customer satisfaction.
If you’re building an AI feature, you need to have a clear idea of how you’re going to measure its success. You need to be able to tie it to a core business metric. Otherwise, you’re just flying blind. You won’t know if you’re making progress, and you won’t be able to justify the investment to your board.
Let’s say you’re building an AI-powered recommendation engine for your e-commerce site. How would you measure its success? You could look at the click-through rate on the recommendations, but that’s a vanity metric. A better metric would be the increase in average order value or the decrease in customer churn. Those are the metrics that actually matter to your business.
I always push my portfolio companies to be rigorous about this. I want to see a clear ROI for every new feature they build. I want to know how it’s going to move the needle on the metrics that matter. If they can’t answer that question, I tell them to go back to the drawing board.
Don't Be a Lemming
The pressure to add AI to your product is intense right now. It feels like if you’re not on the AI bandwagon, you’re going to be left behind. But the worst thing you can do is to blindly follow the herd. That’s how you end up driving your business off a cliff.
Instead of chasing the hype, I encourage you to be thoughtful and strategic. Ask yourself the three questions in this article. Be honest with yourself about the answers. If you can’t come up with good answers, that’s a sign that you’re not ready to get into the AI game. And that’s okay. It’s better to be a fast follower than a failed pioneer.
But if you can answer these questions with confidence, then you might be onto something. You might have a real opportunity to build a product that creates lasting value for your customers and your business. And that’s a lot more exciting than just chasing the latest fad.
Frequently Asked Questions
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.