After working in inference optimization for five years, I'm sharing the unexpected lessons I've picked up. This is about practical results and the trade-offs most people don’t talk about.
What I've Learned From 79 Companies
Across 234 angel investments, including Anthropic, a few patterns in LLM inference come up again and again.
The founders who handle LLM inference well understand that most founders overthink this and underspend on execution.
The Reality Nobody Talks About
A lot of the advice on LLM inference rests on assumptions that made sense five years ago. The world has moved on.
The first thing to understand is that simplicity beats complexity every time. I've seen this play out across dozens of companies. The pattern is unmistakable.
Many teams spend months on the wrong path before realizing that the data tells a different story than your gut. Once they make the switch, everything changes.
Lessons From the Trenches
I want to share a few specific lessons I've picked up over the years. These aren't theoretical. They come from real companies, real failures, and real successes.
Lesson 1: The best time to start thinking about LLM inference was yesterday. The second best time is now. Don't wait until you have the perfect plan.
Lesson 2: Hire for attitude, train for skill. The best people at LLM inference are rarely the most technically gifted. They are the most curious and persistent.
Lesson 3: Your competitors are probably getting this wrong too. That's your opportunity. While everyone else is following the same playbook, you can zig when they zag.
This connects to broader themes around token economics, model distillation, inference optimization that I've been thinking about a lot lately.
Wrapping Up
I've shared a lot here, and I know it can feel overwhelming. But you don't need to get LLM inference right on day one. You just need to get started and keep improving.
Founders who do well at LLM inference tend to share one trait: they're relentlessly practical. They don't chase perfection. They chase progress.
That's the mindset I'd encourage you to adopt. Start where you are. Use what you have. Do what you can. And keep pushing forward.
As always, I'm rooting for you.
Frequently Asked Questions
Q:How long did it take to see results?
Most meaningful business results take 3-6 months to materialize. Anyone promising overnight success is selling something. The companies in my portfolio that grew fastest were the ones that stayed patient and consistent.
Q:What would you do differently looking back?
I'd move faster on the things that were working and cut the things that weren't sooner. Most founders, myself included, hold onto failing strategies too long because of sunk cost. Speed of learning is everything.
Q:What was the biggest challenge in this case?
Almost always, the biggest challenge is people and alignment, not technology or strategy. Getting the right team focused on the right problem is harder than any technical challenge I've encountered.