I’ve seen more AI pitches than I can count. As an investor in over 200 companies, including some of the foundational players like Anthropic and OpenAI, my inbox is a constant stream of “groundbreaking” models and “paradigm-shifting” algorithms. Most of the time, I can sniff out the BS in the first five minutes. But every once in a while, something lands on my desk that’s already out in the wild, making money, and has everyone talking.
That’s what happened last month. A founder I know, who’s deep in the fintech world, pointed me to an AI trading bot that’s been gaining a massive following. The marketing was slick, the promised returns were astronomical, and it was wrapped in a narrative of democratizing Wall Street for the little guy. Through a source, I got my hands on the complete source code. I couldn’t resist. I had to see if the engine matched the paint job.
I spent a weekend tearing it apart. What I found was a fascinating, and frankly, disturbing mix of genuine innovation and outright deception. It’s a perfect case study in the current state of applied AI, and I’m sharing my full analysis because everyone—founders, investors, and users—needs to know what’s really under the hood of these black boxes.
The Good: Some Truly Brilliant Engineering
Let’s start with the positives, because there were some. The data ingestion and feature engineering pipeline was genuinely impressive. Most of the off-the-shelf bots I’ve seen just plug into a generic market data API and maybe a news feed. These guys went several steps further.
They had built a sophisticated system for parsing alternative data, including things like satellite imagery of port activity and credit card transaction data. The clever part was how they used a transformer model to create embeddings from this unstructured data, which were then fed into the main trading model. For example, they weren't just looking at Apple's stock price; they were correlating it with real-time data on foot traffic in their stores. That’s a real edge.
It reminded me of a problem we tackled at one of my previous companies. We were trying to predict user engagement, and we found that the most predictive signals weren't the obvious ones. It was the subtle, second-order data that really mattered. The team behind this bot understood that. They had a real knack for finding signal in the noise. There was a spark of genius there, no doubt.
The Bad: Where the Magic Show Began
My admiration started to fade when I got to the core of the backtesting module. The performance charts on their website were flawless—a beautiful, steady upward curve that would make any investor drool. But as the old saying goes, if it looks too good to be true, it probably is.
The first red flag was lookahead bias. It’s a classic trap. Their model was being trained and tested on data it couldn’t have possibly had in a live trading scenario. For instance, they were using closing prices to make decisions at the start of the day. It’s like betting on a horse race after you already know which horse won. It guarantees a perfect result, but it’s completely meaningless in the real world.
I’ve seen this mistake made by inexperienced teams, but this felt different. It was too perfectly implemented to be an accident. They had systematically built their backtester to produce flattering, but fake, results. This wasn't a bug; it was a feature.
The Ugly: It’s Not AI, It’s a Script
The final nail in the coffin was the “AI” itself. The marketing materials were filled with buzzwords like “self-evolving neural networks” and “dynamic strategy adaptation.” I was expecting to find a complex reinforcement learning model or at least a sophisticated ensemble of networks.
What I found was a giant if-then-else statement. A script.
Seriously. The core trading logic, the part that actually decided to buy or sell, was a series of about 500 hard-coded rules. Things like:
- IF the VIX (volatility index) is above 30;
- AND the 10-day moving average crosses below the 50-day moving average;
- THEN sell 50% of all tech holdings.
That’s it. The brilliant data pipeline I saw earlier? The embeddings were being fed into this system, but they were only one of dozens of variables in a rigid, rule-based system. The “AI” was just a marketing wrapper around a strategy that a day-trader from the 90s would find familiar. It’s a glorified spreadsheet macro being sold as Skynet.
This is more than just questionable practice. It’s a lie. They are selling a dream of advanced artificial intelligence that’s making decisions, when in reality, it’s a simple, brittle script that’s probably going to fall apart the second the market behaves in a way the developers didn't anticipate.
My Takeaway: Trust, but Verify
Why does this matter? Because this single bot is a symptom of a much larger disease in the tech industry. The pressure to have an “AI story” is immense. Founders are slapping the AI label on everything because they know it gets investor attention and customer dollars. And it works.
As an investor, this experience reinforces my core philosophy: trust, but verify. I don’t care how slick your pitch deck is or how many buzzwords you use. Show me the code. Show me the data. Let me see the engine, not just the paint job. I’ve passed on dozens of companies with amazing stories because the underlying tech was just a facade.
For anyone using these platforms, my advice is to be skeptical. Be very skeptical. Ask hard questions. What kind of AI is it? How is it tested? What happens in a market crash? If the answers are vague and full of jargon, run. The future of finance will absolutely be shaped by AI, but the road to get there is filled with mirages like this one. Don’t get lost in the desert.
Frequently Asked Questions
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.