Amazon Isn't Lying to You. It's Just Not Telling You the Whole Story.
There's a story from the Second World War that explains more about Amazon selling than most Amazon courses do.
The Allied air forces had a problem: bombers coming back from raids full of bullet holes. So they did the sensible-looking thing and mapped where the holes were — wings, fuselage, tail — planning to add armour to the most-hit areas. Then a statistician called Abraham Wald pointed out they had it exactly backwards. The planes they were studying were the ones that came back. The holes on those planes marked the places a bomber could take damage and still survive. The parts with no holes on the returning planes — the engines, the cockpit — were the places where a hit meant the plane never came home to be studied at all. Armour the gaps, said Wald. Armour where the survivors aren't hit.
That's survivorship bias, and it is quietly running — and ruining — a huge number of Amazon decisions. Because on Amazon, just like in that hangar, you spend all day staring at the planes that came back.
The data isn't lying. You're only seeing the survivors.
Here's the claim I want to make, and then immediately soften, because the softening is the actual point. It's tempting to say "Amazon is lying to you" — it makes a great headline. But it's not true. The data Amazon gives you is fine. It's accurate. It's not lying.
The problem is that it's incomplete, and incomplete in a specific, dangerous way: it shows you what happened, not what didn't. Your dashboard shows you the searches you appeared in, the clicks you won, the orders you got. It does not show you the searches you never appeared in, the customers who wanted something like yours but never saw you, the demand that quietly went to a competitor because you weren't in the room. You're looking at the returning planes. The ones that didn't come back — the sales you never made, the market you never reached — leave no mark on your dashboard at all.
So when a seller pours more budget into the keywords that are already winning, they're doing exactly what the air force nearly did: reinforcing the parts that are already fine, because those are the parts they can see. The opportunity is almost always in what the data doesn't show — and no amount of staring harder at the dashboard reveals it, because it was never on the dashboard to begin with.
Why more data isn't the answer
The instinct, once you accept the picture is incomplete, is to go and get more data. More tools, more dashboards, more metrics. And more data helps a bit. But it doesn't solve the problem, because the missing piece isn't another metric — it's interpretation. It's the human act of looking at the numbers and asking the questions the numbers can't ask themselves. Why did this spike? What's my competitor doing that would explain this? Is there something outside Amazon entirely — the season, the weather, the news — that's moving this? Where's the demand I'm not capturing?
Those are questions, not fields in a report. A dashboard can tell you sales rose 40% on Tuesday. It cannot tell you it was the sunniest week in a decade and every gardening product on earth just had a good week — that connection lives in a human head or nowhere. The data gives you the what. Only judgment gives you the why, and the why is where every real decision actually gets made.
The edge is judgment, and judgment is graft
So here's the whole thesis, and it's not a comfortable one for anyone hoping for a shortcut. The edge in Amazon isn't the data — everyone has the same data. It isn't the tools — everyone can buy the same tools. The edge is trained human judgment applied to the data: the experience to know which questions to ask, which spikes to distrust, which contextual factor probably explains the thing the dashboard is presenting without explanation.
And that judgment isn't magic fairy dust. There's no secret source, no clever hack that installs it. It's built the slow way — through hard work, through grind, through making calls and watching how they turn out and getting better over time. It's the least sexy answer in Amazon and the only true one: the operators who win are the ones who look at the same data as everyone else, apply better judgment to it, and keep sharpening that judgment year after year.
Don't go data alone — it's an incomplete picture and it'll lead you to armour the wings while the engine goes unprotected. Look at the data, absolutely. Then apply your judgment, and let the two together guide you. Do that consistently and you get better at it, and better, until you're not the plane full of holes limping home — you're the operator who worked out where the real damage was happening, and armoured it, while your competitors were still reinforcing the parts that were never the problem.
FAQ
What is survivorship bias, and why does it matter on Amazon?
It's the mistake of only looking at what made it through and assuming that's the whole story. Your dashboard shows the searches you won, the clicks you got, the orders that came in — the planes that came back. It doesn't show the demand you never saw, or the sale that quietly went to a competitor instead. The data's not wrong. It's just missing everything that didn't survive long enough to get measured.
Why doesn't just adding more data fix this?
Because the problem was never a missing metric. It's missing interpretation. A dashboard will happily tell you sales jumped 40% on Tuesday. It won't tell you it was the sunniest week of the year and every gardening brand on Amazon had a good week too — you have to make that connection yourself. Bolt on ten more tools and you'll still miss the thing the data was never going to show you, because someone has to ask why, and no report does that on its own.
So what's the actual edge on Amazon, if everyone's staring at the same numbers?
Judgment. Nothing shortcuts it. Everyone has access to the same data and mostly the same tools — what separates people is knowing which spikes to trust, which to ignore, and what's happening that the dashboard was never going to catch. You build that by making calls, watching how they turn out, and doing it again. Not glamorous, but it's the one answer that actually holds up.
About the author
Zamir Cajee is co-founder of This Way Up, a UK business specialising in Amazon marketplace strategy, and co-host of The Upside Podcast, where he and the team break down how Amazon actually works — and how it lies to you. Zamir has built multi-million dollar businesses from scratch and has been selling into the EU since 2016.
For a practical way to catch this bias in your own decisions before it costs you, see Survivorship Bias Is Quietly Wrecking Your Amazon Decisions.
Ready to take your Amazon Store to the next level? Start by exploring the tools available to create a standout Storefront or make the process even easier by reaching out to us for tailored guidance.
Subscribe to the This Way Up newsletter for impactful strategies and insights, or tune in to The Upside podcast for expert advice on successfully managing your e-commerce efforts.