Zillow Offers bought 7,000 homes at prices a black-box model promised would hold. They didn't. The lesson isn't "AI is bad." It's that the kind of AI you trust with reporting matters enormously.
In November 2021, Zillow announced it was shuttering Zillow Offers, its much-hyped algorithmic home-buying arm. The company had spent the previous year using its proprietary Zestimate model to make instant cash offers on homes across 25 US metros, betting that a sufficiently good AI valuation tool could outperform human appraisers at scale. The bet did not pay off. Zillow had bought roughly 7,000 homes for prices the algorithm now agreed were too high, and the resulting write-downs eventually totaled more than $500 million in Q3 and Q4 2021 alone. The total program loss exceeded $900 million. Zillow laid off 25% of its workforce in the cleanup.
The technical version of the story is well-documented. Zillow's Zestimate had been trained on years of housing-market data, much of it during a period of strong, stable price growth. When the pandemic-era housing market shifted, first overheating, then cooling rapidly in mid-2021, the algorithm kept extrapolating the patterns it had learned, ignoring the regime change. According to Stanford Graduate School of Business analysis, the model continued to assume strong price appreciation in markets that were actively cooling. Zillow Offers was buying homes at prices that already exceeded what the market would pay, and doing it at industrial scale.
What's less discussed is the organizational dynamic. In early 2021, as part of an initiative internally known as "Project Ketchup," Zillow's pricing experts were explicitly instructed to stop overriding the algorithm's estimates. In some cases, the company would raise the algorithmic price by thousands of dollars to win more deals. Human judgment was deliberately removed from a loop the algorithm was not actually competent to close. By the time anyone could course-correct, Zillow owned thousands of homes priced above what anyone would pay.
The Zillow story is usually told as a parable about AI in real estate, but the deeper lesson is about what kind of AI to trust with what kind of decision. The Zestimate is a black-box predictive model. It produces a number. It cannot explain why. It cannot show its working. When the world it was trained on stopped being the world it was operating in, nobody on the team could see the seam. They could only see the number, which kept saying "buy."
There are two very different kinds of "AI" used in analytics products, and confusing them costs money.
The first is predictive AI: trained models that produce estimates, forecasts, or scores. Zestimate is one. Most "AI-powered forecasting" features in BI tools are another. These systems learn patterns from historical data and project them forward. They're opaque by design: the answer comes out of a tangle of weighted features that nobody, including the people who built it, can fully audit. They work well when the future resembles the past, and they fail invisibly when it doesn't.
The second is an AI that reports your real data and shows its working: a language model that queries your actual warehouse, answers the question, and ties every number back to the exact query that produced it. This system isn't "predicting" anything. It's reporting. And because every figure traces to a query you can open and read, the reporting is reproducible: same data in, same numbers out, every time.
The Zillow disaster is a predictive-AI failure dressed up as an analytics failure. But the same risk profile shows up in any analytics tool that lets the language model "estimate" things instead of compute them. If a tool tells you your conversion rate is 3.2%, you want that number to be the output of a deterministic query against a known data source, not an LLM's "estimate" based on patterns in its training data and a few rows of context.
Askalytics' AI queries your real warehouse, and every number traces back to the query that produced it. No black-box guesses. No "estimated" conversion rates.
The Askalytics architecture is built around a simple rule: every number you see can be traced back to the exact query that produced it. The AI queries your own data warehouse directly and runs the queries it needs, but it doesn't estimate the conversion rate from a black box. It reports the figure your data actually returns, and shows you the query behind it. You can open the receipt for any number, the Advanced Breakdown, and see the query, the rows, and the source it came from. Run the same query twice on the same data and you get the same number twice. Run it on data from yesterday versus today, and the difference reflects only the difference in the data.
The AI is reporting, not predicting. It doesn't guess what the conversion rate is. It runs the query, returns what the query returns, and the figure can be explained line-by-line and reproduced exactly. On top of that, every AI Insight is re-checked by a second, different AI model.
This is a deliberate trade: the numbers you see are computed, auditable, and reproducible, not the output of a black box you have to take on faith. The customer-facing surface stays clean: real queries against your real data, reproducible results, no Zestimate-style black boxes between you and your numbers.
Zillow's $500 million loss is what happens when an organization puts a black-box prediction at the center of an irreversible decision loop. It's an extreme version of the same risk every team takes when it trusts an AI analytics tool to report numbers without showing where they came from. The fix is structural: report the numbers your data actually returns rather than estimating them, tie every figure to a query you can open and read, and have a second AI re-check the conclusion before it ships.
For more on this approach, see how Askalytics handles evidence-grounded analytics and the Confidence Check that backs up every insight.