--- title: "A Compliment Is Not Data" description: "Most customer interviews are flattering theater. Compliments, hypotheticals, and wishlists aren't data. Why interview discipline is the product, not speed." url: "https://luc.so/manifesto/" category: "Manifesto" date: "2026-06-29T19:06:21.702632" source: luc.so --- # A Compliment Is Not Data > **TL;DR:** Compliments, hypotheticals, and wishlists aren't data. Most customer interviews collect them anyway, then pass the warm feeling off as research. The craft that separates a real interview from flattering theater is small and unforgiving, and it breaks the moment you run a script faster instead of better. So discipline is the product. Lùc runs disciplined interviews at volume so the conversation itself stays honest. ## Most customer interviews are theater Someone asks "would you use this?" and the answer is yes. The room wanted a yes, people are kind, and a stranger has no reason to ruin your afternoon. The transcript comes back full of warmth. It looks like research. It decides a roadmap. It was theater. This happens because the interview measured the wrong thing. It captured how the conversation felt, not what the person has done. A pleasant hour with twelve people who all nodded is not evidence. It is twelve people being polite, recorded. The tell is simple. Read a transcript and ask: does any line here describe something this person actually did, with their own time or money, before I walked in? If the answer is no, you ran an opinion poll about your own idea. That is the most flattering data there is, and the least useful. ## The craft is small and unforgiving A real interview runs on a handful of rules. None of them are complicated. All of them are easy to break without noticing, which is why most interviews break them. **Ask what someone did, not what they would do.** The past is a record. The future is a guess people make to be agreeable. "Would you pay for this?" gets a yes. "What did you do the last time this problem cost you a deadline?" gets a story you can check. **Never put the answer inside the question.** "Don't you find the current tools frustrating?" is not a question. It is a request for agreement wearing a question mark. The respondent reads the answer you want and hands it back. This single move, [leading the witness](/articles/customer-interviews-without-leading-questions), corrupts more interviews than any other. **Treat a compliment as noise until a behavior backs it.** "I love this" is a verdict on you, not a fact about the world. It counts for nothing until it is attached to something the person already spent time or money to solve. **Follow the flinch, not the script.** The signal is rarely in the clean answer. It is in the small hesitation, the contradiction, the half-sentence they walk back. A good interviewer hears that and digs. A script marches past it to the next question. These rules sit at the center of [the Mom Test](/articles/mom-test-vs-jtbd), and Jobs-to-be-Done sharpens the same instinct from a different angle: find the forces that pushed someone to switch, and the moment they actually changed what they used. Both methods refuse the same thing. They refuse the hypothetical. ### The three kinds of bad data Fitzpatrick names three ways an interview lies to you, and they show up in almost every loose transcript. A **compliment** is praise with no behavior behind it. It feels like a win and proves nothing. A **hypothetical** is what someone imagines they would do in a future they have not lived. "I'd definitely use that" is a story about a stranger, told by the person sitting across from you. A **wishlist** is a feature request that names a solution instead of the problem underneath it. "You should add X" tells you what they think they want, not what is actually in their way. Each one feels like signal in the room. Each one evaporates the moment you ask what the person has actually done. ## Why agencies feel it hardest For an agency, interview hours are billable cost, not free internal overhead. That single fact bends the work. The math pushes toward a bad trade. Run fewer interviews to protect the margin. Hand them to whoever is free that week instead of your sharpest interviewer. Skip the depth and call a survey "research." None of these choices made the work better. They made teams do less of it, or do it worse, and ship the gaps to a client who paid for rigor. The bottleneck was never a quality problem you could solve by trying harder. It was an economics problem. When discipline costs hours and hours cap the project, discipline is the first thing the schedule cuts. That is also why [the count question](/articles/how-many-customer-interviews-are-enough) misleads people: the constraint that actually decides whether research is real is craft, not sample size. Ten leading interviews never reach saturation, no matter how many you run. ## The wrong fix is faster bad interviews There is an obvious fix on offer right now. Point an AI at your discussion guide and let it run. If the guide leads the witness, the AI now leads the witness at scale. If it asks people to predict their own behavior, it collects hypotheticals faster than any human could. Faster bad interviews are still bad interviews. Speed was never the problem. Discipline was. The same trap shows up one step further out, with [synthetic users](/articles/synthetic-users-vs-real-interviews): a model imagining what a customer would say is a hypothetical with a user interface. It skips the one thing that makes research real, which is a real person's actual past behavior. Volume on top of bad method does not average out into truth. It buries the truth under more noise. ## So discipline is the product This is the whole point, and it is the part most tools get backwards. The valuable thing is not running more interviews. It is holding every interview to the standard, then running that across more conversations than a team could staff by hand. Lùc asks about the last real project, not the imagined one. It reads each answer and probes where the answer is thin. It declines to lead, refuses the compliment, and will not let a pleasant conversation pass as evidence. It runs the volume in parallel to the researcher, so the human spends judgment on synthesis and on the conversations that actually decide something. This is not research at scale. Plenty of tools sell that phrase, and most of what it produces is noise at scale. This is the craft, held to its standard, at a volume a single team could not reach alone. ## What that looks like in practice For the researcher, the change is what you stop doing. You stop spending billable hours pulling verbatims to prove the work was rigorous, because the rigor is in the method, not in your manual effort after the fact. You spend that time on the part only a human should own: deciding what the evidence means. For the agency, the change is the [economics of discovery](/articles/ai-customer-research-agency-economics). When a disciplined interview no longer costs an hour of senior time, the trade that forced you to cut corners stops forcing it. You can scope deeper discovery, take on larger phases, and hand clients evidence built on behavior rather than on a room full of polite yeses. Good interviews change what a company builds. We want to make more of them possible, and we refuse to make the bad ones faster. ## Frequently asked questions ### What's the difference between real feedback and a compliment in customer interviews? A compliment is a verdict on you. Real feedback is a record of what the person actually did. "I love this" tells you the room felt warm. "I paid for a workaround last month" tells you a problem cost someone money. Treat enthusiasm as noise until a past behavior backs it. Ask what they did, not what they think of your idea, and the compliment stops mattering. ### Why are most customer interviews useless? Because they collect agreement instead of evidence. The interviewer plants the answer inside the question, asks people to predict their own future behavior, and accepts a compliment as proof. People are kind, so they say yes. The transcript looks like research and decides a roadmap, but nothing in it describes what anyone has actually done. That is theater, not data. ### What are the three types of bad data in the Mom Test? Rob Fitzpatrick names three: compliments, hypotheticals, and wishlists. A compliment is praise with no behavior behind it. A hypothetical is what someone imagines they would do in a future they have not lived. A wishlist is a feature request that names a solution instead of the problem. All three feel like signal in the room and tell you nothing about what the person will actually do. * * * Lùc is in private beta. If you run discovery and you are tired of flattering data, [join the waitlist](/).