Build to learn, then throw it away: a new habit for product managers

Build to learn, then throw it away: a new habit for product managers

An idea from Lenny's podcast stayed with me: you can build something, learn from it and throw it away without the exercise being a failure.

In his conversation with Anish Acharya, just before the discussion of consumer AI, Lenny describes a forthcoming guest writer's concept: “building is now the new reading where you build to learn.” Anish replies: “It's building as an activity rather than an outcome, right?” The exchange is around 50:53–51:10; those timings and quotations come from the episode's transcript mirror.3

Listen from approximately 50:45. Lenny credits a writer at Google without naming her in this passage, so I would not attribute the idea's invention to either host or guest.

That gives product managers a useful permission slip. A small app can be a way to investigate an idea, practise with a model or understand a workflow. It does not need a market, a launch plan and a permanent place on the roadmap to justify the exercise.

My take is that AI makes this habit easier to bring into everyday product work. Build a small version of the thing you are trying to understand. Put it under pressure. Keep what you learn, even when you decide the implementation should go no further.

An old learning idea with a different cost

Learning through making predates generative AI. In their 1991 account of constructionism, Seymour Papert and Idit Harel describe how constructing an inspectable artefact can support the construction of knowledge. They also resist reducing the idea to a simplistic slogan about making things.4

Reading still supplies concepts, domain knowledge and other people's hard-won mistakes. Building gives you somewhere to test your understanding. The practical opportunity is to move between the two more readily, rather than consume another explanation while avoiding the uncertainty of trying something yourself.

There is also a reason to build alternatives. In a 2010 study with 33 participants completing a banner-ad design task, Steven Dow and colleagues found that parallel prototyping produced better final designs than serial iteration, including stronger click-through performance and expert ratings.5 This was a small, non-AI design study, not evidence that generating more apps automatically improves product decisions. It supports a narrower practice: compare genuinely different approaches before committing to one.

AI can make those alternatives easier to produce. Whether the exercise improves judgement depends on the question, the feedback and the thinking you do afterwards.

A working app does not prove you learned

There is an important catch to building with AI: it can do the thinking you hoped the exercise would teach you.

In a 2026 research preprint, Judy Hanwen Shen and Alex Tamkin report a randomised study of 52 participants learning an unfamiliar Python library. The AI-assisted group scored lower on the subsequent skills quiz, without a statistically significant improvement in task time.6 The experiment involved a short coding session and an immediate assessment. It does not measure product-management judgement or establish that every form of AI-assisted learning is harmful.

It does give us a reason to separate successful output from understanding. If you cannot explain why the prototype behaves as it does, identify its assumptions or recognise when it fails, the tool may have produced more than you learned.

Before asking AI to fix something, make your own prediction about the cause. Afterwards, explain what changed and why. For a PM, the important explanation may concern a customer assumption or a workflow trade-off rather than the code. Keep that part of the work yours.

Three different things you might be learning

A working prototype can answer different questions. Mixing them up creates false confidence.

First, you can learn the tool. Can this model follow your brief? Where does it need examples? Does it recover sensibly when something goes wrong? Building a small, low-stakes app gives you experience a comparison chart cannot.

Second, you can learn about the product. Can a customer understand the proposed workflow? Does the feature solve the problem they brought with them? That requires contact with customers and their circumstances. Your own excitement about the demo cannot stand in for their behaviour.

Third, you can investigate feasibility. Can the real system handle permissions, messy data, delays and failures? A beautiful screen backed by invented data leaves most of those questions open.

Marty Cagan's distinction between feasibility, user and live-data prototypes remains useful. Each tests something different. He specifically warns against treating positive reactions to a high-fidelity prototype as proof that the product will sell.2

A PM can finish an afternoon with better tool intuition and no additional evidence of customer demand. That is a perfectly reasonable result, provided the team names it accurately.

Build an executable question

The useful starting point is a sentence about uncertainty.

“We should build an AI onboarding assistant” is already a solution. “New customers may be abandoning the import because they cannot tell what will happen to their existing records” gives you something to investigate.

Here is a hypothetical example. A team runs a CRM for small service businesses. Support conversations suggest customers hesitate before importing their contacts. The PM proposes two disposable alternatives:

• A short conversation that explains the import and answers questions.
• A preview that shows which records will be added, changed or left untouched, with a clear undo explanation.

Both use synthetic contacts. Neither connects to a customer's account. The team builds only the import decision, leaving billing, settings and the rest of onboarding alone.

Before showing either version, the team writes its prediction: customers need visibility and control more than another explanation. It also writes what would challenge that belief: people understand the preview but remain unable or unwilling to proceed because the underlying import rules are confusing.

Then target users attempt a realistic task. The team observes where they hesitate, whether they can explain what will change, what help they request and which concerns remain. Where practical, it varies the order of the two versions so the first demonstration does not teach everyone how to use the second.

Nobody announces a conversion uplift from this exercise. It provides qualitative evidence about comprehension and trust. A claim about actual completion rates would need an appropriately designed test with real behaviour, safeguards and enough observations for the decision at hand.

Possible outcomes include choosing one approach for further testing, combining useful elements, discovering that neither addresses the problem, or stopping work on the feature.

The prototype has done its job once it has helped the team make that decision.

There is broader evidence for treating stopping as a legitimate result. A 2024 study across four randomised trials involving 759 firms found that training entrepreneurs to formulate and test theories increased idea termination.7 It tested an entrepreneurship training approach, not AI prototyping. The relevant lesson for this workflow is to make room for a decision against further investment.

Agree to throw it away before anyone falls in love with it

A polished demo is persuasive. Once it has a name, a logo and an enthusiastic stakeholder, a learning exercise can quietly become a delivery commitment.

Make the boundary explicit before building:

• Question: Which uncertainty are we investigating?
• Prediction: What do we expect, and what evidence would change our minds?
• Scope: What is real, what is simulated, and which data or actions are prohibited?
• Budget: How much time and tool spend are we willing to use?
• Exit: When will we stop, and who decides whether to discard, investigate further or fund delivery?

Treat the budget as a ceiling rather than a target. A paper sketch, an existing analytics query or a customer conversation may answer the question sooner than a working app. AI availability does not make coding the correct research method.

When a concept deserves investment, make a fresh production decision with engineering and design. Review security, privacy, accessibility, reliability and the cost of maintaining it. Some implementation may be reusable after review. None of it earns production status because it survived a demo.

Keep a learning record small enough to use

Throwing away the app should not mean throwing away the evidence.

Keep the original question, the prediction, a screenshot or recording where appropriate, the observations and the decision. Separate what people did from what the team inferred. Note what remains unknown.

For the hypothetical import experiment, a useful record would explain whether the next step is to investigate customers' mental model of merging records, test a clearer preview or stop exploring the assistant. “We built two prototypes” tells the next PM almost nothing.

This changes the review conversation. Instead of asking how many demos the team made, ask which decisions improved, which assumptions failed and where expensive work was avoided. Track the time from naming an uncertainty to making an evidence-informed decision. Do not turn discarded prototypes into another output quota.

For your next product review, choose one unresolved question and agree the smallest safe way to investigate it. If a prototype is the right tool, agree its stop condition before opening the builder. Bring the evidence and the resulting decision back to the next review, even if there is no app left to show.

Sources

[2] https://www.svpg.com/flavors-of-prototypes — Marty Cagan — Flavors of Prototypes [3] https://podscripts.co/podcasts/lennys-podcast-product-career-growth/why-companies-are-becoming-a-series-of-loops-anish-acharya-a16z — Episode transcript [4] http://www.papert.org/articles/SituatingConstructionism.html — Papert and Harel — Situating Constructionism [5] https://hci.stanford.edu/publications/2010/parallel-prototyping/ParallelPrototyping2010-final.pdf — Dow et al — Parallel Prototyping [6] https://arxiv.org/pdf/2601.20245v2 — Shen and Tamkin — How AI Impacts Skill Formation [7] https://repositorio.ie.edu/bitstreams/f7a19247-bc6c-4639-89b8-9b94e593172e/download — Camuffo et al — Scientific Entrepreneurial Decision-making