Discovery & requirements
The upstream work that determines whether you build the right thing — before a single sprint begins.
The four discovery questions
Every AI PM interview will probe these in some form. Answer them in this order — do not skip to question 3.
How often does it occur? Who experiences it? What is the quantified cost — time, revenue, error rate? Anecdotes are not requirements.
Does solving this problem move a metric the business cares about? What is the addressable impact if you execute well?
Is there a learnable pattern in historical data? Is there a feedback mechanism for improvement? Are the failure modes acceptable for this use case?
Define measurable outcomes before solution design begins. Name the metric, the target value, and the measurement method.
The kill criteria principle
The sharpest CTO interview question: "At what point during discovery would you have killed this project?" Candidates who cannot answer have not done real discovery.
AI readiness triage
Before writing a single requirement, answer these six binary questions. Your score determines whether you proceed to model scoping, investigate blockers first, or stop and re-frame the problem.
Is there a learnable pattern in historical data?
You need labeled examples or implicit feedback signals (clicks, corrections, ratings).
Can you obtain ground-truth labels at acceptable cost?
Crowdsourcing, expert annotation, or implicit signals — each has a cost and quality ceiling.
Is there a post-launch feedback loop to improve the model?
User corrections, engagement signals, or explicit ratings are needed for continuous improvement.
Are the failure modes survivable for your use case?
Define the cost of a false positive and false negative now. A wrong recommendation differs from a wrong medical diagnosis.
Can inference meet your latency budget?
Real-time (<200ms) vs. batch (minutes/hours) changes model choice. Confirm this before scoping.
Do you have legal/privacy access to the required data?
GDPR, CCPA, DPA sign-off, data residency. Legal kills AI projects in procurement — surface this in discovery, not launch week.
Data diagnostic — ask engineering in week 1
1. How much labeled historical data do we have, and what is the label consistency?
Volume and quality are not the same. 1M rows with inconsistent labels can underperform 50k rows with clean labels.
2. Is the data distribution representative of the problem we are solving?
A fraud model trained only on desktop transactions will fail silently on mobile — a distribution mismatch you must catch in discovery.
3. Who owns the data and what are the data access constraints?
This is a legal and privacy question as much as a technical one. Surface DPA, GDPR, CCPA, and data residency constraints before writing any requirements.
4. What is the data refresh cadence and does it match the prediction horizon?
Predicting 30-day churn requires features updated at least weekly. Batch-daily features are a deployment constraint — discover it now, not in sprint 4.
5. Is there a mechanism to capture post-launch corrections and signals?
If users cannot correct the model output, you have no feedback loop. The model will degrade without it.
I am a PM doing discovery for an AI feature that will predict which enterprise accounts are likely to churn in the next 60 days. Ask me the 6 AI readiness triage questions one at a time, then give me a Go/Investigate/No-Go verdict with specific gaps to resolve.
Build vs. buy vs. partner vs. fine-tune
This is the first question a CEO asks when an AI PM presents a discovery readout. Select your context to see the recommended path and a named real-world example.
Select a context above to see the recommended path.
Stakeholder influence map
AI discovery has a third stakeholder layer beyond business and users: the teams that own training data. Missing any of these in discovery week 1 creates a blocker downstream.
BRD structure for AI features
The BRD for an AI feature has two additional fields that a traditional BRD omits. Missing them signals you have not shipped an AI product.
AI requirements spec
This is the artifact that separates AI PMs from regular PMs. A VP will hand you a blank spec and watch what you fill in. Six mandatory fields.
CRM account fields updated within 24 hours; requires 18 months of historical contract data.
Returns a risk score 0–1 with a confidence level. Exposed to the UI only when confidence > 0.75.
False positive (flags healthy account as at-risk): <20%. False negative (misses a churning account): <10%. Note: these are not equal — define asymmetry explicitly.
P50 < 150ms, P99 < 400ms. Batch inference acceptable; scores refreshed nightly and cached.
Precision ≥ 0.82 and recall ≥ 0.78 on held-out eval set. Below this threshold, the model is not launched.
Account managers can mark a flagged account as "reviewed — not at risk." Correction is logged as a training signal.
Discovery exit checklist
Discovery is not a phase — it has a defined end condition. All six gates must be green before moving to solution design.
Problem validated with 5 or more user interviews — in the words of users, not in PM summary language.
Baseline metric measured — you know the current state number you are trying to move.
Data availability confirmed by data engineering — volume, quality, and access sign-off.
Legal/privacy sign-off on data use — GDPR, CCPA, EEOC, or relevant regulation reviewed.
At least one non-AI solution considered and explicitly rejected — with rationale documented.
Kill criteria defined — named data threshold, error rate ceiling, or latency limit that would trigger a stop.
Discovery interview Q&A
Five scenario questions with structured model answers. Each includes the common interview trap for that question — avoid it.
Walk me through how you'd approach discovery for an AI-powered churn prediction feature at Salesforce.
I start with problem validation — not just confirming churn is a pain point, but quantifying it: which customer segment churns, at what LTV, and whether account managers currently have enough signal to intervene in time. That tells me if prediction is the bottleneck, or if the real problem is intervention tooling.
Then I run a data diagnostic with data science: do we have labeled churn events with sufficient history? Is the label definition consistent — what counts as churn? Are at-risk accounts represented proportionally in training data?
Parallel to that, I evaluate build vs. buy: Salesforce already has Einstein Analytics. Is this a configuration problem with existing tooling, or a genuinely new model requirement?
Discovery ends with a written go/no-go decision and defined kill criteria: the precision threshold that makes it worth an account manager's time, and the intervention window required for the prediction to be actionable.
Common trap: Jumping to model selection — "we should use a gradient boosted tree" — before validating that a prediction model is the right solution at all.
A CTO asks: at what point during discovery would you have killed this AI project? What is your falsification criterion?
My falsification criteria had three hard gates.
First: if data science could not confirm more than 50,000 labeled examples with a consistent label definition after two weeks of data audit — stop. Scope a labeling pipeline first, not a model.
Second: if the acceptable false positive rate exceeded 20% — based on account manager capacity and cost of a wasted intervention call — stop. The model cannot meet the business constraint.
Third: if the required prediction horizon was not achievable at above-random precision with available features — stop and investigate whether feature engineering or additional data acquisition could close the gap before committing sprint capacity.
Kill criteria are defined at the start of discovery, not retroactively. If I cannot name them, I have not done real discovery.
Common trap: Treating kill criteria as pessimistic. Frame them as risk management — they protect engineering from building the wrong thing.
Three weeks into discovery, the data science team tells you there is not enough labeled training data. What do you do?
Three paths, evaluated in order.
First: can we unblock the labeling problem? Active learning, weak supervision, or a targeted human-labeling sprint can generate labels efficiently. I ask data science: how many examples do you need, and how long would labeling take at what cost? If the answer is three weeks and $20k, that is worth evaluating.
Second: can we descope to a problem where labeled data already exists? If we wanted to predict cancellation but only have data on login-frequency drop, can we solve a narrower version — detect disengagement, not churn — that still delivers value?
Third: can we use a foundation model with prompt engineering? For some classification tasks, zero-shot with a well-designed prompt outperforms a custom model trained on sparse data.
The answer I will not give: push this to engineering and tell them to figure it out. The PM owns the decision to proceed or pivot.
Common trap: Treating the data constraint as an engineering problem. It is a product scope and strategy decision owned by the PM.
How do you present a discovery readout where you are recommending NOT to build the AI feature?
The structure: evidence, no-go criteria met, alternative recommendation, next step.
Lead with evidence, not opinion. "We interviewed 8 enterprise accounts, audited three years of support ticket data, and ran a data audit with data science."
Then state which pre-defined kill criterion was met. "We specified that if we could not achieve more than 80% precision on a 60-day horizon, we would not proceed. The data audit shows we reach 63%."
Then offer an alternative. Discovery is not a failure — it is a decision. "The better investment is improving the intervention tooling for account managers. A dashboard with usage trend data is achievable in one sprint and directly addresses the churn risk without requiring a model."
End with a specific next step. Discovery that ends with 'we do not know yet' is incomplete.
Common trap: Framing a no-go recommendation as a failure. The best answer reframes it: your discovery process is working — you caught the problem before sprint 1.
You are a PM at Workday. HR managers spend 4 hours per week manually triaging job applications. Leadership wants an AI solution. Walk me through your discovery.
Step one: validate the problem specifics. Four hours per week is the symptom — I need to know where those hours go. Is it reading unqualified applications? Re-reading the same qualified ones? Coordinating with hiring managers? The AI solution is completely different depending on the answer. I run structured interviews with 6–8 HR managers across role segments.
Step two: data audit. For an AI triage solution, I need to know: do we have labeled historical hiring decisions? Is there historical bias in the labels — has a protected attribute influenced past decisions? I bring legal into this before data science, because a biased training set in hiring triggers regulatory exposure under EEOC and the EU AI Act high-risk classification.
Step three: build vs. buy. Workday Skills Cloud already has a structured competency model. The question is whether I extend Skills Cloud inference or build a new ranking model.
Step four: exit gate. Discovery closes with a written document specifying the problem statement, kill criteria, build/buy recommendation, success metrics (time-to-shortlist; quality-of-hire 90-day retention as the lagging indicator), and the legal/compliance sign-off path.
Common trap: Starting with model design. Candidates who immediately discuss NLP for resume parsing have skipped the problem validation and the legal gate that would have caught the bias risk.
I'm preparing for an AI PM interview at Salesforce. Ask me a discovery question — something a VP Product or CTO would ask — then give me feedback on my answer using this framework: problem validation, data feasibility, build vs. buy decision, success definition, and whether I named kill criteria.