AI opportunity spotting
The first PM skill in any AI initiative: telling a real opportunity from hype. Interviewers test whether you evaluate before you build.
Four conditions for AI readiness
"Add AI to it" is not a strategy. Every AI opportunity must pass these conditions before it becomes a product requirement. All four must be met — not three.
The task has a repeatable structure that historical data can encode. Ask: "If I showed 10,000 examples to a smart person, could they learn to do this consistently?"
Stripe fraud detection: historical transaction patterns are learnable — velocity, merchant category, geolocation, device fingerprint. 15 years of transaction data gives 99M+ labeled examples.
"Make our search smarter" — no learnable pattern because success is undefined. What does "smarter" mean in data terms? Until that is answered, no model can be trained.
For supervised learning you need correct-answer examples. Labeling cost is the most underestimated variable in AI project planning — most estimates are off by 3–5x.
GitHub Copilot training data: millions of public code repositories with implicit labels (code that runs vs. code with errors). No manual labeling required — the feedback loop is built in.
A legal contract summary tool: 500 historical contracts exist, but they were summarized by 8 different lawyers with different styles. Labels are noisy and inconsistent — model trained on this data learns inconsistency.
Measure the human baseline first. AI must beat it meaningfully — not by 2% in perfect test conditions. If you do not know the baseline, you cannot evaluate whether AI is ready.
Google's speech recognition: human transcription accuracy is 95% at the word level. Google Speech-to-Text reached 95.1% in 2017 — that is the moment it became a deployable product, not before.
A PM pitching an AI resume screener without knowing how accurately human recruiters currently make decisions. If the baseline is 72% quality-of-hire, AI needs to beat 72% — not just exceed random.
Without a mechanism for real-world feedback to flow back into evaluation and retraining, the model silently degrades. Design for the flywheel from day 1.
Gmail Smart Reply: user sends or dismisses suggestions — both signals feed the training pipeline. The model improves every day without additional labeling cost.
A batch-report AI that generates weekly business summaries. Users read the reports but never indicate which parts were useful vs. wrong. No feedback signal — model cannot improve.
The 10x test
Will AI make this 10x better — in speed, scale, or accuracy — or 10% better with 10x the risk? A good AI opportunity has asymmetric upside: the AI does the task at a scale or speed humans structurally cannot match.
- Personalisation at scale (50M users)
- Real-time fraud detection (10ms decision)
- Document extraction (10,000/day)
- Anomaly detection in telemetry streams
- Meeting summaries (2 min vs 5 min)
- Drafting emails (saves 3 min/email)
- Improving search by 2% on CTR
- Dashboard reports nobody reads
The 6-question opportunity scorecard
Run this before any AI feature enters the roadmap. Score 0 (no), 1 (partly), 2 (yes) for each question. Max score: 12.
0 = vague goal · 1 = partially defined · 2 = exact input→output transformation written down
0 = every case is unique · 1 = some pattern but noisy · 2 = clear repeatable pattern with known examples
0 = no data, labeling impossible · 1 = data exists but expensive · 2 = labeled data available or cheaply obtainable
0 = baseline unknown · 1 = estimated, AI gap unclear · 2 = baseline measured, AI demonstrably beats it
0 = no feedback mechanism · 1 = possible but not designed · 2 = feedback loop designed into product from day 1
0 = errors are catastrophic · 1 = costly but recoverable with HITL · 2 = errors are low-cost or caught downstream
Proceed to scoping
Address gaps before committing
AI is not the right tool yet
AI Opportunity Scorecard
Score 0–2 per question. Max 12. Run this before any AI feature enters the roadmap.
Is there a specific, measurable task — not a vague goal?
Is there a learnable pattern in historical data?
Is labeled data available or feasible to create?
Is the human baseline beatable and known?
Is there a production feedback loop?
Is the error cost acceptable at the intended automation level?
Red flags that kill AI projects
Each of these patterns has destroyed well-funded AI initiatives. Interviewers test whether you can recognise them in discovery — before they become sprint blockers.
Data is not a technical dependency — it is the product. Confirming data readiness before sprint 1 is a non-negotiable discovery gate.
IBM Watson Health: partnerships announced, products launched — then quietly wound down when clinical data quality and access did not match what was needed to train reliable models. The data issue was known; it was deferred.
If you cannot say "we will ship when X reaches Y% on Z dataset," you have no stopping condition. Without a defined eval set, you will ship when you run out of time, not when the model is ready.
A large US bank launched an AI-powered customer service chatbot with "improve CSAT" as the goal. 14 months later, CSAT had not moved because the team had never agreed on what a "successful AI interaction" looked like.
In AI, shifting requirements invalidate your dataset. If the task definition changes, the training labels are wrong. Stabilise the problem definition before labeling begins.
A healthcare startup changed their classification from "appointment urgency" to "care pathway recommendation" mid-training. The 8,000 labels collected under the first definition were discarded. Wasted 6 weeks of clinical team time.
AI amplifies what you give it. An inconsistent, poorly-understood process becomes an inconsistent automated process at scale — just faster and more expensive.
Amazon's internal AI hiring tool trained on 10 years of hiring data. The hiring process had been biased toward male candidates — the model encoded and amplified that bias. Retired in 2018.
For AI in finance, healthcare, or hiring, regulatory ambiguity is a product risk. Bring legal and compliance into discovery week 2 — not post-launch.
Several EU companies launched LLM features only to pause them when the EU AI Act final text clarified high-risk classification requirements. The pause cost 3–6 months of re-scoping.
Vague capability expectations produce vague evaluation criteria. Force precision: exact input → exact output → measured how → against what threshold?
A PM who says "the model needs to understand the context" cannot write acceptance criteria, cannot define a golden test set, and cannot make a launch decision. This is the single most common AI PM interview failure mode.
Framing AI opportunities for stakeholders
Identifying the opportunity is only half the job. The PM must also frame it for different audiences — CEO, CTO, and the end-user team all need a different narrative.
They want: Business impact and competitive position
"This AI feature will reduce our cost-to-serve by 28% on the X workflow while matching what Competitor Y launched in Q2. It is achievable in 16 weeks with the data we already have."
Avoid: Technical capability description. CEOs do not care about model architecture — they care about margin, growth, and defensibility.
They want: Technical feasibility, data readiness, operational risk
"We have 3 years of labeled historical data with 94% label consistency. The task is a structured classification with 12 classes. We expect to beat the human baseline of 82% at a threshold where false positive cost is under $40 per event."
Avoid: Business outcome framing without the technical specifics. CTOs will probe for the data readiness and failure mode analysis — have both ready.
They want: What changes for them and why it will not make their job worse
"The AI will take over the classification step — the one that takes you 4 minutes per ticket. You review the cases it flags as uncertain and correct any mistakes. Your expertise still shapes the final decision; the AI handles the volume."
Avoid: Efficiency or cost-saving framing. The message "we are making you more efficient" is often heard as "we are replacing you." Frame it as: AI handles the repetitive part so you can focus on the complex part.
Worked examples — strong vs. weak AI opportunity framing
- — Add AI to our search
- — Improve CX with AI
- — Build a copilot for our users
- — Make onboarding smarter
No specific task, no success metric, no data validation, no failure mode analysis.
- Reduce invoice routing time from 2.1 days to 4 hours by automating approval for invoices under $5k from verified vendors
- Increase first-contact resolution on billing queries from 68% to 82% using AI-assisted response drafts in the Salesforce case view
- Reduce recruiter time-to-shortlist by 60% by automating minimum qualification screening for the 4 highest-volume job categories
Specific task, measurable baseline, defined scope, achievable with available data.
AI opportunity interview Q&A
Five high-signal questions — including one behavioral (STAR) question — that test whether you evaluate before you build.
How do you decide whether a problem is worth solving with AI versus a rule-based or simpler solution?
I apply a two-stage filter.
Stage one: is this an AI problem at all? I ask three questions: Is there a learnable pattern in historical data? Is labeled data available or feasible to collect? Is the human baseline known and demonstrably beatable by a model? If any answer is no, AI is not yet the right solution — I either solve it differently or invest in the data infrastructure before scoping a model.
Stage two: is AI the right lever? I apply the 10x test. Does AI do this 10x better in speed, scale, or accuracy — or just 10% better with 10x the risk? A rule-based system that achieves 88% accuracy with no training data, no maintenance, and full explainability is often better than an ML model that achieves 91% accuracy with all the operational overhead of a production AI system.
The answer I give stakeholders: "AI is the right tool when the pattern is too complex for rules, the volume is too high for humans, and the data exists to train on. Here is whether this problem meets that bar."
Common trap: Defaulting to AI because it feels more sophisticated. Rule-based systems are underrated. A PM who recommends the simplest solution that meets the requirement signals more judgment than one who defaults to ML.
A stakeholder asks you to "add AI to our product" with no specific use case. What do you do?
I treat this as a discovery prompt, not a feature request.
First, I reframe the conversation: "What specific outcome do you want to improve — faster time to close, fewer support escalations, higher forecast accuracy? Let us start there and then evaluate whether AI is the right lever."
If the stakeholder cannot name a specific outcome, I run a structured opportunity discovery session: map the top three workflows in the product, identify where volume is high and human judgment is repetitive, and apply the four AI readiness conditions to each. This takes two weeks and produces a prioritized list of actual AI opportunities with a feasibility assessment attached to each.
I always come back with something actionable — not "AI is not ready" but "here are three specific places where AI can deliver measurable value, in priority order, and here is what it would take to pursue each one." The PM's job is to convert a vague mandate into a specific decision.
Common trap: Starting a sprint to "explore AI use cases" without a defined discovery output. Exploration without a defined deliverable wastes sprint capacity and damages PM credibility.
Tell me about a time you pushed back on an AI feature request that was driven by hype rather than user need.
Situation: a VP asked us to build an AI chatbot for our customer support flow — "our competitors have one, we need one." No user research, no defined use case beyond "answer questions."
Action: before writing a single requirement, I ran the opportunity scorecard. Specific task? Undefined — "answer questions" covers 300 distinct intent types. Labeled data? Our support tickets existed but were not labeled by intent or resolution quality. Human baseline? We had a 78% first-contact resolution rate but did not know which query types drove the failures.
I proposed three weeks of data discovery: label 1,000 tickets by intent and resolution outcome, identify the top 5 intent types that account for 60% of volume, and build the chatbot only for those intents where we have sufficient data and a defined success metric.
Result: the chatbot launched scoped to 4 intent types, hit 81% resolution rate on those types (vs. 68% agent baseline for the same intents), and expanded from there. The original "answer everything" scope would have shipped something that failed on 80% of questions.
Common trap: Refusing to engage with the request at all. Pushback without an alternative is just obstruction. Come back with a scoped version that is achievable and defensible.
How do you evaluate an AI opportunity in a domain you have never worked in before?
My evaluation framework does not change — the questions are domain-agnostic. What changes is who I partner with to answer them.
I start by identifying two people: a domain expert who knows the process deeply, and a data scientist who can assess data feasibility. My job in the first two sessions is to understand enough about the domain to formulate the right questions, not to become the domain expert.
I then apply the four conditions: is there a learnable pattern (the domain expert tells me if the task is rule-based or judgment-based), is labeled data feasible (the data scientist assesses what exists), is the human baseline knowable (the domain expert can quantify current performance), and is there a feedback mechanism (I design this in discovery).
The thing I am specifically listening for: how much disagreement exists among experts on the right answer to a given case. If two domain experts would give different answers to the same input 40% of the time, the task is not well-defined enough to train a model on — and I surface that finding before any engineering investment.
Common trap: Pretending to have domain expertise you do not have, or deferring all judgment to the domain expert. Both fail. The PM synthesizes — which requires asking better questions than either expert would ask alone.
What is the 10x test for AI opportunities and when would you recommend a simpler non-AI solution?
The 10x test asks: does AI make this 10x better in a dimension that matters — speed (real-time at scale), scale (millions of decisions per day), or accuracy (above human ceiling) — or does it make it 10% better with 10x the risk and operational complexity?
A recommendation engine that personalizes content for 50 million users daily is a 10x case — humans cannot do it at that scale. A meeting summary tool that takes 2 minutes instead of 5 minutes is not a 10x case — the gain does not justify the hallucination risk and maintenance overhead for most enterprise contexts.
I recommend a simpler solution when: the volume is low enough for humans or rules to handle reliably, the edge cases are frequent and costly, the regulatory environment requires full explainability, or the training data does not exist. A well-designed rule-based system at 88% accuracy beats a poorly-trained ML model at 82% accuracy every time — and it is auditable, maintainable, and does not hallucinate.
The interview answer that impresses: "I shipped a rule-based solution first, used it to collect labeled data, and then replaced it with a model once the training data was sufficient. The rule-based system was not a failure — it was the data collection strategy."
Common trap: Treating AI as the default and non-AI as the fallback. A senior PM recommends the simplest solution that meets the requirement. AI is the tool for specific conditions, not the answer to every problem.
I'm preparing for an AI PM interview. Give me a realistic AI opportunity scenario — a business problem at a company like Salesforce, Workday, or Google — and ask me to evaluate whether it is a strong AI opportunity. Then score my answer on: specificity of the task definition, data feasibility awareness, human baseline acknowledgment, and whether I named a kill criterion.