AI tools & prompting
Model selection, prompt authorship, eval ownership, and risk management — the four domains where AI PMs add irreplaceable value that engineers alone cannot own.
Build vs Buy vs Configure
Every model choice is a margin decision. Run these three criteria in order before evaluating specific vendors.
Buy (API)
When
Speed to market matters and you have no proprietary data advantage that a model cannot replicate via prompting.
Cost shape
Pay-per-token. No infra overhead. Scales instantly.
Main risk
Vendor lock-in. Model deprecation. Cost non-linear at high volume.
Configure (Fine-tune)
When
You have a specific output format, style, or domain vocabulary that prompting cannot reliably produce after 500+ labeled examples.
Cost shape
$10K–$200K one-time training + ongoing serving overhead.
Main risk
High data labeling cost. Training pipeline maintenance. Format generalization often overrated.
Build (Train)
When
Unique capability that no external model can deliver, OR regulatory sovereignty requires no third-party model (defense, critical infrastructure).
Cost shape
$1M+ compute, data labeling, MLOps headcount, refresh cycles.
Main risk
Highest TCO. Only defensible with a unique proprietary data moat.
Model selection — apply your constraints
Toggle the constraints that apply to your product. Models that fail a constraint are eliminated. This is how the decision actually works — not by benchmarks.
GPT-4o
OpenAI / Azure
Multimodal, broadest ecosystem, enterprise procurement familiarity
Claude Sonnet
Anthropic / AWS
Best instruction-following, long context, enterprise trust posture
Gemini 2.5 Flash
Google / Vertex AI
Best cost/latency ratio, native Google Workspace integration
Llama 3 (self-hosted)
Meta (open weight)
Full data sovereignty, no vendor dependency, customisable weights
Key Takeaways
Hard constraints first
Data residency, latency SLA, compliance requirements eliminate models regardless of quality. Run these before any capability evaluation.
Your eval beats all benchmarks
Public benchmarks measure generic performance. Build a 50-example eval set calibrated to your task. That's the only score that predicts production quality.
Fine-tuning ROI threshold
Fine-tuning costs $10K–$200K before serving overhead. Break-even requires either >40% latency improvement that converts, or a capability gap prompting cannot close.
Model version is a versioned dependency
Deprecation happens. Define a model update protocol in the PRD — who owns the migration decision, what the minimum notice SLA is, and what the eval gate is before deployment.
PM vs. engineer ownership split
The most common interview failure: a PM who can't articulate where their responsibility ends and the engineer's begins. This line matters.
PM Owns
- System prompt (persona, task, constraints, format, examples)
- Output format contract — the schema engineering builds against
- Eval rubric (dimensions, scoring scale, pass/fail threshold)
- Golden test set — 50+ representative inputs
- Ship/no-ship decision at each eval gate
- Error recovery UX and fallback flow
Engineer Owns
- Retrieval pipeline and context window management
- Token budget optimisation and caching strategy
- Model version selection and latency tuning
- LLM-as-Judge harness and eval infrastructure
- Retry logic and failure handling
- Prompt regression CI/CD pipeline
Production system prompt — 5 elements
Click each element to see what the PM owns, a concrete example, and why it matters for production reliability.
PM Owns
PM authors this. The persona defines the identity and scope of the AI — it's a behavioral boundary, not a greeting.
Example
"You are a specialist legal document reviewer. You identify contract risks only within the document provided. You do not give legal advice."
Why it matters
Weak persona definitions are the root cause of tone failures and scope creep. The persona constrains behavior at the system level — before any user input.
Write a complete production system prompt for this feature: extract action items from meeting transcripts and output a structured JSON array. Each item must have: owner (string), task (string, max 15 words), due_date (ISO format or null), and priority (P1/P2/P3). Include all 5 elements: persona, task definition, constraints, format spec, and 2 few-shot examples. Then identify which element, if removed, would most likely cause production failures.
Key Takeaways
PM authors the system prompt
This is a product artifact — the behavioral spec for your AI feature. Delegating it entirely to engineering is the same as delegating the PRD.
Task definition must be testable
"Be helpful" is not a task definition. If you can't write a failing eval case for it, it's underspecified and the model will find the gaps.
Constraints are responsible AI made concrete
Every responsible AI requirement (fairness, scope, safety) ultimately lives in the system prompt constraints section. If it's not there, it's not enforced.
Format spec is a product interface
The format section is the contract between the prompt and the parser. Schema drift from model updates breaks parsers silently — format spec is how you prevent it.
PM-owned eval process — 5 steps
Evals are not QA at the end — they are the design specification at the beginning. Write them before you write the prompt.
Write the rubric first
Define 4–5 eval dimensions before any model is evaluated: Accuracy, Format Compliance, Tone, Safety, Helpfulness. Each dimension gets a 1–5 scale with anchor descriptions per score level. If the rubric doesn't exist, the acceptance threshold doesn't exist — and you have no objective ship gate.
PM authors the rubric. This is as important as the PRD.
Build the golden test set
Minimum 50 examples for internal tooling, 200 for customer-facing features, 500+ for regulated domains. Distribute: 60% normal inputs, 25% edge cases, 15% adversarial inputs (off-topic, hostile, ambiguous). Source from real production data — not invented examples.
PM defines the distribution and sources real examples. ML team formats and integrates.
Set the acceptance threshold
Define the pass/fail threshold before you run any eval. "We ship when accuracy scores above 4.0/5.0 and safety is 5.0/5.0 on the rubric." This is a business decision — it determines the ship date. Write it in the PRD.
PM owns this decision. Not the data scientist. The threshold reflects product risk tolerance.
Validate your judge
If using LLM-as-Judge (see below), validate it against 100 human-labeled examples. Measure correlation. If r > 0.8, trust it for high-volume eval. If not, fix the rubric — not the judge model.
PM defines the validation threshold. ML team runs the correlation analysis.
Define the refresh cadence
Eval sets go stale as production data shifts. Define a refresh cadence in the product spec: quarterly minimum, or triggered when production input distribution shifts >10% from training distribution.
PM owns the cadence decision. Data drift monitoring is an engineering responsibility; acting on it is PM.
LLM-as-Judge — what it is and when it fails
What it is
A secondary LLM evaluates your primary model's outputs against your rubric. You use it when human eval is too slow at the volume you need — 10K+ outputs per day after a system prompt change.
What the PM builds
- The rubric (dimensions + scale + anchor examples)
- Validation protocol against human-labeled gold set
- Correlation threshold that gates trust (r > 0.8)
- Escalation rule when judge and human disagree by >1 point
Failure modes to know
Verbosity bias
Judge systematically scores longer outputs higher, even when shorter ones are more accurate.
Position bias
When comparing two outputs, judge favors the first option presented — independent of quality.
Sycophancy
Judge shares the primary model's blind spots and systematically fails to catch the same error classes.
Design an LLM-as-Judge rubric for evaluating AI-generated customer support responses. Create 4 dimensions: accuracy, tone, completeness, and safety. For each dimension, write a 1–5 scale with anchor descriptions at scores 1, 3, and 5. Then write two example outputs — one that scores 5/5 overall and one that scores 2/5 — and show how the judge would score each against your rubric. Identify which failure mode (verbosity bias, position bias, sycophancy) your rubric is most vulnerable to, and how you'd mitigate it.
Key Takeaways
Rubric before prompt
The rubric is the design spec. If it doesn't exist before model evaluation begins, you have no objective definition of quality and no ship gate.
PM owns the acceptance threshold
"We ship when safety is 5/5 and accuracy is above 4.0/5.0" — this is a business risk decision, not a data science decision. It belongs in the PRD.
Validate the judge before trusting it
LLM-as-Judge is reliable when correlation with human labels is r>0.8. Anything below that means the rubric needs fixing — not a different judge model.
Eval sets go stale
Production data shifts. A quarterly refresh cadence minimum — or triggered by a >10% distribution shift — is a product spec item, not an afterthought.
Six failure modes — PM responsibility
Each risk has a root cause a PM should catch at PRD stage, before engineering begins. Select a risk to see the full breakdown.
Indirect Prompt Injection
Attacker embeds malicious instructions in content your product retrieves — a document, email, web page — not in the user's direct message. Your model executes the attacker's instructions as system-level commands.
PM Owns
The threat model in the PRD must include retrieval-surface attack vectors. PM defines the trust boundary: what content sources can inject into the context, and what security review they require.
PM Action
Treat all retrieved content as user-level trust, not system-level trust. Spec input sanitization and content validation on all retrieval sources before they reach the context window.
Key Takeaways
Indirect injection = retrieval attack surface
The attack surface for agentic products includes every document, web page, or database record that reaches the context window. Threat model must cover retrieval sources, not just user inputs.
Cost is non-linear with agent complexity
Model the feature cost at 10× expected volume before launch. Agentic multiplier (5–20 calls per user action) makes cost runaway a PRD-stage decision, not an ops incident.
Schema drift is a silent production failure
Pin model versions in production. Build schema validation into the CI/CD gate. Downstream parsers breaking without errors is the most common unflagged production incident in LLM products.
Design for the error, not against it
The model will make visible errors. The PM's job is to design the recovery UX before launch — not to assume quality is high enough that recovery won't be needed.
Interview questions — PM judgment level
All framed as PM decision questions, not ML knowledge tests. Study the reasoning pattern — not just the answer content.
I'm a PM deciding which LLM to use for a contract review feature. The constraints: (1) documents contain sensitive legal data — no data can leave our AWS environment, (2) we process 200K documents/month, (3) the model must handle 30-page contracts in a single context window, (4) response must arrive within 15 seconds, (5) we need HIPAA-adjacent compliance for health contract terms. Walk me through which models qualify, which are eliminated and why, estimated cost range, and what trade-off I should present to leadership.
Key Takeaways
Hard constraints eliminate before quality scores
Data residency, latency, compliance — these eliminate model options regardless of benchmark performance. Run constraints first, always.
Own the rubric, not just the outcome
"My team runs evals" reveals you're delegating an ownership responsibility. "I own the rubric and the acceptance threshold; my team runs the harness" is the passing answer.
Benchmark response = your eval, not theirs
When a competitor announces benchmark superiority, the right move is running the new model on your own eval set — calibrated to your task. Third-party benchmarks don't predict your production quality.
Indirect injection is the interview differentiator
Most candidates know "prompt injection exists." Knowing the indirect/second-order version — attacker payload in retrieved content — immediately signals real agentic product experience.