AI PM Ready

AI tools & prompting

Model selection, prompt authorship, eval ownership, and risk management — the four domains where AI PMs add irreplaceable value that engineers alone cannot own.

Build vs Buy vs Configure

Every model choice is a margin decision. Run these three criteria in order before evaluating specific vendors.

Buy (API)

When

Speed to market matters and you have no proprietary data advantage that a model cannot replicate via prompting.

Cost shape

Pay-per-token. No infra overhead. Scales instantly.

Main risk

Vendor lock-in. Model deprecation. Cost non-linear at high volume.

Configure (Fine-tune)

When

You have a specific output format, style, or domain vocabulary that prompting cannot reliably produce after 500+ labeled examples.

Cost shape

$10K–$200K one-time training + ongoing serving overhead.

Main risk

High data labeling cost. Training pipeline maintenance. Format generalization often overrated.

Build (Train)

When

Unique capability that no external model can deliver, OR regulatory sovereignty requires no third-party model (defense, critical infrastructure).

Cost shape

$1M+ compute, data labeling, MLOps headcount, refresh cycles.

Main risk

Highest TCO. Only defensible with a unique proprietary data moat.

Model selection — apply your constraints

Toggle the constraints that apply to your product. Models that fail a constraint are eliminated. This is how the decision actually works — not by benchmarks.

GPT-4o

OpenAI / Azure

Frontier
Cost$2.50 in / $10 out per 1M tokens
Latency~1.5s avg
DataAPI (Azure for EU residency)
CompliantSOC 2, HIPAA via Azure BAA

Multimodal, broadest ecosystem, enterprise procurement familiarity

Claude Sonnet

Anthropic / AWS

Frontier
Cost$3.00 in / $15 out per 1M tokens
Latency~1.2s avg
DataAPI (AWS Bedrock for residency)
CompliantSOC 2, HIPAA via AWS BAA

Best instruction-following, long context, enterprise trust posture

Gemini 2.5 Flash

Google / Vertex AI

Fast
Cost$0.15 in / $0.60 out per 1M tokens
Latency~0.5s avg
DataAPI (Vertex EU available)
CompliantSOC 2, HIPAA via Vertex BAA

Best cost/latency ratio, native Google Workspace integration

Llama 3 (self-hosted)

Meta (open weight)

Self-host
CostGPU infra cost (~$0.02–0.10/1M tokens)
LatencyInfra-dependent
DataYour VPC — fully air-gapped
CompliantMeets any requirement

Full data sovereignty, no vendor dependency, customisable weights

The agentic cost multiplier. An API feature that costs $0.002/query becomes $0.04/query when it makes 20 LLM calls per user action. Model your feature cost at 10× expected volume before scoping. If the number is surprising, the architecture changes — not the launch plan.

Key Takeaways

Hard constraints first

Data residency, latency SLA, compliance requirements eliminate models regardless of quality. Run these before any capability evaluation.

Your eval beats all benchmarks

Public benchmarks measure generic performance. Build a 50-example eval set calibrated to your task. That's the only score that predicts production quality.

Fine-tuning ROI threshold

Fine-tuning costs $10K–$200K before serving overhead. Break-even requires either >40% latency improvement that converts, or a capability gap prompting cannot close.

Model version is a versioned dependency

Deprecation happens. Define a model update protocol in the PRD — who owns the migration decision, what the minimum notice SLA is, and what the eval gate is before deployment.

Was this page useful?