AI fundamentals
The technical foundations every AI PM must own — from model architecture to the ML lifecycle.
1M+ token context windows. Multimodal models handle text, images, code, and audio natively. Autonomous agents complete multi-step tasks without human checkpoints.
The broadest field — building systems that simulate intelligent behaviour: reasoning, perception, planning, decision-making.
Learns patterns from data rather than following hand-coded rules. Three paradigms: supervised, unsupervised, reinforcement.
Paste these prompts into any LLM and compare the answers across models.
Explain the difference between a rule-based spam filter and an ML-based spam filter. Give me a concrete example of why one beats the other in practice.
I'm a PM. Give me 3 scenarios where I should use a generative AI model and 3 where I should use a classical ML classifier. Use real product examples.
Help me build a decision tree for when to use AI vs rules vs no automation at all, from a PM perspective. Make it practical enough to use in a team meeting.
Nested subsets, not alternatives. When someone says "AI", ask which layer — the answer changes the entire build.
Most production AI (recommendation, fraud, search ranking) is still predictive ML. GenAI is a subset, not a replacement.
The 2017 attention mechanism is the foundation of every major LLM today. Worth understanding at a conceptual level as a PM.
GenAI adds hallucination management, prompt architecture, and non-determinism to the PM skill set. Own all three from Day 1.
LLMs don't read words — they read tokens (~3–4 characters each). Pricing, context limits, and speed are all measured in tokens, not words. "Tokenisation" itself is 4 tokens.
Max tokens the model sees at once. Gemini 2.5 Flash: 1M tokens (~750K words). Claude Sonnet 4.6: 200K tokens. Larger = better for long docs.
0 = deterministic (same output every run). 1+ = creative and unpredictable. Set 0 for facts, structured outputs, and compliance. Higher for brainstorming.
Numerical vectors representing semantic meaning. "King − Man + Woman ≈ Queen." Powers similarity search, RAG retrieval, and recommendation systems.
| Model | Company | Context | Best for | Type |
|---|---|---|---|---|
| Claude Sonnet 4.6 | Anthropic | 200K | Reasoning, code, safety | Frontier |
| GPT-4.1 | OpenAI | 1M | Tool use, vision, breadth | Frontier |
| Gemini 2.5 Flash | 1M | Speed, cost, multimodal | Fast | |
| Llama 3.3 70B | Meta | 128K | Open weights, on-prem deploy | Open |
| Mistral Large 2 | Mistral AI | 128K | EU compliance, enterprise | Enterprise |
Tokenize this sentence and count the tokens: 'Retrieval-augmented generation reduces hallucination by grounding LLM responses in verified documents.' Then explain your counting method.
Write a one-sentence product description for noise-cancelling headphones at temperature 0, then give me 4 more versions at progressively higher temperature. Show the range.
I'm a PM and need to process a 500-page legal document in a single context window. Which 3 current LLMs would you recommend and why? Include context window sizes and cost tradeoffs.
LLMs process tokens (~3–4 chars). Context windows and API costs are token-denominated. Know this before any vendor conversation.
Gemini 2.5 Flash and GPT-4.1 offer 1M token windows. Long-doc analysis no longer requires chunking strategies.
Set temperature=0 for deterministic outputs (legal, compliance, structured data). Document this in the PRD.
For most PM use cases, prompting + RAG outperforms fine-tuning at a fraction of the cost and operational burden.
Risk: Hallucination on specific facts; no private knowledge
Risk: Retrieval quality determines output quality — garbage in, garbage out
Risk: Expensive, data drift, slow to update when requirements change
The query is converted to a vector. The database returns the most semantically similar documents.
Query + retrieved docs are assembled into a prompt. The LLM reasons over real evidence — not just its training memory.
RAG turns the LLM into a reasoning engine, not a knowledge store. Facts come from the retrieval layer — so they're always current and auditable.
Just the instruction, no examples. Works for simple, well-defined tasks with clear vocabulary.
Instruction + 2–5 labeled examples. Dramatically improves accuracy on complex or ambiguous tasks.
Ask the model to reason step-by-step before answering. Best for multi-step or mathematical reasoning.
Defines the model's role, persona, and constraints. Set once, persists across the conversation. PMs own this.
I'm building an internal HR chatbot that answers questions about our leave policy, which changes twice a year. Should I use RAG, fine-tuning, or prompting? Walk me through the tradeoffs.
Name the 5 most cited AI papers from 2024 with their citation counts. Then rate your confidence in each answer from 1–10 and explain why.
Rewrite this vague prompt to be more effective using few-shot techniques: 'Summarize the customer feedback and identify themes.'
For knowledge-grounded products, RAG beats fine-tuning on cost, freshness, and auditability. Start here unless you have a strong reason not to.
The system prompt defines your product's behavior. PMs who can write precise system prompts ship better AI features faster.
Don't block on model improvement. Design around it: citations required, human review for high-stakes outputs, RAG to ground facts.
Adding 3–5 labeled examples to your prompt is the highest-ROI prompt engineering technique before any other optimization.
Own the decision: is ML actually the right tool? Define the prediction task, the success metric, and the acceptable error rate before any engineer writes code.
Starting with a solution ("let's use a neural net") instead of the problem.
Instead of "build an ML model", write: "Predict which support tickets will escalate within 2 hours, targeting 80% precision at 60% recall."
I want to use ML to reduce customer churn. Help me write a precise ML problem statement: what am I predicting, what are the inputs, what's the output, and what does success look like?
I'm building a medical diagnosis classifier. Should I optimise for precision or recall? Walk me through the tradeoff and give me the specific business consequences of each error type.
PMs consistently over-invest in model choice and under-invest in data quality. The training data ceiling is real and non-negotiable.
Once a model is trained, teams anchor on whatever it performs well on. Set the success criteria first, in the PRD, before any code is written.
"We'll add monitoring later" is how model quality silently degrades. Make drift alerts a launch criterion, not a backlog item.
The feedback loop is what compounds. If you don't capture labeled failures from production, you cannot improve. Build it into the initial design.