Applied AI Engineer
A 6-month job-ready program with an optional 3-month Mastery extension (Months 7–9). One flagship application is built across the program: an AI-powered support/research assistant that grows from a single prompt call to a multi-agent, evaluated, guardrailed, production-deployed system.
Prerequisites
Comfortable with Python fundamentals (functions, classes, async/await), or willing to pick it up in Week 1.
REST APIs, JSON, environment variables, basic Docker.
No linear algebra required — this is applied engineering, not model training. Fine-tuning math is covered when it comes up in Month 7.
An LLM API key (OpenAI, Anthropic, or a free-tier alternative), a vector DB account (Pinecone/Qdrant free tier).
Designed for students who have completed (or placed out of) Months 1–4 of the Software Engineering track — working knowledge of REST APIs, basic backend, and comfort reading someone else's code.
Milestone ladder
Foundations, Prompting & RAG
Months 1 through 3. The goal is to take you from zero to RAG engineer.
| Month | Core curriculum | Projects | Milestone |
|---|---|---|---|
| Month 1LLM Foundations | How transformers/LLMs generate text (tokens, context windows, temperature/top-p); calling LLM APIs (OpenAI, Anthropic, open-source via Ollama); streaming responses; function/tool calling basics; structured output (JSON mode, Pydantic schemas); cost and latency tradeoffs across models. | CLI chatbot, structured-data extractor from unstructured text, a streaming Q&A API. | Can call an LLM API correctly, reliably, and cheaply. |
| Month 2Prompting, Embeddings & Vector Search | Prompt patterns (few-shot, chain-of-thought, ReAct), system-prompt design, prompt-injection basics; embeddings and cosine similarity; vector databases (Pinecone, Qdrant, pgvector); chunking strategies; hybrid search (keyword + vector). | Semantic search over a document set, a "chat with your PDF" app, a prompt-injection red-team exercise on their own app. | Understands why naive prompting breaks at scale and how retrieval fixes it. |
| Month 3RAG Systems In-Depth | Full RAG pipeline: ingestion, chunking, embedding, indexing, retrieval, reranking (cross-encoders), context assembly, citation/grounding; query transformation (HyDE, query rewriting, multi-query); multi-document and multi-format sources; RAG failure modes (lost-in-the-middle, retrieval gaps, stale indexes). | A production-grade RAG assistant over a real document corpus, with citations and a re-indexing pipeline. | Can build a RAG system that retrieves the right context and shows where it came from. |
Design note: RAG gets its own full month (Month 3) rather than being folded into general "AI basics" — it's the single most commonly shipped AI feature in production, and the one with the most subtle failure modes.
Agents, Multi-Agent Systems & Eval
Months 4 through 6. The goal is to take you from RAG engineer to agentic AI developer.
| Month | Core curriculum | Projects | Milestone |
|---|---|---|---|
| Month 4AI Agents & Tool Use | Agent loop (plan, act, observe); function/tool calling in depth (schema design, parallel tool calls, error handling); ReAct and Plan-and-Execute patterns; memory (short-term/conversation, long-term/vector-backed); LangGraph, or a hand-rolled agent loop first. | A research agent that searches, reads, and summarizes with citations; a task-automation agent with 3+ real tools. | Can build an agent that reliably picks and uses the right tool. |
| Month 5Multi-Agent Systems & Orchestration | Multi-agent patterns: supervisor/worker, hierarchical, peer-to-peer; agent communication protocols; orchestration frameworks (LangGraph, CrewAI, or AutoGen); shared vs. isolated state; handling agent disagreement and infinite loops; cost/latency of multi-agent systems. | A multi-agent customer-support system (router + specialists + escalation); a research team of agents (planner, searcher, writer, critic). | Can design and debug a multi-agent system, and explain when not to use one. |
| Month 6Evaluation, Guardrails & Deployment | Eval frameworks (Ragas, DeepEval, custom harnesses); eval metrics for RAG (faithfulness, answer relevance, context precision/recall) and agents (task completion, tool-call accuracy); guardrails (input validation, output filtering, PII redaction, jailbreak defenses); human-in-the-loop review; deploying LLM apps (FastAPI, streaming, rate limiting, cost monitoring, caching). | A full eval suite run against the flagship app with a scorecard; guardrails stress-tested with adversarial prompts; deployment to a cloud host with monitoring. | Can build it, evaluate it, guard it, and ship it, and prove it works with numbers, not vibes. |
Capstone, end of Month 6: the flagship assistant is now multi-agent, retrieval-grounded, evaluated against a benchmark set, guarded against common attacks, and deployed — the job-ready checkpoint.
Mastery Specialization (optional)
Months 7 through 9. The goal is to take you from agentic AI developer to AI systems specialist.
| Month | Core curriculum | Challenge |
|---|---|---|
| Month 7Fine-Tuning & Model Customization | When to fine-tune vs. RAG vs. prompt engineering; supervised fine-tuning (LoRA/QLoRA on open-weight models); dataset curation and synthetic data generation; RLHF/DPO at a conceptual level; evaluating a fine-tuned model against the base model. | Fine-tune a small open-weight model on a domain-specific task and benchmark it against prompting alone. |
| Month 8Advanced RAG & Agentic Architectures at Scale | Agentic RAG (agents that decide when/what to retrieve); GraphRAG and knowledge-graph-backed retrieval; long-context vs. RAG tradeoffs; multi-tenant RAG (isolating customer data); RAG over structured data (SQL agents, text-to-SQL). | Rebuild the flagship RAG pipeline as agentic and multi-tenant, with a text-to-SQL fallback for structured queries. |
| Month 9Production AI Systems & Career | Observability for LLM apps (tracing with LangSmith/Langfuse, token-cost dashboards); A/B testing prompts and models in production; incident response for AI systems (hallucination reports, cost spikes); portfolio and interview prep specific to AI engineering roles. | Present a full observability + cost dashboard for the flagship app, plus a mock AI-systems-design interview. |
Mirrors the Software Engineering track's 12-month Mastery extension, for students who want to specialize beyond shipping an AI feature into owning AI infrastructure and making build, fine-tune, or buy decisions.
Deep dive: RAG architecture
Reference material for instructors building slides and labs. Taught across Month 3 (core) and Month 8 (agentic, at scale).
Ingestion & chunking
- Loaders per format: PDF (text + layout-aware), HTML, Markdown, DOCX, CSV/tables, images (OCR).
- Chunking strategies: fixed-size with overlap, recursive character splitting, semantic chunking, sentence-window chunking.
- Metadata enrichment (source, section headers, timestamps) for filtering and citation.
Embedding & indexing
- Choosing an embedding model (dimensionality, domain fit, cost).
- Vector index types: flat, HNSW, IVF, with tradeoffs in recall vs. speed.
- Vector DB options: Pinecone, Qdrant, Weaviate, pgvector, and when to choose managed vs. self-hosted.
Retrieval
- Dense (vector), sparse (BM25/keyword), and hybrid retrieval.
- Query transformation: HyDE, multi-query expansion, query rewriting for conversational context.
- Metadata filtering (date ranges, source type, permissions).
Reranking & context assembly
- Cross-encoder reranking, and why bi-encoder retrieval alone isn’t enough.
- Context window budgeting: how many chunks, in what order (lost-in-the-middle mitigation).
- Citation and source attribution back to the original document.
Failure modes & fixes
- Retrieval gaps: the answer exists but wasn’t retrieved, diagnosed with retrieval-only eval.
- Hallucination despite retrieval: the model ignores context, diagnosed with faithfulness eval.
- Stale indexes and incremental re-indexing strategies.
- Multi-hop questions that need synthesis across chunks.
Deep dive: agents & multi-agent systems
Taught across Months 4 and 5 (core) and Month 8 (at scale).
Single-agent foundations
- The agent loop: plan, act, observe, repeat, and when to stop.
- Tool/function calling: schema design, parallel vs. sequential calls, handling tool errors and retries.
- Reasoning patterns: ReAct, Plan-and-Execute, reflection/self-critique loops.
- Memory: short-term (conversation buffer), long-term (vector-backed), working memory/scratchpad.
- Building an agent loop by hand before using a framework, so the abstractions aren’t magic.
Frameworks
- LangGraph, graph-based agent orchestration, the primary framework taught.
- CrewAI, role-based multi-agent, covered at survey level.
- AutoGen, conversational multi-agent, covered at survey level.
- Choosing a framework on debuggability and state management, not just feature lists.
Multi-agent patterns
- Supervisor/worker: one router agent delegates to specialist agents.
- Hierarchical: supervisors of supervisors for complex workflows.
- Peer-to-peer: agents that negotiate or hand off directly.
- Shared vs. isolated state, and why leaking full context to every agent is expensive and error-prone.
Failure modes & operations
- Infinite loops and circular hand-offs, addressed with timeout and max-turn guards.
- Agent disagreement, resolved by voting, an arbitration agent, or human escalation.
- Cost and latency: multi-agent systems multiply token spend fast.
- Debugging multi-agent traces, which is why observability matters so much for agents specifically.
When not to use multi-agent: a running theme through Month 5 — many problems that look like they need multiple agents are better served by one agent with more tools, or a plain pipeline with no agent at all.
Deep dive: evaluation & guardrails
Taught in Month 6 (core), with observability extending into Month 9.
Evaluation frameworks
- Ragas, DeepEval, promptfoo, and when to use a framework vs. a custom harness.
- LLM-as-judge patterns: using a stronger model to score outputs, and the pitfalls (judge bias, cost, drift).
- Golden datasets: a labeled eval set from real user queries, versioned like code.
RAG-specific metrics
- Faithfulness: does the answer stick to the retrieved context.
- Answer relevance: does the answer address the question.
- Context precision/recall: did retrieval surface the right, and only the right, chunks.
- Retrieval-only eval vs. end-to-end eval, to isolate where a failure actually happened.
Agent-specific metrics
- Task completion rate.
- Tool-call accuracy (right tool, right arguments).
- Trajectory evaluation: a sane path, not just a correct answer.
- Turn/step efficiency, since fewer wasted loops is itself a quality signal.
Guardrails
- Input guardrails: schema validation, prompt-injection detection, PII detection.
- Output guardrails: PII redaction, toxicity/safety filtering, format validation, fact-checking.
- Jailbreak defenses: system-prompt hardening, delimiter strategies, dedicated guardrail models.
- Human-in-the-loop: confidence thresholds routing low-confidence outputs to a reviewer.
Red-teaming: students red-team each other's flagship apps with adversarial prompts, injection attempts, and jailbreak tries, then patch what breaks and build a lightweight regression suite so a guardrail fix doesn't silently break next month.
Capstone projects & career outcomes
Capstone options at Month 6
- AI research/support assistant with citations (RAG-heavy).
- Multi-agent workflow automation tool (agent-heavy).
- Domain-specific assistant with a custom eval suite and guardrails (eval/safety-heavy).
Career prep
- Portfolio: a GitHub repo with architecture diagram, eval scorecard, and a short demo video per project.
- Resume framing specific to AI engineering roles (RAG engineer, AI/ML engineer, applied AI engineer, agent developer).
- Mock interviews: AI system design, take-home-style challenges, and behavioral rounds.
Ready for Applied AI Engineering?
Already comfortable with REST APIs and basic backend? This track picks up right where those skills leave off.