CS2680 Modern AI Systems: Agents and System Optimizations
Fall 2026
Announcement
- Oct 28, 2026Claude Code usage feedback website is live.
- Oct 8, 2026Assignment Leaderboard is live.
- Sept 4, 2026We have increased capacity to 80 students.
- Aug 28, 2026Audit policy: everyone is free to audit as long as it is permitted by school policy and the classroom has space. However, we do not have computing resources for auditing students.
- Aug 17, 2026Welcome to CS2680 Modern AI Systems: Agents and System Optimizations! Class meets Monday and Wednesday in SEC LL2.229.
Overview
This course follows one agentic system from the agent loop down through the serving stack. You first use a modern coding agent (Claude Code), then design your own agent using provided API access, measure it, and optimize it. Then you serve an open-weight model yourself and take responsibility for the layers underneath: batching and scheduling, KV-cache and prefix reuse, routing, quantization, and speculative decoding.
Class time and location
Monday and Wednesday
11:15am – 12:30pm
SEC LL2.229
Prerequisites
CS61, and at least one graduate-level course in computer systems.
Comfort with Python and PyTorch is assumed. See the background we assume.
Assessment
Five assignments and a final project.
Class participation throughout the semester.
There is no exam.
Format. The course runs in two parts.
- Part I — Introduction to LLMs and agents. You will learn about how to use coding agents (Claude Code) and how to design an agent. You will also explore various agent architectures, use cases and optimizations.
- Part II — Systems for LLMs and agents. We will talk about LLM serving (batching, scheduling, routing, caching, quantization, and speculative decoding) and agentic systems.
1 “Mads” abbreviates Modern Agent Design and Systems, the arc of this course, and nods to MaDSys, Harvard’s Measurement and Design of Systems research group, which teaches and supports it.
What this course expects. This course will not teach you every piece of knowledge you need
for the assignments. It focuses instead on the skills to learn with, and points you in directions worth
going. Expect to spend a significant amount of time learning and experimenting on your own after class. If you
want an A, plan on it.
Grading.
- Assignment 1 Mads-Lens: watch the agent work10%
- Assignment 2 Mads-Loop: build the loop10%
- Assignment 3 Mads-Opt: make every token count20%
- Assignment 4 Mads-Serve: own the serving engine20%
- Assignment 5 Mads-Stack: optimize the full stack10%
- Project20%
- Class participation10%
- Bonus optional presentations up to 4% each; course feedback up to 6%; problems current AI cannot solve, 2% eachup to 10%
See policy page for details.
Course website. https://cs2680.com
Staff
Contact: cs2680-staff@g.harvard.edu Please use this address for all
course-related questions. We only respond to emails from Harvard students. Due to the large number of emails we receive, we do not respond to external inquiries.
Office hours:
- Instructor: Appointment only
- TFs: See Ed + by appointment
Schedule
Tentative and subject to change. Rows with Students as the speaker are student-led: either a paper discussion (see the paper discussion page) or an assignment sharing session. Paper presentations are optional and can earn up to four bonus percentage points each.
| Week | Date | Speaker | Topic | Readings | Notes |
|---|---|---|---|---|---|
| Part I — Introduction to LLMs and agents | |||||
| 1 | 2026-09-02 | Juncheng Yang | Course overview and logistics [slide] | Assignment 1 out (Mads-Lens: Watch the agent work); Sign up for AWS | |
| 2 | 2026-09-07 | No class — Labor Day | |||
| 2 | 2026-09-09 | Juncheng Yang | Agents use I: How Claude Code Works [slide] | How I use LLMs; Best practices for Claude Code; Agentic Engineering Patterns; more reading → | |
| 3 | 2026-09-14 | Juncheng Yang | Agents use II: 100x Engineer with Multi-agents [slide] | Claude Code agent teams; Software is changing (again); Research acceleration: The view inside OpenAI; more reading → | |
| 3 | 2026-09-16 | Juncheng Yang | Agents design I: context management and tool design [slide] | ReAct; A practical guide to building agents; LLM Powered Autonomous Agents; more reading → | Assignment 1 due Sun Sep 20, 11:59pm |
| 4 | 2026-09-21 | Students | Student sharing I: Learning from Assignment 1 | Assignment 2 out (Mads-Loop: Build the loop) | |
| 4 | 2026-09-23 | Juncheng Yang | Agents design II: multi-agent architecture, coordination, and communication [slide] | multi-agent research system; scaling agent systems; don’t build multi-agents; more reading → | |
| 5 | 2026-09-28 | Juncheng Yang | Agents design III: agent memory and reliability [slide] | where agents fail; MemGPT; more reading → | |
| 5 | 2026-09-30 | Juncheng Yang | Agent design IV: agent evaluation [slide] | Agent evaluation and benchmarks | Assignment 2 due Sun Oct 4, 11:59pm |
| 6 | 2026-10-05 | Harsh Nilesh Pathak (Snowflake) | Guest lecture: Developing and Evaluating Agents in Enterprise Setting | Assignment 3 out (Mads-Opt: Make every token count) | |
| 6 | 2026-10-07 | Juncheng Yang | Advanced agent design I: Cost and performance [slide] | Cost-effective agents | Pre-proposal due (optional and only submit if you want to discuss your ideas) |
| 7 | 2026-10-12 | No class — Indigenous Peoples’ Day | |||
| 7 | 2026-10-14 | Students | Advanced agent design II: agent safety, self-evolving harness | OWASP agentic top 10; more reading → | |
| Part II — Systems for LLMs and agents | |||||
| 8 | 2026-10-19 | Juncheng Yang | Efficient LLM serving basics I: intro to LLMs [slide] | Transformer; Llama 3; Mixtral; more reading → | Assignment 3 due Tue Oct 20, 11:59pm |
| 8 | 2026-10-21 | Juncheng Yang | Efficient LLM serving basics II: intro to GPUs [slide] | CUDA programming guide; more reading → | Assignment 4 out (Mads-Serve: Own the serving engine) |
| 9 | 2026-10-26 | Students | Student sharing II: Harness optimization [slide] | ||
| 9 | 2026-10-28 | Yunjia Zheng and Juncheng Yang | Assignment 3 feedback [slide] | Project proposal due | |
| 10 | 2026-11-02 | Juncheng Yang | Assignments feedback and agent pitfalls, GPU programming (cont.) [assignment, gpu] | ||
| 10 | 2026-11-04 | Juncheng Yang | Efficient LLM serving basics III: Batching, paging, and parallelism [slide] | Orca; vLLM; more reading → | |
| 11 | 2026-11-09 | Xianzheng Dou and Haythem Memmi (Meta) | Guest lecture: Why Infrastructure sets the pace of AI | Assignment 4 due Tue Nov 10, 11:59pm | |
| 11 | 2026-11-11 | Juncheng Yang | Efficient LLM serving: speculative decoding | Assignment 5 out (Mads-Stack: Optimize the full stack) | |
| 12 | 2026-11-16 | Students | Efficient LLM serving: KV-cache optimization and model architectures | ||
| 12 | 2026-11-18 | Juncheng Yang | Efficient LLM serving: pruning and quantization | ||
| 13 | 2026-11-23 | Juncheng Yang | Efficient agentic systems: scheduling and prefix caching | Assignment 5 due Tue Nov 24, 11:59pm | |
| 13 | 2026-11-25 | No class — Thanksgiving recess | |||
| 14 | 2026-11-30 | Juncheng Yang | Efficient agentic systems: routing and load balancing | ||
| 14 | 2026-12-02 | Students | Student sharing III: Showcase your full stack optimizations (Assignment 5) | ||
| 14 | 2026-12-07 | Students | Final project poster and demo | Present your final project as a poster and a live demo. Location: SEC 1.402 from 9:00 AM to 1:00 PM |
Final project due Tue Dec 8, 11:59pm |
Reference pages
- Agent evaluation and benchmarks. Evaluation design, metrics, reliability, cost, and benchmark selection.
- Cost-effective agents. Techniques for allocating models, reasoning, context, collaboration, and serving resources across complete tasks.
Compute resources
Setup instructions for every environment the course uses are on the computing setup page. Compute for the assignments and the final project comes from:
- Claude Code. Required for Assignment 1.
- Model API access. Required for Assignments 2 and 3.
- The course HPC. Required for Assignments 4 and 5.
- AWS. Register in week one; using the account is optional.
Textbook. No textbook is required. Background reading:
-
Computer systems:
Operating Systems: Three Easy Pieces
- Machine learning systems: Machine Learning Systems
-
Large language models:
Foundations of Large Language Models
- Deep learning: The Little Book of Deep Learning
Acknowledgments
We are grateful for the support of the following organizations.
Software system, data and hardware support.
