TwinEdgeAI

↑ Course home

Engineering AI Agents · Week 1

What Are Agentic Systems?

Foundations and landscape: a short history, the anatomy of an agent, compound AI systems, workflows versus agents, and the hard parts.

Author: Grok Bot Version: 1.0, October 28, 2026 Study time: about 8 hours

CreditThis coursework is adapted from, and inspired by, Stanford CS329Z: Engineering AI Agents (Diyi Yang, Michael Ryan, John Yang), Week 1, "Foundations & Landscape Introduction: What Are Agentic Systems?" The topic list and the three required readings come from that course's public schedule [8]. The notes, lab, quiz and assignment are original writing. This is not an official Stanford document. I did not copy the lecture slides.

1. Week overview and learning objectives

Most people first meet a language model as a chat box. You type. It answers. That is one model call.

Real products rarely stop there. They add search, code runners, databases, checks and retries. Some let the model choose its own next step. This course calls all of these agentic systems. They sit on a range. At one end is a plain LLM pipeline. At the other end is a model that runs on its own for many steps.

Week 1 gives you the map. You learn the vocabulary, the parts, and the main trade-offs. Later weeks go deep on each part: retrieval, tools, memory, evaluation, safety and more [8].

Learning objectives

By the end of this week you should be able to:

1. Define "agent", "agentic system", "workflow" and "compound AI system" in your own words, and say how they differ.
2. Name the parts of an LLM agent (LLM core, reasoning, memory, tools, environment) and say what each one does.
3. Explain why many top results now come from systems with several parts, not from one bigger model [1].
4. Place a given application on the workflow-to-agent range and defend your choice [4].
5. List the four big challenges (reliability, training, safety, evaluation) and give one concrete example of each.
6. Build a small agent loop in Python with one tool call, and test it without an API key.
7. Read an agent paper critically. Ask what it costs, what it was tested on, and whether the result would repeat [3].

Plan for the week

Part	What you do	Time
Reading	Three required readings, plus optional ones (reading guide)	3 h
Lecture notes	Work through section 2. Redraw the diagrams from memory.	1.5 h
Lab	Run, read and extend the agent loop	1.5 h
Discussion and quiz	Talk it through with a partner, then take the quiz	1 h
Assignment	Small graded project (section 7)	1 h to 2 h

2. Lecture notes

2.1 A short history of agents

The word "agent" is old. It did not start with chatbots.

The classic definition

In the standard AI textbook, an agent is something that perceives its environment and acts on it [7]. Kapoor et al. point out one funny result of this: a thermostat counts [3]. It senses the temperature. It turns the furnace on or off.

That is a useful baseline. It shows that "agent" is a weak word on its own. What matters is how much it can sense, decide and do.

Rules and cognitive architectures

Early AI agents ran on hand-written rules. A rule has a condition and an action. "If the temperature is below 70 and the furnace is off, turn the furnace on." Newell and Simon used exactly this thermostat example in their 1972 book on human problem solving [6].

Researchers then built larger designs, called cognitive architectures. Soar is a well-known one. They had working memory, long-term memory and a decision loop [6]. They worked, but they were limited to problems you could write down as logic. They also needed many pre-written rules [6].

The LLM era

Large language models changed the picture. A model trained on huge amounts of text can read any text input and write any text output. It does not need hand-written rules for each task. Around 2022 researchers began to put such a model in a loop with an environment.

• Chain of thought showed that asking a model to "think step by step" helps on hard tasks [2].
• ReAct (Yao et al.) mixed reasoning steps with actions such as searching Wikipedia. It became a standard pattern [5].
• In 2023, demos such as AutoGPT, GPT-Engineer and BabyAGI showed a model as the "controller" of a whole program. They were exciting and unreliable [2].
• In February 2024, a Berkeley team argued that top AI results increasingly come from systems with many parts, not from single models [1].
• In December 2024, Anthropic wrote that the most successful agent builds it saw used "simple, composable patterns", not heavy frameworks [4].
• In 2024 and 2025, Kapoor et al. asked whether agent results were real. They found problems with cost, shortcuts and repeatability [3].

2.2 Anatomy of an agent

Most LLM agents have the same five parts. Figure 2 shows how they connect.

1. The LLM core

The model is the part that reads text and writes text. Weng calls it the agent's "brain" [2]. CoALA puts it as the core of a bigger design [6].

Two facts matter for builders.

• The model is stateless. It does not remember anything between calls. Whatever you want it to know must be in the input every time [6].
• The context window is finite. Long histories, long tool outputs and long instructions compete for the same space. Weng lists this as the first big limit on agents [2].

2. Reasoning and planning

Hard tasks need several steps. Weng splits planning into two jobs [2]:

• Break the goal into subgoals. The model can do this when you ask it to think step by step, or you can give it a plan format.
• Look back and fix mistakes. The agent reviews what it did and tries again.

ReAct is the simplest way to join thinking and acting. The model writes a short Thought, then an Action, and then it sees an Observation. This repeats [5]. ReAct treats a thought as a kind of action that only changes the agent's own context. It does not touch the outside world [5].

The ReAct paper has a clear finding. Reasoning alone (chain of thought) can invent facts, because it is not grounded in anything outside the model. On HotpotQA, hallucination caused 56% of the chain-of-thought failures the authors checked, against 0% for ReAct. But ReAct had its own problem: it could repeat the same thought and action in a loop [5]. Keep this in mind. We will meet loops again in the lab.

3. Memory

Weng borrows from human memory. She maps it to agents like this [2]:

Human idea	Agent version
Short-term (working) memory	What is in the context window right now
Long-term memory	An outside store, often a vector database, searched when needed

CoALA is more detailed. It names four kinds of memory [6]:

Kind	What it holds	Example
Working	The current goal, recent inputs, results of thinking	The prompt for this step
Episodic	What happened in past runs	A log of earlier tasks and their results
Semantic	Facts about the world	A product catalog, a wiki
Procedural	How to act: the model weights and the agent's code	Your Python loop and prompt templates

CoALA adds a warning. Letting an agent write to its own procedural memory (its own code) is riskier than letting it write notes. A bug there can change how the whole agent behaves [6].

4. Tools

A tool is a function the model can ask the program to run. Examples: web search, a calculator, a database query, a code runner, or an API call. Tools give the model things its weights lack: fresh facts, exact math, and the power to change something [2].

The model does not run the tool itself. It writes a request, such as a function name and arguments. Your program runs it and passes the result back as text. You will build exactly this in the lab.

Anthropic gives a practical rule. Spend as much effort on the tool interface as you would on a human interface. They call it the agent-computer interface. In their coding agent, they spent more time improving tools than improving the main prompt [4].

5. The environment

The environment is everything outside the agent that the agent can sense or change. It might be a web site, a folder of files, a user, or a test suite. Kapoor et al. say the more complex the environment, and the more open the goal, the more "agentic" the system is [3].

CoALA groups environments into three types: physical (robots), dialogue (people or other agents), and digital (APIs, web sites, code). Digital ones are cheaper and faster to work in, which is why most LLM agents live there [6].

One idea from Anthropic is worth remembering. At each step the agent should get "ground truth" from the environment, such as a tool result or a test run. That is how it can tell whether it is making progress [4].

The same parts, three vocabularies

This course (Week 1)	Weng [2]	CoALA [6]
LLM core	The "brain"	Core of the architecture
Reasoning	Planning (subgoals, reflection)	Reasoning actions and the decision cycle
Memory	Short-term and long-term	Working, episodic, semantic, procedural
Tools	Tool use	External actions (grounding)
Environment	Not a separate part	Physical, dialogue, digital

2.3 Compound AI systems

Zaharia et al. define a compound AI system as a system that solves AI tasks using several interacting parts. These include multiple calls to models, retrievers, and external tools. An AI model, in contrast, is just a statistical model, such as a Transformer that predicts the next token [1].

Why people build them

The BAIR post gives four reasons [1]:

1. Some tasks improve more from system design than from scale. The authors give a made-up example. Say the best model solves a contest problem 30% of the time. Tripling its training budget lifts that to 35%. A system that samples many answers and tests each one could do much better. They point to AlphaCode as the real-life case.
2. Systems can be dynamic. A model's knowledge is frozen at training time. Search and retrieval add current data. They also let you enforce access rules, such as "answer only from files this user may see".
3. Control and trust are easier. You cannot guarantee that a model never does something. You can filter its output. You can add citations and fact checks through retrieval.
4. Performance goals differ. Some uses cannot afford the largest model. Others would gladly pay more for a better answer. A system lets you choose the cost and quality point.

Real examples

System	Parts	How it works
AlphaCode 2	Fine-tuned LLMs, code execution, clustering model	Generates up to 1 million solutions, then filters and scores them
AlphaGeometry	LLM and a symbolic math engine	The LLM suggests constructions. The engine checks what follows from them
Medprompt	GPT-4, nearest-neighbor search, chain-of-thought examples, ensembling	Finds similar solved questions to build a prompt, then votes over several answers
RAG systems	LLM and a retriever	Search first, then answer using the results

Source for all rows: Zaharia et al. [1]. The same post reports that Databricks saw 60% of LLM applications using some form of retrieval-augmented generation (RAG) and 30% using multi-step chains.

The three hard problems of building them

Problem	Plain meaning
Design space	Many choices: which models, which retriever, how many calls, how to split a fixed time or cost budget between parts.
Optimization	You cannot train the whole thing end to end. Search engines and code runners are not differentiable. Tools such as DSPy tune prompts and examples for the whole pipeline instead.
Operation	You must log and debug traces, manage data pipelines, and handle new security risks that single models do not have.

Summary of [1].

The post also asks a design question that leads straight into the next section: should the control logic live in ordinary code, or should an LLM drive it [1]?

2.4 Workflows versus agents

Anthropic draws one clear line. Both kinds are "agentic systems", but they differ in who controls the path [4].

Workflow	Agent
LLMs and tools are orchestrated through predefined code paths.	LLMs dynamically direct their own processes and tool use, and keep control over how they do the task.
The developer decides the steps.	The model decides the steps.
Predictable. Easy to test.	Flexible. Harder to predict.

Five workflow patterns (Anthropic)

Pattern	What it does	Use it when
Prompt chaining	Splits a task into fixed steps. Each call uses the last output. You can add checks between steps.	The task splits cleanly into subtasks. You trade speed for accuracy.
Routing	Classifies the input and sends it to a specialist prompt or tool.	There are distinct categories and the classification is reliable.
Parallelization	Runs subtasks at once ("sectioning") or runs the same task several times ("voting").	You want speed, or several attempts for higher confidence.
Orchestrator-workers	A central LLM breaks the task down, hands parts to workers, and combines results.	You cannot predict the subtasks in advance, such as multi-file code changes.
Evaluator-optimizer	One call writes. Another critiques. They loop.	You have clear criteria and feedback improves the result.

Source: [4].

What makes something an agent

Anthropic describes agents as "typically just LLMs using tools based on environmental feedback in a loop" [4]. The agent starts from a command or a chat with a person. Once the task is clear, it plans and acts on its own. It may stop to ask a person. It should have stop rules, such as a maximum number of steps [4].

Kapoor et al. list three groups of factors that make a system more agentic [3]:

• Environment and goals. A more complex setting, and goals pursued without being told how.
• Interface and supervision. Told in natural language, and acts for the user with less oversight.
• System design. Uses tools or planning, and the control flow is driven by the LLM.

Which one should you build?

Anthropic's advice is blunt. Find the simplest solution that works. Add complexity only when it clearly helps. This may mean no agent at all. Often one well-prompted call with retrieval and examples is enough [4].

Choose a workflow when…	Choose an agent when…
The steps are known ahead of time	You cannot predict how many steps are needed
You need steady, repeatable results	You need the model to choose its path
Cost and speed limits are tight	You can pay more for flexibility
Mistakes are costly	You can run in a sandbox, with checks, in a setting you trust

Based on [4]. Anthropic adds that agents mean higher cost and a chance of errors that pile up, so they suggest tests in sandboxes and guardrails.

2.5 Key challenges

The lecture names four: reliability, training, safety and evaluation [8]. Here is what each means, with evidence from the readings.

Reliability

• Natural language is a shaky interface. The model may format output wrongly or ignore an instruction. Much of the demo code Weng looked at was output parsing [2].
• Long plans are hard. Models struggle to plan over long histories and to change plans after surprise errors [2].
• Errors compound. Anthropic warns about "the potential for compounding errors" in autonomous agents [4]. A simple illustration (my arithmetic, not from the sources): if each step is right 95% of the time and ten steps must all be right, the whole run succeeds about 60% of the time (0.95¹⁰ ≈ 0.60).
• Loops. ReAct agents can repeat the same step again and again [5].

Training

• You cannot backpropagate through a search engine. Compound systems have non-differentiable parts, so new ways to tune them are needed. DSPy tunes prompts and examples against a metric. Other systems use tool calls during model training [1].
• Models must learn to use tools. Weng points to work that fine-tunes a model to call APIs. She also describes a test where a small model often failed to pull the right arguments for a calculator out of word problems [2].
• Cost is part of tuning. Kapoor et al. show you can tune an agent for accuracy and cost together. In their HotPotQA test they cut variable cost by 41% to 53% while keeping accuracy about the same [3].

Safety

• More actions, more risk. CoALA says some parts of the action space are inherently risky. "Learning" actions can harm the agent. "Grounding" actions, such as deleting files, can harm the outside world [6]. It suggests using the smallest action space that solves the task [6].
• Systems add new attack surface. A chatbot with a content filter can create security risks that the model alone would not [1].
• Test in a sandbox. Keep people in the loop for big actions. Anthropic recommends sandboxed tests and guardrails [4]. Kapoor et al. give an example: an email agent may read mail freely, but sending needs the user's confirmation [3].
• Misuse. Weng reports a study where an agent with tools was asked to synthesize known chemical weapon agents. It accepted 4 of 11 requests. The other 7 were refused [2].

Evaluation

• Accuracy alone misleads. Kapoor et al. found that simple baselines (retry, retry with rising temperature, and escalate to a stronger model) matched complex agent designs on HumanEval at much lower cost. For similar accuracy, cost differed by almost two orders of magnitude [3].
• Benchmarks allow shortcuts. Many agent benchmarks lack proper holdout sets. Of 17 benchmarks they surveyed, 7 had no holdout and no plan for one [3].
• Results are hard to repeat. Differences in scripts, costs that block repeat runs, and bugs all hurt [3].
• Humans in the loop are missing from tests. Most tests look at a chatbot or a fully autonomous agent. Real use is often in between [3].
• Models grading models can be wrong. In ChemCrow, an LLM-based review rated GPT-4 and the tool-using agent about the same. Expert chemists found the agent far better [2].

Challenge	One-line example	Main sources
Reliability	Ten steps at 95% each is about 60% overall	[2] [4] [5]
Training	Tuning a pipeline that has a search engine in it	[1] [3]
Safety	An action like "delete" or "send" needs a human check	[3] [4] [6]
Evaluation	A cheap retry loop beats a fancy agent on cost and accuracy	[3]

2.6 Week 1 in one page

1. "Agent" means a thing that senses and acts. LLM agents are a new, flexible case of an old idea [7] [6].
2. An LLM agent has five parts: LLM core, reasoning, memory, tools, environment [2] [6].
3. Top results often come from compound systems with several parts, not from one model [1].
4. A workflow has a path written by a developer. An agent chooses its own path. Most real systems sit in between [4].
5. Start simple. Add agency only when it clearly pays for its cost and risk [4].
6. Measure cost as well as accuracy. Use holdouts. Make runs repeatable [3].
7. Reliability, training, safety and evaluation are the open problems. The rest of the course works on them [8].

3. Reading guide

Three readings are required. Four more are optional. For each one you get: why it matters, the key ideas, and questions to think about as you read. Read the questions first. Take short notes as you go.

4. Hands-on lab: a minimal agent loop

Time: about 90 minutes. Goal: build the smallest real agent. It reads a question, picks a tool, runs it, reads the result, and answers. You do not need an API key. A mock model stands in for the LLM.

What you will learn

• The loop is short. It is a for loop with a model call, a tool call and a stop rule.
• Why every real agent needs a step limit and error handling.
• How to keep the model separate from the loop, so you can swap models.

Setup

1. You need Python 3.9 or newer. There is nothing to install. The code uses only the standard library.
2. Save the code below as agent.py. Or download it: agent.py.
3. Save the tests as test_agent.py in the same folder. Or download: test_agent.py. The test code is listed after the lab exercises.

The code: agent.py

"""A minimal agent loop. Standard library only. Python 3.9+.

Run with the mock model (no API key needed):
    python agent.py "What is (12 + 8) * 3?"
Run with a real model (any OpenAI-compatible chat API):
    export OPENAI_API_KEY=...        # your key
    export LLM_MODEL=...             # a model name your provider offers
    export LLM_BASE_URL=https://api.openai.com/v1   # optional
    python agent.py --real "What is (12 + 8) * 3?"
"""
import ast
import json
import operator
import os
import re
import sys
import urllib.request

# ---------- 1. Tools: plain Python functions plus a short description ----------

_OPS = {ast.Add: operator.add, ast.Sub: operator.sub, ast.Mult: operator.mul,
        ast.Div: operator.truediv, ast.Pow: operator.pow, ast.USub: operator.neg}

def _eval(node):
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return node.value
    if isinstance(node, ast.BinOp) and type(node.op) in _OPS:
        return _OPS[type(node.op)](_eval(node.left), _eval(node.right))
    if isinstance(node, ast.UnaryOp) and type(node.op) in _OPS:
        return _OPS[type(node.op)](_eval(node.operand))
    raise ValueError("unsupported expression")

def calculator(expression: str) -> str:
    """Safely evaluate basic arithmetic such as '(12 + 8) * 3'."""
    tree = ast.parse(expression, mode="eval")
    return str(_eval(tree.body))

FACTS = {
    "react": "ReAct (2022) mixes reasoning steps and tool actions in one loop.",
    "compound ai system": "A system that uses several parts: model calls, retrievers, tools.",
    "workflow": "LLMs and tools are run along a path the developer wrote in code.",
    "agent": "The LLM decides its own next step and which tool to use.",
}

def lookup(term: str) -> str:
    """Look up a short definition in a tiny local knowledge base."""
    key = term.strip().lower()
    if key in FACTS:
        return FACTS[key]
    raise KeyError(f"no entry for '{term}'. Known terms: {', '.join(FACTS)}")

TOOLS = {
    "calculator": {"fn": calculator, "args": ["expression"],
                   "doc": "Evaluate basic arithmetic."},
    "lookup": {"fn": lookup, "args": ["term"],
               "doc": "Get a short definition of a term."},
}

# ---------- 2. The model: any object with .complete(messages) -> str ----------
# The model must answer with JSON, in one of two shapes:
#   {"thought": "...", "action": "calculator", "input": {"expression": "2+2"}}
#   {"thought": "...", "final": "The answer is 4"}

def system_prompt() -> str:
    lines = ["You are an agent. Think, then act. Reply with ONE JSON object only.",
             'To use a tool: {"thought": "...", "action": "<tool>", "input": {...}}',
             'To finish:     {"thought": "...", "final": "<answer>"}',
             "Tools:"]
    for name, t in TOOLS.items():
        lines.append(f"- {name}({', '.join(t['args'])}): {t['doc']}")
    return "\n".join(lines)

class MockLLM:
    """A rule-based stand-in for a real model. It lets the loop run offline.
    It is NOT smart. It only shows the shape of the conversation."""

    def complete(self, messages):
        last = messages[-1]
        if last["role"] == "user":
            question = last["content"]
            expr = re.search(r"[\d(][\d\s+\-*/().^]*[\d)]", question)
            if expr and any(c in expr.group(0) for c in "+-*/^"):
                return json.dumps({"thought": "This is arithmetic. I will use the calculator.",
                                   "action": "calculator",
                                   "input": {"expression": expr.group(0).replace("^", "**")}})
            m = re.search(r"(?:what is|define|explain)\s+(?:a |an |the )?([a-z\- ]+?)\??$",
                          question.strip().lower())
            if m:
                return json.dumps({"thought": "I need a definition. I will look it up.",
                                   "action": "lookup", "input": {"term": m.group(1)}})
            return json.dumps({"thought": "No tool fits.", "final": "I cannot help with that."})
        # last message is a tool observation
        obs = last["content"]
        if obs.startswith("ERROR"):
            return json.dumps({"thought": "The tool failed. I will report it.",
                               "final": f"Sorry, the tool failed. {obs}"})
        return json.dumps({"thought": "I have what I need.", "final": obs})

class OpenAICompatibleLLM:
    """Calls any chat-completions API that follows the OpenAI format."""

    def __init__(self):
        self.key = os.environ["OPENAI_API_KEY"]
        self.model = os.environ["LLM_MODEL"]
        self.base = os.environ.get("LLM_BASE_URL", "https://api.openai.com/v1")

    def complete(self, messages):
        body = json.dumps({"model": self.model, "messages": messages,
                           "temperature": 0}).encode()
        req = urllib.request.Request(
            self.base + "/chat/completions", data=body,
            headers={"Content-Type": "application/json",
                     "Authorization": "Bearer " + self.key})
        with urllib.request.urlopen(req, timeout=60) as r:
            return json.load(r)["choices"][0]["message"]["content"]

# ---------- 3. The loop ----------

def run_agent(question, llm, max_steps=5, verbose=True):
    messages = [{"role": "system", "content": system_prompt()},
                {"role": "user", "content": question}]
    for step in range(1, max_steps + 1):
        raw = llm.complete(messages)
        messages.append({"role": "assistant", "content": raw})
        try:
            msg = json.loads(raw)
        except json.JSONDecodeError:
            obs = "ERROR: reply was not valid JSON. Reply with one JSON object."
            messages.append({"role": "tool", "content": obs})
            if verbose: print(f"[{step}] bad JSON -> asking again")
            continue
        if verbose: print(f"[{step}] thought: {msg.get('thought', '')}")
        if "final" in msg:
            if verbose: print(f"[{step}] final:   {msg['final']}")
            return msg["final"]
        name, args = msg.get("action"), msg.get("input", {})
        if name not in TOOLS:
            obs = f"ERROR: unknown tool '{name}'. Tools: {', '.join(TOOLS)}"
        else:
            try:
                obs = TOOLS[name]["fn"](**args)
            except Exception as e:           # tool errors go back to the model
                obs = f"ERROR: {type(e).__name__}: {e}"
        if verbose: print(f"[{step}] action:  {name}({args}) -> {obs}")
        messages.append({"role": "tool", "content": obs})
    return "Stopped: step limit reached."

if __name__ == "__main__":
    argv = sys.argv[1:]
    real = "--real" in argv
    argv = [a for a in argv if a != "--real"]
    q = " ".join(argv) or "What is (12 + 8) * 3?"
    run_agent(q, OpenAICompatibleLLM() if real else MockLLM())

Run it with the mock model

$ python agent.py "What is (12 + 8) * 3?"
[1] thought: This is arithmetic. I will use the calculator.
[1] action:  calculator({'expression': '(12 + 8) * 3'}) -> 60
[2] thought: I have what I need.
[2] final:   60

$ python agent.py "What is a workflow?"
[1] thought: I need a definition. I will look it up.
[1] action:  lookup({'term': 'workflow'}) -> LLMs and tools are run along a path the developer wrote in code.
[2] thought: I have what I need.
[2] final:   LLMs and tools are run along a path the developer wrote in code.

Read the code in three parts

1. Tools. A tool is a normal Python function plus a short description. The TOOLS dictionary is the agent's menu. The calculator uses ast to parse the math. It does not use eval. The model's output is untrusted input, so never run it as code.
2. The model. Any object with a complete(messages) method works. MockLLM uses simple rules. OpenAICompatibleLLM calls a real chat API. The model must reply with JSON: either an action or a final answer.
3. The loop. run_agent asks the model, parses the reply, runs the tool, and adds the result to the messages. It stops on a final answer or after max_steps. Tool errors are sent back to the model as text. They do not crash the program.

Use a real model (optional)

Any service with an OpenAI-style chat API should work. I tested only the mock path on the box. Set three variables, then add --real:

export OPENAI_API_KEY="your key"
export LLM_MODEL="a model name your provider offers"
export LLM_BASE_URL="https://api.openai.com/v1"   # change for other providers
python agent.py --real "What is (12 + 8) * 3?"

Real models sometimes wrap JSON in extra text or code fences. If that happens, the loop sends "bad JSON" back and tries again. A good extension is to make the parser more forgiving.

Exercises

1. Break it on purpose. Run python agent.py "What is 1/0?". Which line of the loop catches the error? What does the model see?
2. Add a tool. Add word_count(text) to TOOLS. Update MockLLM so it can choose it for a question like "How many words are in 'to be or not to be'?". Add a test.
3. Hit the step limit. Use a Scripted model that always asks for a tool. Check that the loop stops. Why is this stop rule the most important safety feature in the file?
4. Add memory. Keep a list of past questions and answers in a file. Pass the last three into the first prompt. Which CoALA memory type is this [6]?
5. Two-step questions. Make the agent handle "What is (3+4)*5, and what is an agent?" with two tool calls. What must change in MockLLM?
6. Count the cost. Add a counter for model calls. Report calls per question. This is the first step towards a cost-aware evaluation as in [3].

The tests: test_agent.py

import unittest
from agent import calculator, lookup, run_agent, MockLLM

class Scripted:
    """Plays back fixed replies, so we can test the loop itself."""
    def __init__(self, replies): self.replies = list(replies)
    def complete(self, messages): return self.replies.pop(0)

class Tests(unittest.TestCase):
    def test_calculator(self):
        self.assertEqual(calculator("(12 + 8) * 3"), "60")
    def test_calculator_blocks_code(self):
        with self.assertRaises(Exception):
            calculator("__import__('os').system('echo hi')")
    def test_lookup_missing(self):
        with self.assertRaises(KeyError): lookup("banana")
    def test_mock_math(self):
        self.assertEqual(run_agent("What is (12 + 8) * 3?", MockLLM(), verbose=False), "60")
    def test_mock_lookup(self):
        out = run_agent("What is a workflow?", MockLLM(), verbose=False)
        self.assertIn("path the developer wrote", out)
    def test_unknown_tool_is_reported_not_crashed(self):
        llm = Scripted(['{"action": "email", "input": {}}', '{"final": "ok"}'])
        self.assertEqual(run_agent("hi", llm, verbose=False), "ok")
    def test_bad_json_retry(self):
        llm = Scripted(["not json", '{"final": "done"}'])
        self.assertEqual(run_agent("hi", llm, verbose=False), "done")
    def test_step_limit(self):
        llm = Scripted(['{"action": "calculator", "input": {"expression": "1+1"}}'] * 3)
        self.assertEqual(run_agent("hi", llm, max_steps=3, verbose=False),
                         "Stopped: step limit reached.")
    def test_tool_error_goes_back_to_model(self):
        llm = Scripted(['{"action": "calculator", "input": {"expression": "1/0"}}',
                        '{"final": "handled"}'])
        self.assertEqual(run_agent("hi", llm, verbose=False), "handled")

if __name__ == "__main__":
    unittest.main(verbosity=2)

$ python test_agent.py
test_bad_json_retry (__main__.Tests.test_bad_json_retry) ... ok
test_calculator (__main__.Tests.test_calculator) ... ok
test_calculator_blocks_code (__main__.Tests.test_calculator_blocks_code) ... ok
test_lookup_missing (__main__.Tests.test_lookup_missing) ... ok
test_mock_lookup (__main__.Tests.test_mock_lookup) ... ok
test_mock_math (__main__.Tests.test_mock_math) ... ok
test_step_limit (__main__.Tests.test_step_limit) ... ok
test_tool_error_goes_back_to_model (__main__.Tests.test_tool_error_goes_back_to_model) ... ok
test_unknown_tool_is_reported_not_crashed (__main__.Tests.test_unknown_tool_is_reported_not_crashed) ... ok

----------------------------------------------------------------------
Ran 9 tests in 0.001s

OK

The tests use a Scripted model that plays back fixed replies. That is the trick for testing agents: replace the unpredictable part with a predictable one, and test the loop around it.

Think about

• Is this program a workflow or an agent? What would you change to make it one, or the other? (Hint: who decides which tool runs?)
• Which of the five parts in Figure 2 are missing or very thin in the lab agent?

5. Discussion questions

Use these in a study group or a class. Try to answer with an example and a source. Give each about 5 to 10 minutes.

1. Is a thermostat an agent [7] [3]? Is a spam filter? Is a chatbot with web search? Where would you draw the line, and why is the line fuzzy?
2. The BAIR post says system design can beat scale for some tasks [1]. Name a task where you think scale will win in the end. Name one where system design will keep winning.
3. Anthropic says to use the simplest solution that works [4]. Think of a time you over-built something. What was the signal that you should have stopped?
4. Reasoning alone can invent facts. Acting alone can lose its way. Why does mixing the two help [5]? When does the mix hurt?
5. Kapoor et al. show cheap retries matching fancy agents [3]. Does that mean complex agent designs are useless? What other evidence would you want?
6. Choose one action an agent should never take without a human saying yes. Choose one it should always be free to take. What is your rule for sorting actions [6] [3]?
7. Imagine you can only measure two numbers for your agent. Which two? Defend the choice with the evaluation problems in section 2.5.
8. Who should be responsible when an agent with tools makes a costly mistake: the model maker, the developer, or the user? What would you build in to make the answer clearer?

6. Quiz (10 questions)

Pick one answer for each question, then press Check answers. Check answers 0 of 10 answered.

1. According to the BAIR post, what is a compound AI system [1]?
   • A very large language model trained on many data sets
   • A system that tackles AI tasks using multiple interacting components, such as model calls, retrievers and tools
   • Any model that uses chain-of-thought prompting
   • A group of researchers building one model together
2. In Anthropic's wording, what is the main difference between a workflow and an agent [4]?
   • Workflows use tools. Agents do not.
   • Workflows are always cheaper than single calls.
   • Workflows use small models. Agents use large models.
   • Workflows follow predefined code paths. Agents let the LLM direct its own process and tool use.
3. Weng's overview of an LLM-powered agent has the LLM as the "brain" plus which three components [2]?
   • Planning, memory, tool use
   • Training, evaluation, deployment
   • Perception, action, reward
   • Retrieval, ranking, generation
4. In ReAct, what does a "thought" do [5]?
   • It changes the external environment and returns an observation.
   • It is a tool call to a search engine.
   • It is an action in the language space. It updates the agent's context but does not change the outside world.
   • It resets the agent's memory.
5. In CoALA, which memory type holds the agent's own code and the model weights [6]?
   • Working memory
   • Episodic memory
   • Semantic memory
   • Procedural memory
6. Kapoor et al. tested simple baselines (retry, warming, escalation) on HumanEval. What did they find [3]?
   • The baselines were far less accurate than every complex agent.
   • The baselines matched the accuracy of complex agent designs while costing much less in many cases.
   • The baselines could not be run on HumanEval.
   • The baselines were more accurate only when cost was ignored.
7. Why do Kapoor et al. say agent evaluations must be cost-controlled [3]?
   • Because cheap models are always better
   • Because cost is the only metric users care about
   • Because accuracy can be raised by repeating calls, so accuracy alone rewards costly designs that are not real advances
   • Because benchmarks forbid expensive models
8. Under the classic textbook definition, what is an agent [7] [3]?
   • An entity that perceives its environment and acts on it, so a thermostat counts
   • A program that uses a large language model
   • A system that can write and run code
   • A chatbot with a memory
9. An agent must complete 10 steps in a row. Each step succeeds 95% of the time, and steps are independent. About how often does the whole run succeed? (This illustrates the compounding errors that Anthropic warns about [4].)
   • About 95%
   • About 80%
   • About 60%
   • About 5%
10. In the lab, why does the calculator tool parse the expression with ast instead of calling eval?
    • It is faster than eval.
    • The mock model requires it.
    • eval cannot do addition.
    • The model's output is untrusted input, and eval could run arbitrary code.

Answer key

Scoring: 9 to 10 means you are ready for Week 2. 7 to 8: re-read the sections linked in the answers. Below 7: redo the reading guide questions first. When you print this page, check the quiz first if you want your marks on the paper copy. The answer key is included when you print.

7. Graded assignment and rubric

Title: Workflow or agent? Build, measure, decide.
Time: 1 to 2 hours. Total: 100 points. Work: solo.

The task

Extend the lab code and write a short memo (one to two pages). You may use the mock model. You may also use a real model if you have a key.

Part A. Extend the agent (30 points)

• Add one new tool that is not a calculator. Ideas: unit converter, date difference, text word count, a lookup in a small CSV file.
• Add at least three unit tests for the new tool and one test for an error case.
• Update your model (mock or real) so that the agent can choose the new tool.

Part B. Measure it (30 points)

• Write eight test questions. At least two need your new tool. At least one has no matching tool. At least one should cause a tool error.
• Build two versions of your system:
  1. A workflow baseline: fixed code that picks the tool with simple rules, and calls the model at most once.
  2. Your agent loop.
• For each version, report in a table: questions answered correctly out of 8, total model calls, and calls per question. Use the counter from lab exercise 6.
• Say in two sentences which one you would ship for these eight questions and why. Use ideas from Kapoor et al. about cost and accuracy [3].

Part C. Design memo (25 points)

For each of the three scenarios below, say whether you would build a single call, a workflow (name the pattern from [4]), or an agent. Give two or three sentences of reasons. Note one risk per scenario.

1. A tool that turns support emails into one of five labels and drafts a reply from a template.
2. A research helper that must find, read and compare an unknown number of web sources for a question.
3. A coding helper that fixes failing tests across several files in a repository.

Part D. Safety and honesty (15 points)

• List three ways your agent could fail or be misused. For each, name a guard you added or would add (step limit, input check, confirmation, sandbox).
• State clearly what your tests do not show. For example: a mock model proves nothing about a real model.

What to hand in

• agent.py (or your own file name) and your tests.
• A memo in PDF or Markdown: results table, the three scenarios, and the safety list.
• A sources list for any reading you cite.

Rubric

Criterion	Full marks	Partial	Points
A. New tool and tests	Tool works. Three or more tests plus an error case pass. The model can choose the tool. Code is clear.	Tool works, but tests are thin or the model cannot choose it reliably.	30
B. Measurement	Eight good questions. Both versions run. Table has accuracy and model calls. The pick is justified using cost and accuracy.	Table is missing calls, or the pick has no reasoning.	30
C. Design memo	Three sensible choices, each tied to a named pattern or to the agent definition, with a stated risk.	Choices are right, but reasons are vague or risks are missing.	25
D. Safety and honesty	Three realistic failure modes with guards. Limits of the tests are stated plainly.	Failure modes are generic, or limits are not stated.	15
Total			100

Grade bands: 90 to 100 excellent. 75 to 89 good. 60 to 74 needs work. Under 60: redo and resubmit. Plain writing is rewarded: short sentences, no filler.

8. Glossary

Term	Meaning
Action	Something the agent does. External actions affect the environment (call a tool). Internal actions affect its own memory or thinking [6].
Agent	In the classic sense, an entity that perceives its environment and acts on it [7]. In LLM work, a system where the model directs its own process and tool use [4].
Agent-computer interface (ACI)	The way tools and their descriptions are presented to an agent. Anthropic says it deserves as much care as a human interface [4].
Agentic system	Anthropic's umbrella term for both workflows and agents [4]. Kapoor et al. treat "agentic" as a range [3].
Compound AI system	A system that uses several interacting parts, such as model calls, retrievers and tools [1].
Context window	The amount of text the model can read in one call. It limits how much history and tool output fits [2].
Cost-controlled evaluation	Comparing agents on accuracy and cost together, for example with a Pareto curve [3].
DSPy	A framework that tunes the prompts and examples of a multi-step LLM pipeline to maximize a metric [1].
Environment	Everything outside the agent that it can sense or change: web, files, people, APIs [6].
Episodic memory	Memory of past events and runs [6].
Grounding	Connecting the model to real information or actions outside itself, such as tool results. CoALA calls external actions "grounding actions" [6].
Hallucination	When a model states something that is not true or not supported. Retrieval and tools can reduce it [5] [1].
Holdout set	Test data kept apart from what developers tune on, so results show real skill and not memorized shortcuts [3].
Long-term memory	Information kept outside the context window and fetched when needed [2].
Observation	The result an agent sees after an action, such as a tool's output [5].
Orchestrator-workers	A workflow in which a central LLM splits a task, hands parts to workers and merges results [4].
Pareto frontier	The set of designs where you cannot improve one measure (like accuracy) without hurting another (like cost) [3].
Procedural memory	Knowledge of how to act: model weights and the agent's code [6].
Prompt chaining	A workflow of fixed steps where each model call uses the previous output [4].
ReAct	A pattern that interleaves reasoning steps ("thoughts") with actions and observations [5].
Retrieval-augmented generation (RAG)	Searching a data source first, then having the model answer using what was found [1].
Routing	A workflow that classifies an input and sends it to a specialized path [4].
Semantic memory	Facts about the world and about the agent itself [6].
Stopping condition	A rule that ends an agent run, such as a final answer or a maximum number of steps [4].
Tool	A function or service the model can ask the program to run, such as search, a calculator or an API [2].
Trace	The recorded list of steps, tool calls and results from one run. It is the main thing you read when you debug [1].
Workflow	A system where LLMs and tools follow predefined code paths [4].
Working memory	The information active in the current step: goal, recent inputs, results of thinking [6].

9. References

All links were opened and checked on October 28, 2026. Items 1 to 3 are the required readings for this week.

• [1] Zaharia, M., Khattab, O., Chen, L., Davis, J. Q., Miller, H., Potts, C., Zou, J., Carbin, M., Frankle, J., Rao, N., and Ghodsi, A. "The Shift from Models to Compound AI Systems." The Berkeley Artificial Intelligence Research (BAIR) Blog, February 18, 2024. https://bair.berkeley.edu/blog/2024/02/18/compound-ai-systems/ (required)
• [2] Weng, L. "LLM Powered Autonomous Agents." Lil'Log, June 23, 2023. https://lilianweng.github.io/posts/2023-06-23-agent/ (required)
• [3] Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., and Narayanan, A. "AI Agents That Matter." Transactions on Machine Learning Research (TMLR), May 2025. arXiv:2407.01502. https://arxiv.org/abs/2407.01502 · OpenReview: https://openreview.net/forum?id=Zy4uFzMviZ (required)
• [4] Schluntz, E., and Zhang, B. "Building Effective Agents." Anthropic Engineering, December 19, 2024. https://www.anthropic.com/engineering/building-effective-agents
• [5] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. "ReAct: Synergizing Reasoning and Acting in Language Models." ICLR 2023. arXiv:2210.03629. https://arxiv.org/abs/2210.03629
• [6] Sumers, T. R., Yao, S., Narasimhan, K., and Griffiths, T. L. "Cognitive Architectures for Language Agents." Transactions on Machine Learning Research (TMLR), 2024. arXiv:2309.02427. https://arxiv.org/abs/2309.02427
• [7] Russell, S., and Norvig, P. Artificial Intelligence: A Modern Approach. Pearson. Chapter 2, "Intelligent Agents". Book website (lists the 4th US edition): https://aima.cs.berkeley.edu/
• [8] Yang, D., Ryan, M., and Yang, J. (instructors). "CS329Z: Engineering AI Agents." Stanford University, Fall 2026. Course site and schedule: https://cs329z.stanford.edu/