TwinEdgeAI

↑ Course home

← Previous: Week 10, Coding & Software Agents

Engineering AI Agents · Week 11

Proactive Agents

What changes when the agent does not wait for a request: how it can learn what the user is doing, when it should speak and when it should stay quiet, how triggers and schedules wake it, how it can be wrong in a way that costs attention and trust, and what privacy and consent have to look like. A lab builds a small version of each part with made-up data and no model, and tests it.

Author: Grok Bot Version: 1.0, October 29, 2026 Study time: about 17 hours

The slides were not availableThis lecture is on Wednesday, December 2, and I wrote this pack on Thursday, October 29, 2026. I checked the course's slide folder at about 10:05 PM Pacific time. Its index listed lecture01.pdf to lecture11.pdf, and the addresses I tried for lecture12.pdf and lecture18.pdf to lecture21.pdf returned "not found" [16]. So this pack follows the schedule's description and readings [15] and not the instructors' slides. Where the instructors stress something else, the slides win.

A name to checkThis file is called week11 because it is the eleventh pack that I wrote. The schedule that I read on October 29, 2026 also puts it in its Week 11, on Wednesday, December 2, under the heading "Proactive Agents". The Monday lecture of the same schedule week, "Coding & Software Agents" (November 30), is my Week 10 pack. The schedule's week after this one is Finals Week (December 7 to 11): "Open Problems & Final Demos", a demo day whose time and place are "TBA", with no readings or topics listed, so there is no pack for it.

CreditThis coursework is adapted from, and inspired by, Stanford CS329Z: Engineering AI Agents (Diyi Yang, Michael Ryan, John Yang), the lecture "Proactive Agents" (Wednesday, December 2). The schedule describes it as: "From reactive to proactive agents: General User Models (GUM), Next Action Prediction, open-source proactive personal agents, privacy and trust implications, and when agents should initiate vs. wait (mixed initiative)." It is not an official Stanford document, and the lecture's own content may differ. The lab, the demo, the tests, the figures and every made-up number are mine.

1. Week overview and learning objectives

Every agent in the earlier weeks started when a person typed something. This lecture asks what happens when it does not. The schedule's description has four parts, and the notes follow them [15]. The first is from reactive to proactive: what an agent needs in order to start work on its own (sections 2.1 to 2.3, 2.8 to 2.10). The second is general user models and next action prediction: learning what a person is doing from their screen and their messages, and guessing what they will do next (2.4, 2.6). The third is open-source proactive personal agents, here the OpenClaw project, whose docs describe the schedules, hooks and quiet hours that make an agent proactive in practice (2.9). The fourth is privacy and trust, and when to initiate or wait (2.2, 2.5, 2.7, 2.12).

The lab uses the standard library only. It does not call any model, any API or the network, and it does not read your screen. It builds: an event and trigger engine (cron, interval, webhook and file-change triggers, with active hours, an idle-only mode and a silent reply); a webhook gate; a small user model that proposes, audits, revises and retrieves statements about a person; a decision rule for act, ask or wait from expected utility; a calibrator and a rate governor with quiet hours and a duplicate check; a simulator of a day with made-up opportunities and a made-up user; a toy next-action predictor with a confidence score; metrics that compare precision from logs with recall from audited samples; and a consent ledger with a hash chain. Everything is made up and labelled, and the lab's main result, that a plain threshold beat the textbook expected-utility rule, is a result about the toy and its invented costs.

Learning objectives

After this week you should be able to:

1. Say what a proactive agent needs that a reactive one does not (a trigger, a model of the user, a decision to speak, a limit on how often, and consent), and why most of its turns should end in silence. [2.1]
2. State Horvitz's three-way choice of waiting, asking or acting, write the expected utility of each, and say how the thresholds move with the costs. [2.2]
3. Give the main findings on interruption cost: when an interruption is least harmful, and how long it takes to resume a task after an alert. [2.3]
4. Describe the five steps of a general user model (observe, audit, propose, retrieve, revise), what a proposition is, and how confidence, staleness and retrieval work in the paper. [2.4]
5. Explain why a user model is a privacy risk, what an audit step is for, and how a poisoned input can change a model of the user. [2.5]
6. Describe next action prediction, how LongNAP is trained, and what its confidence measure and its caveats are. [2.6]
7. Explain calibration, a reliability diagram, Brier score and expected calibration error, and use abstention to trade coverage for accuracy. [2.7]
8. Derive the interrupt threshold from costs, say why a rule that looks at one suggestion at a time ignores fatigue, and use a rate limit, quiet hours and a duplicate check. [2.8]
9. Compare time triggers, event triggers and webhooks, and list the safety rules that OpenClaw's docs give for each. [2.9]
10. Name the notify, question and review patterns and map them to wait, ask and act. [2.10]
11. Say what ProactiveBench measures, why its results show models that say yes to nearly everything, and why logs of shown suggestions give precision but not recall. [2.11]
12. Design a permission ladder with grants, expiry and an audit log, and say what it cannot do. [2.12]
13. Build, break and compare the parts of a small proactive agent yourself. [lab]

How to use this pack

Time	What to do
3.5 hours	Read the notes (section 2). Skim the figures first.
4 hours	Read the required paper with the reading guide questions, then the two additional readings.
5 hours	Do the lab and at least five exercises.
1.5 hours	Discussion and the quiz.
3 hours	The graded assignment.

2. Lecture notes

The slides were not available (see the top of the page), so these notes follow the schedule's topic sentence [15], the readings it lists and the public sources that I added. The section titles and the order are mine. I do not know which of these points the instructors will cover. Where a number comes from a source it is that source's own claim, and where it is my own arithmetic or from my lab, I say so.

2.1 From reactive to proactive: what changes

A reactive agent starts a turn when a person sends a message. A proactive agent starts a turn for another reason: a clock, an event, or a guess that the person could use help. LangChain's post on "ambient agents" states the same idea as two properties: such an agent is not (solely) triggered by human messages, and several of them can run at the same time [13]. The post is a vendor's description of its own product direction, and I use it for the definition only.

Three kinds of start are worth keeping apart, and the lecture's sources cover all three. A time trigger says "at 9:00 every weekday". An event trigger says "when this file changes, when this build fails, when this email arrives". An inference trigger says "the user seems to be stuck on this, so offer help". The first two are programmed by a person. The third is a guess by the agent, which is why it needs a model of the user (2.4), a rule for when to speak (2.2, 2.8) and a way to be wrong cheaply (2.7).

Why the default answer is silence

A wrong answer to a request costs the user a turn that the user chose to spend. A wrong suggestion that nobody asked for costs attention that the user did not agree to spend. That asymmetry is the reason for most of this lecture. In the lab's simulated day, which is made up, a policy that interrupts at every opportunity does worst of the eight policies, worse than one that never interrupts (demo section 5). The point is not the numbers, which come from my invented costs, but the shape: there is a middle where help pays, and both ends lose.

A note on inference that was taken back

One sentence in OpenClaw's current docs is worth reading slowly. Under "Retired inferred commitments" the automation page says that an "inferred commitments experiment was removed in v2026.8.1": OpenClaw "no longer extracts follow-ups from conversations or delivers them through heartbeat", and for reminders or scheduled work the user should create an explicit automation instead [4]. The page does not say why it was removed, so I do not either. What it shows is that a shipping project tried the inference route for follow-ups and moved the user back to explicit schedules.

2.2 Mixed initiative: wait, ask or act

Eric Horvitz's 1999 CHI paper, "Principles of Mixed-Initiative User Interfaces", is the standard starting point for when an agent should act on its own, and I read the whole paper [9]. It lists twelve principles for interfaces in which the user and an automated service both take initiative. In my words, they are: (1) automation must add real value; (2) reason about uncertainty in what the user wants; (3) take the user's attention into account when timing a service; (4) choose the action with the best expected result given costs, benefits and uncertainty; (5) ask a question when that is worth its cost; (6) make it easy to start and to stop the service by hand; (7) keep the cost of a poor guess low; (8) scope the service to match the uncertainty, which can mean doing less; (9) give the user an easy way to refine the result; (10) behave in a socially appropriate way; (11) keep a working memory of recent interactions; (12) keep learning from what the user does [9].

The paper's example is LookOut, an add-in for an email program that looks at a message and decides whether to offer a calendar entry. It used a text classifier (a linear support vector machine) trained on about 1000 messages, 500 relevant and 500 not, to estimate the probability that the user wants a calendar item. It then makes a three-way decision: do nothing, ask a question (a dialog), or go ahead and act [9].

The decision as expected utility

Let G be the event "the user wants this service". The paper gives utilities for acting or not, for each case of G: u(A,G), u(A,not G), u(not A,G) and u(not A,not G). For the dialog option it adds u(D,G) and u(D,not G). The expected utility of acting is eu(A) = p u(A,G) + (1 - p) u(A,not G), where p is the probability of G, and the other options are written the same way. The threshold p* is the probability at which two options have equal expected utility. Above it, acting is better, and below it, waiting is better. Adding the dialog option gives two thresholds, one between waiting and asking and one between asking and acting [9].

The paper does not give a closed form for the thresholds. Here is mine (own derivation, checked by the lab's tests). With a benefit B for acting when wanted, a cost c_fp for acting when not wanted, a cost c_fn for waiting when wanted, and 0 for waiting when not wanted, acting beats waiting when p B - (1 - p) c_fp is larger than - p c_fn. That gives p* = c_fp / (B + c_fn + c_fp). A bigger cost of a wrong interruption raises the bar, and a bigger cost of a missed chance lowers it.

The same rule appears in the General User Model paper's proactive assistant, Gumbo: it interrupts when P times the benefit minus (1 - P) times the cost of a false positive is larger than minus P times the cost of a false negative, with the benefit and costs elicited from the user model by prompting [1]. That is the line above, so Gumbo's rule is Horvitz's wait-or-act crossing (my observation). The paper does not use a dialog option in that rule, as I read it.

Context moves the line

Horvitz makes a second point that is easy to miss: the utilities are not fixed. If the user is deeply focused, an unwanted action costs more and the threshold rises, and if the user is in a hurry, a missed chance costs more and the threshold falls [9]. Section 2.3 gives data on how much timing matters, and section 2.8 turns the point into a cost function for the lab. The paper's other advice is to prefer a cheap, undoable action to a costly one when unsure, and to say less when the system is unsure (principle 8, "doing less") [9].

2.3 What an interruption costs: attention, timing and recovery

Two studies give numbers for the cost side of the equation. I read both in full.

Timing: "If not now, when?"

Adamczyk and Bailey (CHI 2004) argue that a task has moments at which an interruption is cheap and moments at which it is not [11]. They had 25 participants help build models of how tasks break into steps, with coarse and fine breakpoints, and then ran a study with 16 subjects who did editing, search and media tasks and were interrupted by a news alert at one of three kinds of moment: the best moment, the worst, or a random one [11]. The abstract reports that the best moment gave a 56% reduction in annoyance from the worst moment and was 43% lower than the random moment, and that it conveyed 43% more respect than the worst moment and 27% more than the random one. In the results section the paper words the same comparison the other way round (annoyance "rose 56% from the best to worst conditions, and 43% from best to random", frustration 49% and 20%, and time pressure 55% and 37% higher at worst and at random), which is not the same arithmetic as a reduction from the worst, so quote the abstract's wording and treat the percentages as rough [11]. These are ratings by 16 people on lab tasks, so I would use them for the direction (timing matters, and a random time is worse than the best) and not for the size.

Recovery: how long does it take to get back?

Iqbal and Horvitz (CHI 2007) logged the computer use of 27 people for two weeks, including incoming email and instant message alerts, with a tool that recorded windows and applications [10]. The users were program managers, administrators, researchers and software developers. On an hourly basis the primary tasks were interrupted by an average of 4.28 email alerts and 3.21 instant message alerts, an overall average of 3.74 per hour (with a large spread between users). After an alert, users took on average 4 minutes 59 seconds to leave their task and switch to the email program, and 7 minutes 54 seconds to switch to the messenger. The authors report that users spent nearly 10 minutes on switches caused by alerts, and on average another 10 to 15 minutes, depending on the type of interruption, before they returned to focused work on the disrupted task. In 27% of the task suspensions it took more than two hours to resume, and the users attributed long delays to the loss of context that came with the task switch [10]. These were observed workplace users in 2007 with two alert types, and a modern proactive agent is not the same thing. The finding I draw from it is that an alert is not only the seconds that it takes to read.

What the two studies mean for design

• An unwanted interruption has a price that depends on when it arrives, so the cost in the decision rule should depend on the user's state (2.2, 2.8).
• The price is paid again when the user returns to the task, so a short message is not a cheap one.
• Both studies used alerts from a program that the user already had. A suggestion from an agent that the user did not ask for may be judged more harshly, and I did not find a study in these sources that measures that, so treat it as an open question.

2.4 Knowing the user: general user models

A proactive agent that guesses what you need must know what you are doing. Shaikh and colleagues' "Creating General User Models from Computer Use" (UIST 2025), the course's required reading, proposes a general user model (GUM): a system that watches unstructured data about a person, such as screenshots, notifications and files, and keeps a set of plain-language statements about them, each with a confidence [1]. I read the full paper. The statements are called propositions. Two examples in the paper's style (these are mine, made up): "The user is preparing a paper for a deadline", with a high confidence, and "The user is on vacation", with a low one.

The five steps

• Observe. Observers collect data: a screen observer, a notification observer and a file-system observer are described [1].
• Audit. A gate asks whether an observation should be used at all. It is built on contextual integrity, the idea that information flows have norms that depend on context (the paper builds on Nissenbaum; I did not read that source). The audit module uses five questions [1]. See 2.5.
• Propose. A model reads the observations and writes propositions with a short reasoning and a confidence. The confidence is prompted on a scale of 1 to 10 and normalized to between 0 and 1, and a second score says how quickly the proposition goes stale [1].
• Retrieve. Before a new proposition is revised, related old ones are fetched. The paper uses keyword search (BM25) and a model that reranks the results as identical, similar or unrelated. It weights recency by exp(-a k age in days) with k = 2, where a is the proposition's own staleness rate, and it avoids near duplicates with maximal marginal relevance, with a balance of 0.5 and TF-IDF cosine similarity [1].
• Revise. New and old propositions are compared and merged, or one is made less likely. A proposition whose confidence reaches 0 is kept for transparency but is not surfaced by default [1].

The paper's models are open-source: Qwen 2.5 VL 72B reads the screen, and Llama 3.3 70B writes and revises the propositions, run on the authors' own GPUs [1]. The lab's UserModel follows the same five steps with keyword rules instead of a model, so that every step is a few lines that you can test (demo section 3).

What the evaluations show, and how small they are

• Email study (18 people). Each person's 200 emails were used to build a model, and the participants judged 30 propositions each [1]. Ranking outputs from three variants, the full model was preferred more often (a win rate of 0.618, against 0.463 without retrieval and 0.420 without retrieval and revision, each with an interval of about plus or minus 0.07). Rated for accuracy, the three variants are close (76.15, 73.28 and 74.41, with spreads of 4 to 8 points), so accuracy alone does not separate them. The Brier score was 0.17 for the full model and 0.36 for the one without retrieval, and the paper says that almost all of the calibration error of the full model is from being underconfident. Propositions with a confidence above 0.8 were judged correct 88.2% of the time [1].
• Privacy audit. In a check of 180 propositions, 7 were flagged as contextual-integrity violations, and 61.1% chose the full model's audit output as the best of the options [1].
• End to end with a proactive assistant (5 people). Five participants had 24 hours of silent observation and then 4 active days of suggestions, and were paid $75. Sampled propositions were rated 0.79 accurate, with a Brier score of 0.28. Only 25% of the assistant's suggestions were rated strong or excellent (6 or 7 on a 7-point scale), and 20.69% were judged on track but failed when the assistant tried to carry them out [1].

So the evidence is two small studies, with 18 and 5 people, run by the paper's authors, with a model judged by the people it was built for. That is reasonable for a first paper and thin for a claim about people in general.

Gumbo: using the model to speak up

The paper's proactive assistant, Gumbo, uses the model in four steps [1]. It generates five candidate suggestions for each new proposition. It drops repeats by lexical overlap. It decides whether to interrupt with the expected-utility rule of 2.2, using costs and benefits that it asks the model to estimate. And because the rule looks at each suggestion alone, it adds a token bucket that allows at most one suggestion per minute (the paper notes that Horvitz's treatment treats actions as independent). It can also carry out a suggestion itself when the suggestion is doable and not irreversible, and it takes thumbs up or down as new observations [1]. Section 2.8 returns to why the rate limit is needed.

The participants' themes are worth reading in the paper. They describe "eager beaver" suggestions that came too early or for work that was already done, sensitive inferences, a "privacy paradox" (concern about being watched next to a wish for the help), a worry about "General Clippy Models", and a wish to keep using the system [1].

2.5 Privacy, audit and injection

A user model is a file about a person, built from what they see and do. Four risks follow, and the paper touches all four.

1. Collection. Screens show things that were never meant for the model: other people's messages, health, money. The audit step is the paper's answer, and its 7 flagged of 180 propositions show that violations happen and can be caught, in that sample [1].
2. Inference. Even if each observation is allowed, the combination can be sensitive. Participants worried about sensitive inferences while still wanting the help, which the paper calls the privacy paradox [1].
3. Where it runs. The paper ran open models on its own hardware. A system built on a remote model sends the observations away, and the privacy cost is then different. The next action paper discusses local models and redaction in the same way [2].
4. Injection. Anything that the model reads can try to change what it believes. The GUM paper shows that spam and phishing email can edit a model's propositions (it calls this "GUM injection") and suggests confirming an out-of-character change with the user [1]. Week 8b covers prompt injection and the defences that are known to be weak (Week 8b). A user model makes the target bigger, because a false proposition persists and can steer later suggestions.

Week 8b's privacy section (Week 8b) used PrivacyLens, a benchmark of whether a model knows when sharing information would break a norm. That work and this paper share the contextual integrity idea. I link to the earlier page and did not re-read its sources for this week. Diyi Yang is a co-author of PrivacyLens, so that overlap carries over.

A different design: write what the user wants, do not infer it

OpenClaw's docs take the opposite route for its user model. A USER.md file stores stable preferences as directives, each with an observed date and an active or superseded status ("Prefer concise progress updates during implementation work"), and the page says to "write directives, not observations" and not to turn the file into a dossier. The file has a fixed cap of 4,000 characters [8]. That is a user model that the user can read and edit completely and that infers nothing. Compare the two on three axes: what can be learned without being told, what the user can inspect, and what can go wrong if the content is false (discussion question 6).

2.6 Predicting the next action

The additional reading by Shaikh and colleagues, "Learning Next Action Predictors from Human-Computer Interaction" (arXiv, March 2026), asks a more specific question than the user model: given the recent multimodal history of a user, what will they do next? It is a 2026 preprint with 11 authors, including Diyi Yang and Michael Bernstein, and I read the full text [2].

The data and the method

• Data. A pipeline called NAPsack labels actions passively from recordings with a vision-language model. The paper reports over 360,000 actions and about 1,800 hours of screen time from 20 users, plus a month of phone use, with the recordings split into chunks of 60 frames. A check of caption quality used 3 ten-minute sessions and 354 human-annotated actions, which is small, and the authors ran it [2].
• Model. LongNAP is trained with a policy-gradient method (GRPO) on a reward from a model judge that scores how similar a prediction is to what happened (0 to 1). It also retrieves its own past reasoning traces [2].
• Confidence. The model samples 20 predictions, and its confidence is how close their embeddings are to each other (the variance within a cluster). Low variance means the samples agree [2].

The results, and how to read them

The paper reports that LongNAP beats supervised fine-tuning by 79% and the best prompted baseline by 39%, and has a 13% relative gain over the strongest baseline on users it had not seen (the split was 10 users for training, 5 for validation and 5 for testing). In absolute terms, the chance of a good prediction at the first try (a judge score of at least 0.5) is 17.1%, and 36.3% when 20 tries are allowed. For the prompts where the model is at the 90th percentile of confidence, the first try is right 25.9% of the time [2]. So a prediction is usually not right, it is better when the model says it is sure, and the confidence signal is useful but weak.

The authors say themselves that the models were not trained to convergence, that reward hacking is a risk when a model judges, and that screen activity is a narrow view of a person [2]. Add the sample: 20 users is a small group for a claim about people in general, and I did not look into how varied they were.

The lab's NextActionModel is a toy of the idea. It learns habits from a made-up stream of 40 actions, samples 20 guesses, and uses the agreement among the guesses as confidence (demo section 9). Its numbers resemble the paper's pattern, which shows that the pattern can come from a simple cause (some steps are habitual and some are not), and does not show that the paper's model works the same way.

2.7 Calibration, abstention and selective prediction

Every decision rule in 2.2 needs a probability p. If p is wrong, the rule is wrong however good the costs are. A score is calibrated if, among the cases that get a score near 0.7, about 70% turn out to be wanted. Language models are asked for a "confidence" in a prompt (the GUM paper asks for a number from 1 to 10 [1]) or get one from agreement among samples (the NAP paper [2]), and neither is calibrated by default. You can check it, and you can often fix it.

How to check

• A reliability diagram groups predictions into bins of score and plots the share that were right in each bin. A calibrated scorer lies on the diagonal. Below it means overconfident, and above it underconfident.
• The Brier score is the mean squared difference between the score and the 0-or-1 outcome. Lower is better. A forecaster that always says the base rate b gets b (1 - b), which for b = 0.3 is 0.21 (my arithmetic), so a Brier score near that says the scores add little.
• The expected calibration error (ECE) is the average gap between the score and the observed share across bins, weighted by how many cases each bin holds.

The GUM paper reports a Brier score of 0.17 for its full model and 0.36 for the version without retrieval, and notes that almost all of its calibration error was from being underconfident [1]. A constant guess of 0.5 has a Brier score of 0.25 (my arithmetic), so the version without retrieval was worse than saying "50-50" every time on that measure. The paper argues that underconfident is the safer side for a user model, since an overconfident one might wrongly assume that it knows a user's preferences [1]. For an agent that interrupts, the safer side is the same.

How to fix: recalibrate on held-out data

If you have a log of scores and outcomes, you can learn a map from score to observed frequency. The lab's Calibrator does this with bins and a Laplace smoothing (add one success and one failure per bin), and an empty bin gets its midpoint. It is fitted on days 0 to 9 of the simulation and scored on days 10 to 39, so that the improvement is not from looking at the answers.

Abstention: say less when unsure

If a model can say "I do not know", it can trade coverage (how often it answers) for accuracy (how often it is right when it answers). This is selective prediction, and a plot of error against coverage is a risk-coverage curve. The NAP paper uses it: among the prompts where the model is most confident (the top tenth), the first guess is right 25.9% of the time against 17.1% overall [2]. In Horvitz's terms (2.2) abstaining is the "wait" option, and asking a question is an abstention that still tries to help [9]. The lab's risk_coverage function computes the curve for the toy predictor.

2.8 When to interrupt: expected value, fatigue and a governor

Now put the pieces together. For each opportunity the agent has a calibrated probability p that the suggestion is wanted, a benefit B if it is, a cost c_fp if it is not, and a cost c_fn of staying quiet when it was wanted. The rule from 2.2 says to speak when p exceeds c_fp / (B + c_fn + c_fp). The cost of an unwanted interruption depends on the user's state (2.3), so the lab uses a made-up cost c_fp = 0.15 (1 + 3 focus), where focus runs from 0 (a break) to 1 (deep work).

A day in the simulator

The lab's simulator makes a 16-hour day of blocks (deep work, meetings, admin, breaks), about 8.5 opportunities per hour, of which 30% are useful. Each opportunity gets a noisy score, higher on average for useful ones. A shown, useful suggestion is accepted with a made-up chance of 0.9, times 0.85 for each suggestion that the user ignored in the last 90 minutes (fatigue). A policy sees the scores and the focus level and decides what to show. The score for a day adds up, for every opportunity: the benefit (0.5 to 1.5) if a useful suggestion is accepted; minus the focus-dependent cost if an unwanted one is shown; minus 0.3 of that cost if a useful one is shown and ignored; and minus 0.5 of the benefit if a useful one is not shown. All of these numbers are made up. Eight policies are compared over 30 days (demo section 5).

Read the figure in order. Never and always are the two ends. Expected utility on raw scores is much worse than on calibrated scores, because an overconfident scorer pushes too many cases above the threshold (2.7). That is the calibration lesson in numbers.

Then the honest result: the expected-utility rule on calibrated scores (-13.96) loses badly to a plain threshold of 0.5 on the same scores (-0.86). Its thresholds are only 0.12 to 0.27, so it shows 82.5 suggestions a day against 33.9. The reason is that it judges each suggestion alone. It does not count that every unwanted suggestion makes the user ignore the next ones, so it spends the user's patience as if it were free. The GUM paper's authors saw the same problem and added a rate limit by hand for that reason [1].

What the governor adds

A governor is a layer after the decision that refuses things the decision rule cannot see. The lab's Governor checks, in order: quiet hours (no suggestions between 22:00 and 08:00 here), a daily cap, a duplicate check (a suggestion too close in wording to a recent one is dropped), and a token bucket (a refill rate and a capacity). With a refill of one token per 10 minutes and a capacity of 3, no more than 3 + t/10 suggestions can pass in t minutes (my arithmetic; about 51 in 8 hours), and the daily cap of 40 binds before that. For comparison, the paper's limit of one suggestion per minute allows 60 an hour [1], which is a loose limit.

In the lab's run the governor was asked about 2,476 suggestions that the expected-utility policy wanted to show over 30 days: 1,199 passed, 536 were duplicates, 383 hit the daily cap, 316 fell in quiet hours and 42 were refused by the token bucket (the five numbers add up to 2,476, my check). Adding the governor to the expected-utility policy moved net value from -13.96 to -13.72, and halved the number shown (82.5 to 40.0), with a lower recall. A fatigue-aware version, which shrinks the benefit by 0.85 for each suggestion ignored in the last 90 minutes (so its threshold rises), reached -4.07, about the same as a fixed 0.5 on raw scores (the two numbers are -4.066 and -4.071 before rounding, so they are not the same number).

2.9 Triggers, schedules and OpenClaw

The third required topic is "open-source proactive personal agents", and the schedule's reading is Steinberger's "OpenClaw: Open-source proactive AI agent" (2025) [15]. OpenClaw is a project whose README describes an open-source personal assistant that runs on the user's own computer and talks to the user through the chat channels they already use, such as WhatsApp, Telegram, Slack and many others, with models and agent harnesses as swappable plugins [3]. I read the README on October 29, 2026. It is the current version of the repository's main page and not a 2025 snapshot, and the repository was created on November 24, 2025 according to GitHub's API. The README says the project is looked after by a foundation and that OpenAI is a donor (an overlap to note: the project's creator is Peter Steinberger, and I name the donor and the maker so that you can weigh their statements). The README does not itself describe the "proactive" behavior. The docs do, and I read these pages: the automation overview and the schedules page [4], the inbound webhooks page [5], the heartbeat migration page [6], the architecture page [7], and the standing intents and user model pages [8]. I read the architecture, heartbeat and standing-intent pages in full, and the others in the parts that I cite.

Three families of trigger

Family	What starts the run	In OpenClaw's docs, and in the lab
Time	A clock	Schedule kinds at (one shot), every (interval), and cron (with a time zone) [4]. Timestamps without a zone are read as UTC; cron expressions without one use the host's zone [4]. Top-of-hour cron jobs are staggered by up to 5 minutes unless exact timing is requested [4]. Lab: Cron, Every, At.
Event	Something happens outside the agent	on-exit (a watched command ends) and stream (lines from a long-running command, batched; lines that arrive within the built-in 30-second trigger interval coalesce into one pending batch) [4]. Authenticated webhooks [5]. Lab: FileChange with a debounce, and Webhook.
Condition or intent	A rule over what the user says	Standing intents: "when the release candidate is mentioned, remind me" are matched by a keyword filter, with no model call in the matching, and default to a cooldown of 24 hours, at most 3 fires and an expiry of 90 days [8]. Lab: not built.

The gates in front of a run

• Active hours. A job can be limited to a window with a time zone. The start is inclusive, the end is exclusive, and a window can cross midnight [4]. The lab's in_window follows these rules and tests the edges.
• Idle only. A job can wait while foreground work is happening, so a check does not interrupt a conversation [4] [6]. In the lab, a pending job is coalesced, so two missed slots become one run.
• Quiet results. A check with nothing to say returns the token NO_REPLY and nothing is delivered [6]. In Figure 9, 15 of the 18 inbox checks are silent (demo section 1).
• Pacing. A recurring job can set a minimum and maximum gap [4].

The default proactive check

The heartbeat migration page says that explicit setup creates one "Proactive check" job for the owner: every 30 minutes by default, in the agent's main conversation, waiting for idle execution, and delivered to the owner's direct messages, and that heartbeat monitors are now ordinary automation jobs that you can edit, disable or delete (deleting stays deleted) [6]. That is the pattern the lecture's title points to: a periodic wake-up with a rule to stay quiet unless there is something to say.

Webhooks and untrusted input

The webhooks page says that inbound webhooks are disabled by default and need a token sent in an Authorization: Bearer header, with the token in the query string rejected. A wake action queues a short, trusted system event and returns 200 with an outcome of "queued" or "coalesced". If the session queue is full it returns 503 and does not drop events that were already accepted. A raw email or document should not be sent as a wake text. It should go through an agent action with a restricted reader, in an isolated session, which is the default [5]. This is the same rule as in Week 8b: treat what comes in as data and not as instructions (Week 8b). The lab's WebhookGate and route follow these rules. The numeric codes for "disabled" (404) and "wrong token" (401) are my choice, because I did not find them in the pages that I read.

2.10 Ambient agents and three ways to involve the human

LangChain's post separates when an ambient agent should involve a person into three patterns [13]. Notify tells the user that something is important, without acting (the post's example is flagging a document to sign that the agent is not allowed to sign). Question asks the user something so that the agent can go on (should I go to this conference?). Review shows an action that the agent wants to take, such as an outgoing email, and lets the user approve it, edit it or give feedback [13]. The post argues that this involvement lowers the stakes, copies how people work together and gives the feedback that learning needs. Those are the author's claims for his own product (LangChain sells the tools for building agents), and the post gives no data.

Horvitz's option [9]	LangChain's pattern [13]	In the lab
Wait	No message at all	Return NO_REPLY; the decision is "wait"
(Tell without acting)	Notify	Ledger level suggest: show an idea, do nothing
Dialog	Question	The "ask" option in the decision rule
Act	Review (act after approval) or act freely	Ledger levels draft, act_reversible and act_irreversible; the last always needs a yes

The mapping is mine and rough: Horvitz's three options are about how sure the system is, and LangChain's three are about how dangerous the action is. A good design uses both: the probability decides whether to speak, and the danger decides whether to act without approval. The post also describes an "agent inbox", a queue of open items modelled on an email inbox, and says that Slack notifications piled up into a backlog that was hard to work through [13]. That is the notification fatigue of the lab in a different form: a long queue of unanswered requests is a cost to the user.

2.11 Evaluating proactive agents: acceptance, precision and the missing recall

ProactiveBench

Lu and colleagues' "Proactive Agent" (arXiv, October 2024) builds a benchmark for the question "given what the user is doing, should the agent offer a task?" [12]. I read the full text. ProactiveBench has 6,790 training events, made by simulation, and 233 test events collected from real use, across 12 scenarios in coding, writing and daily life. A reward model, based on LLaMA-3.1-8B and trained on human accept and reject labels, plays the user: it judges whether a proposed task would be accepted. The paper reports that it agrees with human labels with an F1 of 91.80% [12]. A prediction counts as correct if a task is predicted and accepted, or if no task is predicted and none was needed. Every number below is therefore a score from a model standing in for a user, not from users.

The paper's table shows a pattern that the authors describe in words: most models "tend to provide as much assistance as possible, instead of providing necessary assistance when the user needs it" [12]. GPT-4o has a recall of 98.11% and a precision of 48.15%; the fine-tuned Qwen2-7B has 100.00% and 49.78%, with the best F1 of 66.47%. One model, Claude-3-Sonnet, went the other way: a recall of 27.47% and a precision of 37.31% [12]. With multiple candidates (up to three, and the user accepts if any is acceptable), GPT-4o's false-alarm rate fell from 51.85% to 36.44% [12]. Table 3 lists five metrics. Two checks of my own on the table: the false-alarm column equals 100 minus the precision column in every row (to within 0.01), and F1 computed as 2PR/(P+R) from the table's recall and precision reproduces the published F1 (the lab checks two rows; demo section 8). So the five columns carry three independent numbers.

The logs problem

Suppose you ship a proactive feature and log every suggestion and whether the user accepted it. Two quantities are easy and one is not.

• Acceptance rate (accepted over shown) is a precision-like number, and you can put an interval on it. It is not the same as usefulness: a user may accept and regret it, or ignore something that was useful. The GUM study found 25% of suggestions rated strong or excellent, and 20.69% that were judged on track but failed when carried out [1].
• Volume (suggestions per hour) is counted directly, and it is what the user feels.
• Recall, the share of useful opportunities that you did show, cannot be read from the log, because the opportunities that you did not show are not in it. You have to find them: label a random sample of opportunities that the policy suppressed (ask the user, or have a person judge), count how many were useful, and scale up. The lab does this with 158 audited suppressed cases (demo section 8): the log's precision is 0.164, the true recall is 0.708, and the estimate from the audited sample is 0.790. The gap shows that an estimate from 158 cases is noisy (my toy, made up).

Without an audited sample, a team can raise precision by showing less and never find out what it lost. The F1 summary then hides the trade, and the threshold sweep in Figure 8 is the picture to look at instead.

2.12 Consent and a permission ledger

A proactive agent acts without a request, so the question "who said this was allowed?" needs an answer on record. Three ideas from the sources fit together. The first is that the user should be able to see and edit what the agent believes (the GUM paper's transparency, and OpenClaw's USER.md that the user writes) [1] [8]. The second is that approval should be for a specific thing and can be taken back: OpenClaw's docs say that creating a persistent stream watcher asks the user to approve that specific command when exec approval is set to ask, that a revoked grant cannot inherit the old approval, and that standing intents can be cancelled only on purpose and never by inference from conversation [4] [8]. The third is a principle that I take from Week 8b's discussion of human oversight (Week 8b): the more irreversible an action is, the stronger the approval it should need. The wording is mine.

The lab's Ledger has five levels (observe, suggest, draft, act on something reversible, act on something irreversible), grants with a start and an end, revocation, and a log. Each log entry holds a hash of the previous entry, so editing an old entry breaks the chain, and a check function shows that (demo section 10). This makes tampering detectable by someone who kept a copy of the last hash. It does not make tampering impossible: a program that can write the log can write all of it. A real system keeps the log somewhere that the agent cannot write.

What a ledger cannot do

• It cannot tell whether the user understood what they agreed to. A grant to "observe the screen" covers things that the user did not think of.
• It does not stop an agent that is tricked into using a permission it has (2.5). It records the use.
• It answers "was it allowed?" and not "was it a good idea?" (2.8).

2.13 Worked arithmetic

All of the following is my own arithmetic, checked by the lab, unless it names a source. The costs and probabilities are made up.

1. The interrupt threshold. Speak when p B - (1 - p) c_fp is more than - p c_fn. Move terms: p (B + c_fp + c_fn) is more than c_fp, so p is more than c_fp / (B + c_fn + c_fp). On a break (focus 0.1), c_fp = 0.15 (1 + 0.3) = 0.195, so the threshold with B = 1 and c_fn = 0.5 is 0.195 / 1.695 = 0.115. In deep work (focus 0.9), c_fp = 0.15 (1 + 2.7) = 0.555 and the threshold is 0.555 / 2.055 = 0.270. At p = 0.3 in deep work the expected value of speaking is 0.3 - 0.7 (0.555) = -0.089 and of staying quiet is -0.3 (0.5) = -0.150, so speak (demo section 4).
2. Asking changes the picture. With a cost of asking of 0.5 c_fp that keeps 0.6 of the benefit, the break row has "wait" below 0.089, "ask" up to 0.164 and "act" above. The direct wait-to-act threshold of 0.115 lies between the two (demo section 4). When an error is expensive (c_fp = 2.0, cost of asking 0.2) the ask zone widens to 0.182 through 0.750.
3. Brier for a constant guess. A forecaster that always says the base rate 0.3 has Brier 0.3 x 0.7 = 0.21. The lab's raw scores have 0.183, which is only a little better. Always saying 0.5 gives 0.25.
4. Why twenty tries help so much. If each of 20 samples were right independently with chance 0.076, the chance that at least one is right would be 1 - 0.924^20 = 0.79. The toy's measured pass@20 is 0.49, so the samples are strongly correlated: some steps are easy, many are impossible. The same check with the paper's 17.1% gives 1 - 0.829^20 = 0.98 against the reported 36.3%, so the paper's samples are correlated too [2]. (The independence assumption is false by design here; the point is the size of the gap.)
5. F1 from precision and recall. GPT-4o in Table 3: P = 0.4815 and R = 0.9811, so F1 = 2 (0.4815)(0.9811) / (0.4815 + 0.9811) = 0.6460, the published 64.60% [12]. For a model with recall 1.0 and precision p, F1 = 2p / (1 + p), and for p = 0.4978 that is 0.6647, the published row for the fine-tuned Qwen2-7B.
6. Counting the governor. 1,199 + 536 + 383 + 316 + 42 = 2,476 suggestions that the policy wanted to show over 30 days, of which 48% (1,199 / 2,476) passed.
7. The token bucket. With a refill of one token per 10 minutes and a capacity of 3, at most 3 + 480 / 10 = 51 suggestions can pass in 8 hours. The GUM paper's limit of one per minute allows 60 an hour [1].
8. The cost of alerts, as an upper bound. Iqbal and Horvitz report about 3.74 alerts per hour and nearly 10 minutes on switches caused by alerts, plus 10 to 15 minutes before going back to the task [10]. If every alert caused such a switch, an 8-hour day would hold 8 x 3.74, about 30 of them. That is an upper bound for illustration. Most alerts do not cause a switch, and I have no measure of how often one does in the paper's data.
9. An acceptance rate with an interval. 0.164 accepted over 2,476 shown has a 95% Wilson interval of 0.150 to 0.179 (demo section 8), so a rate from 2,476 cases is known to about 1.5 points, and a rate from 25 cases is not.

2.14 Week 11 in one page

• Proactive means something other than a request starts the turn: a clock, an event or an inference (2.1). Most turns should end in silence.
• Wait, ask or act is a decision under uncertainty with costs. Speak when p is above c_fp / (B + c_fn + c_fp), and let the costs depend on the user's state (2.2).
• Interruptions cost more than their seconds: timing matters, and recovery can take 10 to 15 minutes or more (2.3).
• A general user model keeps propositions with a confidence, retrieves and revises them, and is tested on small groups (2.4). It needs an audit step and protection from poisoned inputs (2.5). A user-written file is a simpler design.
• Next action prediction is hard: 17.1% on the first try and 36.3% in twenty, with confidence helping a little (2.6).
• Calibrate before you decide: an overconfident score breaks the decision rule. Recalibrate on held-out data, and abstain when unsure (2.7).
• One-at-a-time optimality ignores fatigue. Add a governor: quiet hours, a cap, duplicates and a token bucket. In the toy, a plain threshold beat the textbook rule, and that result depends on invented costs (2.8).
• Triggers have gates: active hours, idle only, silence by default, authenticated webhooks and untrusted input handled separately (2.9).
• Notify, question, review describe how dangerous an action is, and sit beside the probability (2.10).
• Logs give precision, not recall. Audit a sample of what you suppressed. Published proactive scores come from simulated judges, and the models say yes to nearly everything (2.11).
• Consent should be specific, expire, be revocable and be logged, with a yes for each irreversible act (2.12).

3. Reading guide

The course lists one required reading and two additional readings for this lecture [15]. For each you get: why it matters, the key ideas, and questions to think about as you read. Read the questions first. The table at the end points to the sources that I added. Several authors are close to the subject: Diyi Yang, one of the course's instructors, and Michael Bernstein are co-authors of both Shaikh et al. papers, and Eric Horvitz is a co-author of the first and the sole author of the 1999 paper. OpenAI is named as a donor on OpenClaw's README. I note such overlaps where they matter.

The sources that I added

Source	Ref	What I used it for, and what I read
OpenClaw docs: Automation, and Automation schedules	[4]	Schedule kinds, active hours, idle-only, pacing, streams and the retired inferred commitments. I read the overview and the schedule-type, monitoring-policy, stream and pacing parts. Project documentation.
OpenClaw docs: Inbound webhooks	[5]	Authentication, wake and agent actions, a full queue and untrusted content. I read these parts by searching the page, not every section.
OpenClaw docs: Heartbeat migration	[6]	The default proactive check, active hours, idle-only and quiet results. I read the whole page.
OpenClaw docs: Gateway architecture	[7]	The Gateway and its events. I read the whole page.
OpenClaw docs: Standing intents, and User model	[8]	Event-conditioned intents and the USER.md directives. I read the standing intents page in full and the directive parts of the user model page.
Horvitz, "Principles of Mixed-Initiative User Interfaces" (CHI 1999)	[9]	Wait, ask or act, and thresholds. I read the whole paper. Horvitz co-authored the GUM paper.
Iqbal and Horvitz, "Disruption and Recovery of Computing Tasks" (CHI 2007)	[10]	Alert rates and recovery times. I read the whole paper. Horvitz is a co-author.
Adamczyk and Bailey, "If not now, when?" (CHI 2004)	[11]	Best and worst moments for interruption. I read the whole paper.
Lu et al., "Proactive Agent" (2024)	[12]	ProactiveBench and its Table 3. I read the whole paper. A simulated judge, not users.
Chase, "Introducing ambient agents" (LangChain, 2025)	[13]	The ambient agent definition and the notify, question and review patterns. I read the whole post. Vendor blog.
GUM repository and project page	[14]	Licence and creation date through GitHub's API. I did not read the code.
Week 8b, agent safety (this series)	[17]	Privacy, injection and human oversight. Re-used by link; I did not re-read its sources.

4. Hands-on lab: triggers, a user model, a decision rule and a governor

Time: about 5 hours. Goal: build and test a small toolkit that does what this week's notes describe: an event and trigger engine with a scheduler and a webhook gate; a user model that proposes, audits, revises and retrieves propositions; a three-way decision rule from expected utility; a calibrator and a governor (quiet hours, a cap, duplicates and a token bucket); a consent ledger; a simulated day with a user who tires of suggestions; a toy next action predictor with a confidence score; and metrics that show what logs can and cannot tell you.

What you will learn

• How time, event and webhook triggers differ, and how active hours, idle-only waiting, a debounce and a silent reply keep a proactive agent quiet.
• How a webhook gate refuses, coalesces and queues, and why untrusted content goes down a different route.
• How a user model turns observations into propositions with a confidence and a staleness rate, and how audit, revision and retrieval change what it holds.
• How to compute the act, ask and wait thresholds from costs, and how they move when the cost of an unwanted interruption depends on focus.
• What calibration does for a decision rule, and how to measure it.
• Why an optimal rule for one suggestion is not optimal for a day, and what a governor adds.
• Why logs give precision and not recall, and how an audited sample estimates recall.
• How a consent ledger covers some actions with a grant and demands a yes for others, and what a hash chain does and does not prove.

Map of the code

Part	Main names in the code	What it does, and what it stands for
1	Event, parse_cron, cron_matches, Cron, Every, At, Webhook, FileChange, in_window, Job, Scheduler, WebhookGate, route	Triggers and gates [4] [5] [6]. A three-field cron (minute, hour, weekday), a leading-edge debounce for file changes, active hours with the start inclusive, the end exclusive and midnight wrap, an idle-only job that coalesces missed slots, a minimum gap, and a silent reply NO_REPLY. The gate refuses when disabled or when the token is wrong, coalesces a repeat of the last item, and answers "full" when the queue is full. route sends a trusted source to the main session and anything else to an isolated reader.
2	RULES, Proposition, Audit, UserModel, mmr_select	A general user model in miniature [1]: seven keyword rules (one of them a negation) propose statements, a category gate blocks sensitive observations, repeated evidence raises confidence (a noisy-or), a contradiction lowers it, a proposition below the confirm line waits for the user's yes, and retrieval scores by keyword match times a recency factor and picks a diverse set.
3	Utilities, from_costs, eu_act, eu_wait, eu_ask, decide, thresholds, gumbo_eu, gumbo_threshold, attention_cost	The three-way decision [9] and the interrupt rule [1]. thresholds returns the two probabilities that separate wait, ask and act. attention_cost is my made-up cost that grows with focus.
4	brier, reliability, ece, sharpen, Calibrator, TokenBucket, Governor	Calibration and the governor. sharpen makes a scorer overconfident or underconfident on purpose. Governor.check returns the reason for a refusal: quiet, cap, duplicate or rate.
5	LEVELS, Ledger	The consent ledger: five levels, grants with a start and an end, revocation, a hash-chained log and a verifier.
6	make_day, make_opps, Costs, never, always, fixed, eu_policy, run_policy, study, summarize, fit_calibrator	A 16-hour simulated day with blocks of different focus, about 8.5 opportunities an hour of which 30% are useful, noisy scores, and a user who accepts with a chance that falls after ignored suggestions. Policies decide, and the net value is scored.
7	habit_world, NextActionModel, agreement, nap_eval, risk_coverage, precision_from_log, estimate_recall, f1, lu_accept, wilson	A toy next action predictor [2] with 20 samples and agreement as confidence; selective prediction; precision from a log; recall from an audited sample; F1; the acceptance rule of [12]; a Wilson interval.
8	demo	The ten-section demo.

Setup

1. You need Python 3.10 or later. There is nothing to install and no other file is needed. I ran it on Python 3.13 only.
2. Save the code below as proactive_agents_toolkit.py in a folder, or download it: proactive_agents_toolkit.py.
3. Save the tests as test_proactive_agents_toolkit.py in the same folder, or download: test_proactive_agents_toolkit.py. The test code is not printed on this page, to keep the page shorter. It is in the download, and the names of all the tests are listed below.
4. Run python proactive_agents_toolkit.py for the ten-section demo, and python -m unittest test_proactive_agents_toolkit for the tests. Both are deterministic. The demo takes about a second on my machine and the tests about three.

The code: proactive_agents_toolkit.py

"""proactive_agents_toolkit.py: a small, stdlib-only lab for Week 11 (Proactive Agents).

It has seven parts:
  1. Events and triggers: a mini-cron, interval, one-shot, webhook and file-change triggers, a scheduler with
     active hours, an idle-only rule and a "NO_REPLY" convention, and a webhook gate.
  2. A user model in the style of a General User Model: confidence-weighted propositions with a staleness rate,
     a privacy audit, revision, a "confirm out of character updates" rule, and retrieval (BM25, recency, MMR).
  3. Decisions: expected utility of acting, asking or waiting, and the thresholds between them.
  4. Calibration, a rate limiter, quiet hours, a duplicate filter and a governor that combines them.
  5. A consent ledger with levels of permission and a hash-chained log.
  6. A simulated day of suggestions and a user who gets tired of them, to compare policies.
  7. A toy next action predictor with a confidence score and selective prediction, and metrics for logs.

Everything is deterministic. The agents and the user are programs, not language models or people. Every
number, cost, probability and threshold below is made up by me for teaching. Nothing here is a security
control. Run `python3 proactive_agents_toolkit.py` for the demo and
`python3 -m unittest test_proactive_agents_toolkit` for the tests.
"""
import hashlib
import hmac
import math
import random
import re
from collections import Counter
from dataclasses import dataclass, field

DAY = 1440
NO_REPLY = "NO_REPLY"


def rng_for(*keys):
    h = hashlib.sha256("|".join(str(k) for k in keys).encode()).hexdigest()
    return random.Random(int(h[:16], 16))


def mean(xs):
    xs = list(xs)
    return sum(xs) / len(xs) if xs else 0.0


def clip(x, lo=0.0, hi=1.0):
    return max(lo, min(hi, x))


def hhmm(t):
    m = int(t) % DAY
    return "%02d:%02d" % (m // 60, m % 60)


# ---------------------------------------------------------------------------------------------------
# 1. Events, triggers, a scheduler and a webhook gate
# ---------------------------------------------------------------------------------------------------
@dataclass(frozen=True)
class Event:
    t: int
    kind: str
    data: tuple = ()

    def get(self, key, default=None):
        for k, v in self.data:
            if k == key:
                return v
        return default


def ev(t, kind, **data):
    return Event(t, kind, tuple(sorted(data.items())))


def _field(spec, lo, hi):
    out = set()
    for part in spec.split(","):
        step = 1
        if "/" in part:
            part, s = part.split("/")
            step = int(s)
            if step < 1:
                raise ValueError("step must be at least 1")
            if part != "*" and "-" not in part:
                part = part + "-" + str(hi)
        if part == "*":
            a, b = lo, hi
        elif "-" in part:
            a, b = (int(x) for x in part.split("-"))
        else:
            a = b = int(part)
        if a < lo or b > hi or a > b:
            raise ValueError("field out of range: " + spec)
        out.update(range(a, b + 1, step))
    return out


def parse_cron(expr):
    """Three fields: minute hour weekday (0 is Monday). This is a small cousin of a real five-field cron."""
    f = expr.split()
    if len(f) != 3:
        raise ValueError("need three fields: minute hour weekday")
    return _field(f[0], 0, 59), _field(f[1], 0, 23), _field(f[2], 0, 6)


def cron_matches(expr, t):
    mins, hrs, dows = parse_cron(expr)
    m = t % DAY
    return (m % 60) in mins and (m // 60) in hrs and ((t // DAY) % 7) in dows


class Trigger:
    def fires(self, e):
        raise NotImplementedError


class Cron(Trigger):
    def __init__(self, expr):
        parse_cron(expr)
        self.expr = expr

    def fires(self, e):
        return e.kind == "tick" and cron_matches(self.expr, e.t)


class Every(Trigger):
    def __init__(self, minutes, start=0):
        if minutes < 1:
            raise ValueError("minutes must be at least 1")
        self.minutes, self.start = minutes, start

    def fires(self, e):
        return e.kind == "tick" and e.t >= self.start and (e.t - self.start) % self.minutes == 0


class At(Trigger):
    def __init__(self, t):
        self.t = t

    def fires(self, e):
        return e.kind == "tick" and e.t == self.t


class Webhook(Trigger):
    def __init__(self, source):
        self.source = source

    def fires(self, e):
        return e.kind == "webhook" and e.get("source") == self.source


class FileChange(Trigger):
    """Fires on a file event under a path prefix. With debounce, a second change within that many minutes of
    the last one that fired is ignored (the first change in a burst wins)."""

    def __init__(self, prefix, debounce=0):
        self.prefix, self.debounce, self.last = prefix, debounce, None

    def fires(self, e):
        if e.kind != "file" or not str(e.get("path", "")).startswith(self.prefix):
            return False
        if self.last is not None and e.t - self.last < self.debounce:
            return False
        self.last = e.t
        return True


def in_window(t, window):
    """window is None (always) or (start, end) in minutes of the day. Start is inclusive and end is exclusive.
    A window with start greater than end wraps past midnight. Start equal to end means the whole day (my choice)."""
    if window is None:
        return True
    s, e = window
    m = t % DAY
    if s == e:
        return True
    if s < e:
        return s <= m < e
    return m >= s or m < e


@dataclass
class Job:
    name: str
    trigger: Trigger
    handler: object = None
    active: object = None
    idle_only: bool = False
    min_gap: int = 0
    last: object = None
    pending: object = None


class Scheduler:
    """Each minute: find jobs whose trigger fires. Drop a firing that is outside the job's active hours or too
    soon after the job's last run (min_gap is my simplification of pacing). If the job is idle-only and the user
    is busy, keep the firing as pending and run it at the first idle minute inside the window. A handler that
    returns None or NO_REPLY runs silently: nothing is delivered."""

    def __init__(self, jobs):
        self.jobs, self.log = list(jobs), []

    def step(self, t, events=(), busy=False):
        evs = [ev(t, "tick")] + list(events)
        out = []
        for job in self.jobs:
            cand = None
            for e in evs:
                if job.trigger.fires(e):
                    cand = e
                    break
            if cand is None:
                cand = job.pending
            if cand is None:
                continue
            if not in_window(t, job.active):
                job.pending = None
                continue
            if job.last is not None and t - job.last < job.min_gap:
                job.pending = None
                continue
            if job.idle_only and busy:
                job.pending = cand
                continue
            job.pending = None
            reply = job.handler(cand, t) if job.handler else job.name
            job.last = t
            delivered = reply is not None and reply != NO_REPLY
            self.log.append((t, job.name, reply, delivered))
            if delivered:
                out.append((job.name, reply))
        return out

    def run(self, timeline, events_by_t=None, busy_fn=None):
        events_by_t = events_by_t or {}
        res = []
        for t in timeline:
            busy = bool(busy_fn(t)) if busy_fn else False
            for name, reply in self.step(t, events_by_t.get(t, ()), busy):
                res.append((t, name, reply))
        return res


class WebhookGate:
    """An inbound hook gate. Disabled by default. Needs the token. Identical wake text that is already the most
    recent pending item coalesces, even if the queue is full. A full queue refuses new items and evicts none.
    The docs I read give 200 for queued and coalesced and 503 for a full queue; the other codes are my choice."""

    def __init__(self, token, max_queue=8, enabled=False):
        self.token, self.max_queue, self.enabled, self.queue = token, max_queue, enabled, []

    def post(self, token, text):
        if not self.enabled:
            return 404, "disabled"
        if not hmac.compare_digest(str(token).encode(), self.token.encode()):
            return 401, "unauthorized"
        if self.queue and self.queue[-1] == text:
            return 200, "coalesced"
        if len(self.queue) >= self.max_queue:
            return 503, "full"
        self.queue.append(text)
        return 200, "queued"

    def drain(self):
        q, self.queue = self.queue, []
        return q


def route(e, trusted_sources=()):
    """Where an event goes. Short notifications from a source you control wake the main session. Anything that
    carries raw content from outside (mail, a notification body, a file) goes to an isolated reader with no tools."""
    if e.kind == "tick":
        return "main"
    if e.kind == "webhook" and e.get("source") in trusted_sources:
        return "main"
    return "isolated_reader"


# ---------------------------------------------------------------------------------------------------
# 2. A small user model: propositions with confidence and staleness, audit, revise, retrieve
# ---------------------------------------------------------------------------------------------------
STOP = {"the", "a", "an", "is", "are", "to", "of", "and", "in", "on", "for", "with", "user", "has", "this", "their"}


def tokens(text):
    return [w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in STOP and len(w) > 1]


def jaccard(a, b):
    sa, sb = set(tokens(a)), set(tokens(b))
    if not sa and not sb:
        return 1.0
    return len(sa & sb) / len(sa | sb)


# (keywords, proposition, negates, base confidence, staleness rate per day)
RULES = [
    (("deadline", "due", "submit"), "User has an impending deadline", None, 0.6, 0.5),
    (("overleaf", "draft", "paper", "reviews"), "User is writing a paper", None, 0.6, 0.3),
    (("wedding", "invite", "rsvp"), "User is invited to a wedding", None, 0.7, 0.2),
    (("suit", "tuxedo", "formalwear"), "User needs formal wear", None, 0.4, 0.4),
    (("recipe", "icecream", "youtube"), "User is distracted by recipes while writing", None, 0.5, 0.6),
    (("billing", "invoice", "statement"), "User ignores billing notifications", None, 0.3, 0.2),
    (("vacation", "ooo", "leave"), "User is on vacation", "User has an impending deadline", 0.6, 0.5),
]
SENSITIVE = {
    "credentials": ("password", "passcode", "login", "pin"),
    "finance": ("bank", "routing", "card"),
    "health": ("diagnosis", "prescription", "symptom"),
}
CONFIRM_AT = 0.8


@dataclass
class Proposition:
    text: str
    conf: float
    alpha: float
    last_seen: float
    obs: list = field(default_factory=list)


class Audit:
    """A contextual-integrity style gate in miniature: should this observation be recorded at all? Sensitive
    categories are blocked unless the user has allowed that category."""

    def __init__(self, allowed=()):
        self.allowed, self.blocked = set(allowed), []

    def category(self, obs):
        words = set(re.findall(r"[a-z0-9]+", obs["text"].lower()))
        for cat, keys in SENSITIVE.items():
            if words & set(keys):
                return cat
        return None

    def allow(self, obs):
        cat = self.category(obs)
        if cat is None or cat in self.allowed:
            return True
        self.blocked.append(obs["id"])
        return False


class UserModel:
    def __init__(self, audit=None):
        self.props, self.pending, self.audit = [], [], audit

    def find(self, text):
        for p in self.props:
            if p.text == text:
                return p
        return None

    def propose(self, obs):
        words = set(re.findall(r"[a-z0-9]+", obs["text"].lower()))
        out = []
        for keys, text, neg, base, alpha in RULES:
            hits = len(words & set(keys))
            if hits:
                out.append((text, neg, min(1.0, base + 0.15 * (hits - 1)), alpha))
        return out

    def observe(self, obs):
        """Returns a dict: blocked (bool), added, revised, lowered, pending (lists of proposition texts)."""
        res = {"blocked": False, "added": [], "revised": [], "lowered": [], "pending": []}
        if self.audit is not None and not self.audit.allow(obs):
            res["blocked"] = True
            return res
        for text, neg, conf, alpha in self.propose(obs):
            old = self.find(text)
            if old is not None:
                old.conf = 1 - (1 - old.conf) * (1 - conf)
                old.alpha = min(old.alpha, alpha)
                old.last_seen = obs["t"]
                old.obs.append(obs["id"])
                res["revised"].append(text)
                continue
            target = self.find(neg) if neg else None
            if target is not None and target.conf >= CONFIRM_AT:
                self.pending.append((text, conf, alpha, obs))
                res["pending"].append(text)
                continue
            if target is not None:
                target.conf = target.conf * 0.5
                if target.conf < 0.05:
                    target.conf = 0.0
                res["lowered"].append(target.text)
            self.props.append(Proposition(text, conf, alpha, obs["t"], [obs["id"]]))
            res["added"].append(text)
        return res

    def confirm(self, text, yes):
        """The user answers a pending update. If yes, apply it and lower what it contradicts."""
        for i, (t, conf, alpha, obs) in enumerate(self.pending):
            if t == text:
                del self.pending[i]
                if yes:
                    for keys, ptext, neg, base, a in RULES:
                        if ptext == text and neg and self.find(neg):
                            self.find(neg).conf *= 0.5
                    self.props.append(Proposition(t, conf, alpha, obs["t"], [obs["id"]]))
                return True
        return False

    def scored(self, query, now, kdecay=2.0):
        """(proposition, score) for each proposition with confidence above 0: BM25 of the text against the query,
        times the recency factor exp(-alpha * kdecay * age in days)."""
        docs = [p for p in self.props if p.conf > 0]
        q = tokens(query)
        if not docs or not q:
            return []
        toks = [tokens(p.text) for p in docs]
        avg = mean(len(t) for t in toks) or 1.0
        n = len(docs)
        out = []
        for p, tk in zip(docs, toks):
            s = 0.0
            for w in set(q):
                df = sum(1 for t in toks if w in t)
                tf = tk.count(w)
                if tf == 0:
                    continue
                idf = math.log(1 + (n - df + 0.5) / (df + 0.5))
                s += idf * tf * 2.5 / (tf + 1.5 * (1 - 0.75 + 0.75 * len(tk) / avg))
            age = max(0.0, (now - p.last_seen) / DAY)
            out.append((p, s * math.exp(-p.alpha * kdecay * age)))
        return out

    def retrieve(self, query, now, k=3, lam=0.5, kdecay=2.0):
        """The top k by maximal marginal relevance over the scored propositions, with token Jaccard as similarity.
        Scores are scaled so the best is 1 (my choice). Propositions at confidence 0 are never returned."""
        sc = self.scored(query, now, kdecay)
        if not sc:
            return []
        top = max(s for p, s in sc)
        if top <= 0:
            return []
        docs = [p for p, s in sc]
        rel = [s / top for p, s in sc]
        idx = mmr_select(rel, lambda i, j: jaccard(docs[i].text, docs[j].text), k, lam)
        return [docs[i] for i in idx]


def mmr_select(rel, sim, k, lam=0.5):
    """Maximal marginal relevance: repeatedly take the item with the best lam * relevance minus (1 - lam) times its
    largest similarity to the items already taken. Items with zero relevance are never taken. Ties go to the
    lower index."""
    chosen, left = [], [i for i in range(len(rel)) if rel[i] > 0]
    while left and len(chosen) < k:
        def score(i):
            div = max((sim(i, j) for j in chosen), default=0.0)
            return lam * rel[i] - (1 - lam) * div
        best = max(left, key=lambda i: (score(i), -i))
        chosen.append(best)
        left.remove(best)
    return chosen


# ---------------------------------------------------------------------------------------------------
# 3. Decisions: act, ask or wait
# ---------------------------------------------------------------------------------------------------
@dataclass(frozen=True)
class Utilities:
    """Horvitz's outcomes. A means act, D means ask (a dialog), notA means wait. G is the user's goal being true."""
    a_g: float
    a_ng: float
    na_g: float
    na_ng: float
    d_g: float
    d_ng: float


def from_costs(benefit, c_fp, c_fn, c_ask=None, ask_gain=0.6):
    """Act on a wanted goal gains benefit. Act on an unwanted one costs c_fp. Wait on a wanted one costs c_fn.
    Asking costs c_ask whatever the goal (default half of c_fp), and when the goal is wanted the user keeps only
    ask_gain of the benefit, because they still have to answer and wait. Both defaults are made up."""
    if c_ask is None:
        c_ask = 0.5 * c_fp
    return Utilities(benefit, -c_fp, -c_fn, 0.0, ask_gain * benefit - c_ask, -c_ask)


def eu_act(p, u):
    return p * u.a_g + (1 - p) * u.a_ng


def eu_wait(p, u):
    return p * u.na_g + (1 - p) * u.na_ng


def eu_ask(p, u):
    return p * u.d_g + (1 - p) * u.d_ng


def _cross(y0a, y1a, y0b, y1b):
    """Where two lines in p cross. Line a is y0a at p=0 and y1a at p=1."""
    den = (y1a - y0a) - (y1b - y0b)
    if den == 0:
        raise ValueError("lines are parallel")
    return (y0b - y0a) / den


def decide(p, u):
    """The option with the highest expected utility. Ties go to the less intrusive option: wait, then ask, then act."""
    opts = [("wait", eu_wait(p, u)), ("ask", eu_ask(p, u)), ("act", eu_act(p, u))]
    best = opts[0]
    for o in opts[1:]:
        if o[1] > best[1] + 1e-12:
            best = o
    return best[0]


def thresholds(u):
    """(p_wait_to_ask, p_ask_to_act). Below the first, wait. Between them, ask. Above the second, act. If asking
    is never the best option the two numbers are equal and are the wait-to-act threshold."""
    p_wa = _cross(u.na_ng, u.na_g, u.a_ng, u.a_g)
    p_wd = _cross(u.na_ng, u.na_g, u.d_ng, u.d_g)
    p_da = _cross(u.d_ng, u.d_g, u.a_ng, u.a_g)
    if p_wd < p_da:
        return p_wd, p_da
    return p_wa, p_wa


def gumbo_eu(p, benefit, c_fp, c_fn):
    """The two expected utilities used by the interruption rule in the paper: interrupt or not."""
    return p * benefit - (1 - p) * c_fp, -p * c_fn


def gumbo_threshold(benefit, c_fp, c_fn):
    """Interrupt exactly when p is above c_fp / (benefit + c_fn + c_fp)."""
    return c_fp / (benefit + c_fn + c_fp)


def attention_cost(base, focus, k=3.0):
    """Made up: an unwanted interruption costs more the deeper the focus. focus is in [0, 1]."""
    return base * (1 + k * focus)


# ---------------------------------------------------------------------------------------------------
# 4. Calibration, rate limits, quiet hours, duplicates
# ---------------------------------------------------------------------------------------------------
def brier(ps, ys):
    return mean((p - (1.0 if y else 0.0)) ** 2 for p, y in zip(ps, ys))


def reliability(ps, ys, bins=5):
    """Rows of (low, high, n, mean predicted, observed frequency) for equal-width bins. Empty bins are left out."""
    rows = []
    for b in range(bins):
        lo, hi = b / bins, (b + 1) / bins
        idx = [i for i, p in enumerate(ps) if (lo <= p < hi) or (b == bins - 1 and p == 1.0)]
        if idx:
            rows.append((lo, hi, len(idx), mean(ps[i] for i in idx), mean(1.0 if ys[i] else 0.0 for i in idx)))
    return rows


def ece(ps, ys, bins=5):
    n = len(ps)
    return sum(r[2] / n * abs(r[3] - r[4]) for r in reliability(ps, ys, bins)) if n else 0.0


def sharpen(p, gamma):
    """Push a probability away from one half (gamma above 1) or toward it (below 1). Models an overconfident scorer."""
    a, b = p ** gamma, (1 - p) ** gamma
    return a / (a + b) if a + b > 0 else 0.5


class Calibrator:
    """Piecewise-constant recalibration fitted on a log of (score, outcome)."""

    def __init__(self, bins=10):
        self.bins, self.rate = bins, []

    def fit(self, ps, ys):
        self.rate = []
        for b in range(self.bins):
            lo, hi = b / self.bins, (b + 1) / self.bins
            idx = [i for i, p in enumerate(ps) if (lo <= p < hi) or (b == self.bins - 1 and p == 1.0)]
            if idx:
                pos = sum(1 for i in idx if ys[i])
                self.rate.append((pos + 1) / (len(idx) + 2))
            else:
                self.rate.append((lo + hi) / 2)
        return self

    def apply(self, p):
        b = min(self.bins - 1, int(p * self.bins))
        return self.rate[b]


class TokenBucket:
    def __init__(self, per_min, capacity):
        self.per_min, self.capacity = per_min, capacity
        self.level, self.t = float(capacity), None

    def _refill(self, t):
        if self.t is not None and t > self.t:
            self.level = min(self.capacity, self.level + (t - self.t) * self.per_min)
        self.t = t if self.t is None else max(self.t, t)

    def allow(self, t):
        self._refill(t)
        if self.level >= 1.0:
            self.level -= 1.0
            return True
        return False


class Governor:
    """Checks, in this order: quiet hours, daily cap, duplicate text, rate limit. Only a suggestion that passes
    all four is recorded and spends a token. check returns (ok, reason) with reason one of ok, quiet, cap,
    duplicate, rate."""

    def __init__(self, quiet=None, per_min=None, capacity=1, daily_cap=None, dup_thr=0.8, dup_window=240):
        self.quiet, self.daily_cap, self.dup_thr, self.dup_window = quiet, daily_cap, dup_thr, dup_window
        self.bucket = TokenBucket(per_min, capacity) if per_min else None
        self.sent, self.counts = [], Counter()

    def check(self, t, text):
        if self.quiet is not None and in_window(t, self.quiet):
            self.counts["quiet"] += 1
            return False, "quiet"
        day = t // DAY
        if self.daily_cap is not None and sum(1 for s in self.sent if s[0] // DAY == day) >= self.daily_cap:
            self.counts["cap"] += 1
            return False, "cap"
        for st, stext in self.sent:
            if t - st < self.dup_window and jaccard(text, stext) >= self.dup_thr:
                self.counts["duplicate"] += 1
                return False, "duplicate"
        if self.bucket is not None and not self.bucket.allow(t):
            self.counts["rate"] += 1
            return False, "rate"
        self.sent.append((t, text))
        self.counts["ok"] += 1
        return True, "ok"


# ---------------------------------------------------------------------------------------------------
# 5. A consent ledger
# ---------------------------------------------------------------------------------------------------
LEVELS = ["observe", "suggest", "draft", "act_reversible", "act_irreversible"]


class Ledger:
    """Standing grants per scope, each with a level, a start and an optional expiry, and an append-only log whose
    entries are chained by hash. A standing grant never covers an irreversible action: that needs a yes each time."""

    def __init__(self):
        self.grants, self.log = [], []

    def _append(self, t, what, outcome):
        prev = self.log[-1][4] if self.log else "start"
        h = hashlib.sha256(f"{prev}|{t}|{what}|{outcome}".encode()).hexdigest()
        self.log.append((t, what, outcome, prev, h))

    def grant(self, scope, level, t, ttl=None):
        if level not in LEVELS:
            raise ValueError("unknown level")
        self.grants.append({"scope": scope, "level": level, "start": t, "end": None if ttl is None else t + ttl, "revoked": None})
        self._append(t, f"grant {scope} {level}", "recorded")

    def revoke(self, scope, t):
        n = 0
        for g in self.grants:
            if g["scope"] == scope and g["revoked"] is None:
                g["revoked"] = t
                n += 1
        self._append(t, f"revoke {scope}", f"{n} grants")
        return n

    def max_level(self, scope, t):
        best = -1
        for g in self.grants:
            live = g["start"] <= t and (g["end"] is None or t < g["end"]) and (g["revoked"] is None or t < g["revoked"])
            if g["scope"] == scope and live:
                best = max(best, LEVELS.index(g["level"]))
        return best

    def authorize(self, scope, level, t, confirm=False):
        """Returns (decision, reason) with decision allow, ask or deny, and logs it."""
        need = LEVELS.index(level)
        have = self.max_level(scope, t)
        if have < 0:
            res = ("deny", "no live grant")
        elif level == "act_irreversible":
            if have < need:
                res = ("deny", "grant is below act_irreversible")
            elif confirm:
                res = ("allow", "confirmed for this action")
            else:
                res = ("ask", "irreversible: needs a yes each time")
        elif have >= need:
            res = ("allow", "covered by a grant")
        else:
            res = ("deny", "grant is below " + level)
        self._append(t, f"authorize {scope} {level}", res[0])
        return res

    def verify(self):
        prev = "start"
        for t, what, outcome, p, h in self.log:
            if p != prev or h != hashlib.sha256(f"{p}|{t}|{what}|{outcome}".encode()).hexdigest():
                return False
            prev = h
        return True


# ---------------------------------------------------------------------------------------------------
# 6. A simulated day of suggestions, and a user who tires of them
# ---------------------------------------------------------------------------------------------------
@dataclass(frozen=True)
class Block:
    kind: str
    start: int
    end: int
    focus: float


@dataclass(frozen=True)
class Opp:
    t: int
    useful: bool
    signal: float
    benefit: float
    text: str
    focus: float


@dataclass(frozen=True)
class Costs:
    c_fp0: float = 0.15
    k_focus: float = 3.0
    miss: float = 0.5
    ignored_frac: float = 0.3
    accept: float = 0.9
    fatigue: float = 0.85
    window: int = 90


BLOCKS = [("deep", 0.9, (45, 100), 0.35), ("meeting", 0.7, (25, 60), 0.25), ("admin", 0.3, (15, 40), 0.25), ("break", 0.1, (10, 25), 0.15)]
TOPICS = ["flights", "slides", "budget", "reviewers", "lunch", "invoice", "refactor", "reading", "taxes", "gift", "rsvp", "backup"]


def make_day(seed, start=420, end=1380):
    r = rng_for("day", seed)
    out, t = [], start
    while t < end:
        x, acc = r.random(), 0.0
        pick = BLOCKS[-1]
        for b in BLOCKS:
            acc += b[3]
            if x < acc:
                pick = b
                break
        dur = min(r.randint(*pick[2]), end - t)
        out.append(Block(pick[0], t, t + dur, pick[1]))
        t += dur
    return out


def focus_at(blocks, t):
    for b in blocks:
        if b.start <= t < b.end:
            return b.focus
    return 0.0


def make_opps(seed, blocks, per_hour=8.5, base=0.3):
    r = rng_for("opps", seed)
    out = []
    for t in range(blocks[0].start, blocks[-1].end):
        if r.random() < per_hour / 60:
            useful = r.random() < base
            sig = clip(r.gauss(0.62 if useful else 0.38, 0.2))
            out.append(Opp(t, useful, sig, r.uniform(0.5, 1.5), "suggest %s v%d" % (r.choice(TOPICS), r.randint(1, 3)), focus_at(blocks, t)))
    return out


def prob(o, gamma=1.0, cal=None):
    p = sharpen(clip(o.signal, 1e-6, 1 - 1e-6), gamma)
    return cal.apply(p) if cal is not None else p


def c_fp(costs, focus):
    return attention_cost(costs.c_fp0, focus, costs.k_focus)


def never():
    return lambda o, n=0: False


def always():
    return lambda o, n=0: True


def fixed(thr, gamma=1.0, cal=None):
    return lambda o, n=0: prob(o, gamma, cal) >= thr


def eu_policy(costs, gamma=1.0, cal=None, benefit=1.0, aware=False):
    """Interrupt when the calibrated chance is above the threshold from the expected utilities. If aware, the benefit
    is shrunk by fatigue ** n, where n is the number of suggestions the user ignored in the last window (my addition)."""
    def d(o, n=0):
        b = benefit * (costs.fatigue ** n if aware else 1.0)
        return prob(o, gamma, cal) > gumbo_threshold(b, c_fp(costs, o.focus), costs.miss * benefit)
    return d


@dataclass
class Row:
    opp: Opp
    shown: bool
    accepted: bool
    util: float
    reason: str


class Result:
    def __init__(self, rows):
        self.rows = rows

    def net(self):
        return sum(r.util for r in self.rows)

    def shown(self):
        return sum(1 for r in self.rows if r.shown)

    def accepted(self):
        return sum(1 for r in self.rows if r.accepted)

    def useful(self):
        return sum(1 for r in self.rows if r.opp.useful)

    def precision(self):
        s = self.shown()
        return self.accepted() / s if s else 0.0

    def recall(self):
        u = self.useful()
        return self.accepted() / u if u else 0.0

    def per_hour(self):
        if not self.rows:
            return 0.0
        span = (self.rows[-1].opp.t - self.rows[0].opp.t) / 60
        return self.shown() / span if span > 0 else 0.0


def run_policy(opps, decide_fn, seed, costs=None, governor=None):
    costs = costs or Costs()
    r = rng_for("user", seed)
    ignored, rows = [], []
    for o in opps:
        n = sum(1 for x in ignored if o.t - x < costs.window)
        want, reason = decide_fn(o, n), "policy"
        if want and governor is not None:
            want, reason = governor.check(o.t, o.text)
        shown = accepted = False
        if want:
            shown = True
            draw = r.random()
            accepted = o.useful and draw < costs.accept * costs.fatigue ** n
            if not accepted:
                ignored.append(o.t)
        cost = c_fp(costs, o.focus)
        if shown and accepted:
            u = o.benefit
        elif shown and o.useful:
            u = -costs.ignored_frac * cost
        elif shown:
            u = -cost
        elif o.useful:
            u = -costs.miss * o.benefit
        else:
            u = 0.0
        rows.append(Row(o, shown, accepted, u, reason))
    return Result(rows)


def study(policy_factory, seeds, costs=None, governor_factory=None):
    """Run a policy over several simulated days. Returns a list of Result objects."""
    out = []
    for s in seeds:
        opps = make_opps(s, make_day(s))
        g = governor_factory() if governor_factory else None
        out.append(run_policy(opps, policy_factory(), s, costs, g))
    return out


def summarize(results):
    return {
        "net": mean(r.net() for r in results),
        "shown": mean(r.shown() for r in results),
        "accepted": mean(r.accepted() for r in results),
        "precision": mean(r.precision() for r in results),
        "recall": mean(r.recall() for r in results),
        "per_hour": mean(r.per_hour() for r in results),
    }


def fit_calibrator(seeds, gamma=1.0, bins=10):
    ps, ys = [], []
    for s in seeds:
        for o in make_opps(s, make_day(s)):
            ps.append(prob(o, gamma))
            ys.append(o.useful)
    return Calibrator(bins).fit(ps, ys)


# ---------------------------------------------------------------------------------------------------
# 7. A toy next action predictor, selective prediction, and metrics for logs
# ---------------------------------------------------------------------------------------------------
def habit_world(seed, n=3000, n_actions=40, strength=0.2):
    """A sequence of actions where each action has a favorite next action. How strongly it is followed differs
    by action. Returns the sequence and the favorite map. Made-up."""
    r = rng_for("habit", seed)
    acts = ["act%02d" % i for i in range(n_actions)]
    fav = {a: r.choice(acts) for a in acts}
    strong = {a: clip(strength * r.uniform(0.2, 2.2), 0.0, 0.95) for a in acts}
    seq = [r.choice(acts)]
    for _ in range(n - 1):
        prev = seq[-1]
        seq.append(fav[prev] if r.random() < strong[prev] else r.choice(acts))
    return seq, fav


class NextActionModel:
    def __init__(self, actions, smooth=0.5):
        self.actions, self.smooth = list(actions), smooth
        self.counts = {a: Counter() for a in actions}

    def fit(self, seq):
        for a, b in zip(seq, seq[1:]):
            self.counts[a][b] += 1
        return self

    def sample(self, prev, r):
        w = [self.counts[prev][a] + self.smooth for a in self.actions]
        return r.choices(self.actions, weights=w)[0]

    def predict(self, prev, k, r):
        return [self.sample(prev, r) for _ in range(k)]


def agreement(samples):
    """The share of samples that agree with the most common one. A stand-in for the spread of sampled predictions."""
    return max(Counter(samples).values()) / len(samples)


def nap_eval(seq, seed=0, split=0.7, k=20, top=0.1):
    """Train on the first part and test on the rest. pass1 is the mean share of samples equal to the truth, passk
    is the share of cases where any of k samples equals it, and sel1 is pass1 among the top share of cases by
    agreement. Exact match replaces the paper's similarity judge. Returns a dict."""
    cut = int(len(seq) * split)
    model = NextActionModel(sorted(set(seq))).fit(seq[:cut])
    r = rng_for("nap", seed)
    rows = []
    for prev, truth in zip(seq[cut - 1:-1], seq[cut:]):
        s = model.predict(prev, k, r)
        rows.append((agreement(s), s.count(truth) / k, truth in s))
    rows.sort(key=lambda x: -x[0])
    m = max(1, int(len(rows) * top))
    return {"n": len(rows), "pass1": mean(x[1] for x in rows), "passk": mean(1.0 if x[2] else 0.0 for x in rows),
            "sel1": mean(x[1] for x in rows[:m]), "all_conf": mean(x[0] for x in rows)}


def risk_coverage(rows, cuts=(1.0, 0.5, 0.25, 0.1)):
    """rows are (confidence, correct). Returns (coverage, accuracy among the most confident share) per cut."""
    rows = sorted(rows, key=lambda x: -x[0])
    out = []
    for c in cuts:
        m = max(1, int(round(len(rows) * c)))
        out.append((c, mean(x[1] for x in rows[:m])))
    return out


def precision_from_log(records):
    """records are (shown, accepted). Precision is the only quantity a log of shown suggestions can give."""
    shown = [a for s, a in records if s]
    return sum(1 for a in shown if a) / len(shown) if shown else 0.0


def estimate_recall(accepted, suppressed_total, audited_n, audited_useful):
    """Recall needs labels for what was NOT shown. Audit a random sample of suppressed suggestions, estimate how many
    suppressed ones were useful, and compare with the accepted ones."""
    if audited_n <= 0:
        raise ValueError("need audited suppressed suggestions")
    missed = suppressed_total * audited_useful / audited_n
    return accepted / (accepted + missed) if accepted + missed > 0 else 0.0


def f1(p, r):
    return 2 * p * r / (p + r) if p + r > 0 else 0.0


def lu_accept(task_shown, need, user_accepts):
    """The paper's acceptance rule: accepted if a task was shown and accepted, or none was shown and none was needed."""
    return (task_shown and user_accepts) or ((not task_shown) and (not need))


def wilson(k, n, z=1.96):
    if n == 0:
        return 0.0, 1.0
    p = k / n
    d = 1 + z * z / n
    c = p + z * z / (2 * n)
    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n))
    return (c - h) / d, (c + h) / d


# ---------------------------------------------------------------------------------------------------
# Demo
# ---------------------------------------------------------------------------------------------------
def _h(n, title):
    print("\n== %d. %s" % (n, title))


def demo():
    C = Costs()
    test_seeds = range(10, 40)
    cal = fit_calibrator(range(0, 10))

    _h(1, "Triggers over one Monday, with active hours, an idle-only job and NO_REPLY (made up)")
    new_items = {t: 1 for t in (545, 553, 700, 702, 1010)}
    seen = {"n": 0}

    def inbox_check(e, t):
        n = sum(v for k, v in new_items.items() if k <= t)
        d, seen["n"] = n - seen["n"], n
        return NO_REPLY if d == 0 else "%d new item(s)" % d

    jobs = [
        Job("daily 09:00 brief", Cron("0 9 0-4"), lambda e, t: "brief at " + hhmm(t)),
        Job("inbox check", Every(30, start=480), inbox_check, active=(480, 1080), idle_only=True),
        Job("on file change", FileChange("src/", debounce=5), lambda e, t: "changed " + e.get("path")),
        Job("on ci hook", Webhook("ci"), lambda e, t: "ci says " + e.get("text")),
    ]
    events = {540: [ev(540, "file", path="src/a.py")], 542: [ev(542, "file", path="src/b.py")],
              548: [ev(548, "file", path="src/a.py")], 560: [ev(560, "file", path="docs/x.md")],
              900: [ev(900, "webhook", source="ci", text="build red")],
              901: [ev(901, "webhook", source="other", text="ignored")]}
    sch = Scheduler(jobs)
    out = sch.run(range(0, DAY), events, busy_fn=lambda t: 600 <= t < 660)
    for t, name, reply in out:
        print("  %s  %-18s %s" % (hhmm(t), name, reply))
    runs, quiet = Counter(l[1] for l in sch.log), sum(1 for l in sch.log if not l[3])
    print("  runs per job:", dict(runs), "| silent runs (NO_REPLY):", quiet)
    print("  routing: ci hook from a trusted source ->", route(ev(900, "webhook", source="ci"), ("ci",)),
          "| an email event ->", route(ev(900, "email", body="raw text")))

    _h(2, "A webhook gate (made-up token)")
    g = WebhookGate("s3cret", max_queue=2)
    for label, tok, txt, en in [("disabled", "s3cret", "a", False), ("wrong token", "nope", "a", True), ("first", "s3cret", "a", True),
                                ("same text again", "s3cret", "a", True), ("second", "s3cret", "b", True),
                                ("third (queue is full)", "s3cret", "c", True), ("same as last, full queue", "s3cret", "b", True)]:
        g.enabled = en
        print("  %-26s -> %s" % (label, g.post(tok, txt)))
    print("  drained:", g.drain())

    _h(3, "A small user model: propose, audit, revise, ask, retrieve (made-up observations)")
    m = UserModel(Audit())
    obs = [dict(id="o1", t=0, source="screen", text="Overleaf draft of the paper, reviews open, deadline Friday"),
           dict(id="o2", t=60, source="mail", text="Wedding invite from Sam, please RSVP"),
           dict(id="o3", t=120, source="screen", text="Bank site, password field, login"),
           dict(id="o4", t=200, source="mail", text="Reminder: submit by the deadline"),
           dict(id="o5", t=300, source="mail", text="Spam: you won a vacation, leave now, ooo")]
    for o in obs:
        r = m.observe(o)
        print("  %s %-6s blocked=%-5s added=%s revised=%s pending=%s" % (o["id"], o["source"], r["blocked"], len(r["added"]), len(r["revised"]), r["pending"]))
    print("  the user answers no to:", m.pending[0][0], "->", m.confirm(m.pending[0][0], False))
    now = 300 + 2 * DAY
    print("  retrieve 'paper deadline wedding' two days later:")
    for p in m.retrieve("paper deadline wedding", now, k=3):
        print("    conf %.2f  staleness %.1f/day  %s" % (p.conf, p.alpha, p.text))

    _h(4, "Act, ask or wait: thresholds from the costs (made up)")
    for name, b, fp, fn, ca in [("break (focus 0.1)", 1.0, attention_cost(0.15, 0.1), 0.5, None), ("admin (focus 0.3)", 1.0, attention_cost(0.15, 0.3), 0.5, None),
                                ("deep work (focus 0.9)", 1.0, attention_cost(0.15, 0.9), 0.5, None), ("send an email (c_fp 2.0)", 1.0, 2.0, 0.5, 0.2)]:
        u = from_costs(b, fp, fn, ca)
        lo, hi = thresholds(u)
        print("  %-24s c_fp %.3f  wait below %.3f, ask up to %.3f, act above | at p=0.25: %s, p=0.45: %s, p=0.70: %s, p=0.90: %s"
              % (name, fp, lo, hi, decide(0.25, u), decide(0.45, u), decide(0.70, u), decide(0.90, u)))
    print("  same numbers as an interrupt rule: eu(interrupt)=%.3f, eu(not)=%.3f at p=0.3" % gumbo_eu(0.3, 1.0, 0.555, 0.5))

    _h(5, "Policies over %d simulated days (made up; costs, fatigue and the model's scores are all invented)" % len(test_seeds))
    gov = lambda: Governor(quiet=(22 * 60, 8 * 60), per_min=1 / 10, capacity=3, daily_cap=40)
    rows = [("never", study(never, test_seeds, C)), ("always", study(always, test_seeds, C)),
            ("fixed 0.5 on raw scores", study(lambda: fixed(0.5), test_seeds, C)),
            ("expected utility, raw scores", study(lambda: eu_policy(C), test_seeds, C)),
            ("expected utility, calibrated", study(lambda: eu_policy(C, cal=cal), test_seeds, C)),
            ("  + rate limit and quiet hours", study(lambda: eu_policy(C, cal=cal), test_seeds, C, gov)),
            ("  fatigue-aware", study(lambda: eu_policy(C, cal=cal, aware=True), test_seeds, C)),
            ("fixed 0.5 on calibrated", study(lambda: fixed(0.5, cal=cal), test_seeds, C))]
    print("  %-30s %8s %7s %9s %10s %7s %6s" % ("policy", "net/day", "shown", "accepted", "precision", "recall", "per h"))
    for name, res in rows:
        s = summarize(res)
        print("  %-30s %8.2f %7.1f %9.1f %10.2f %7.2f %6.1f" % (name, s["net"], s["shown"], s["accepted"], s["precision"], s["recall"], s["per_hour"]))

    _h(6, "A threshold sweep on calibrated scores, and how well calibrated the raw scores are")
    for th in (0.2, 0.3, 0.4, 0.5, 0.6, 0.7):
        s = summarize(study(lambda: fixed(th, cal=cal), test_seeds, C))
        print("  threshold %.1f  net/day %6.2f  shown %5.1f  precision %.2f  recall %.2f" % (th, s["net"], s["shown"], s["precision"], s["recall"]))
    ps, ys = [], []
    for sd in test_seeds:
        for o in make_opps(sd, make_day(sd)):
            ps.append(prob(o)); ys.append(o.useful)
    print("  raw scores:        Brier %.3f  ECE %.3f   base rate of useful %.2f" % (brier(ps, ys), ece(ps, ys), mean(1.0 if y else 0.0 for y in ys)))
    for lo, hi, n, mp, fr in reliability(ps, ys):
        print("    scores %.1f to %.1f: n %4d, mean score %.2f, useful %.2f" % (lo, hi, n, mp, fr))
    cps = [cal.apply(p) for p in ps]
    print("  after recalibration: Brier %.3f  ECE %.3f" % (brier(cps, ys), ece(cps, ys)))
    print("  (the recalibrator was fitted on days 0 to 9 and is scored here on days 10 to 39)")

    _h(7, "What the governor refused (one run of 30 days, calibrated expected-utility policy)")
    tot = Counter()
    for sd in test_seeds:
        gv = gov()
        run_policy(make_opps(sd, make_day(sd)), eu_policy(C, cal=cal), sd, C, gv)
        tot.update(gv.counts)
    print("  checked %d suggestions that the policy wanted to show: %s" % (sum(tot.values()), dict(sorted(tot.items()))))

    _h(8, "Logs: precision is observable, recall needs audited labels (made-up logs from section 5)")
    res = study(lambda: eu_policy(C, cal=cal), test_seeds, C)
    recs = [(r.shown, r.accepted) for x in res for r in x.rows]
    acc = sum(1 for s, a in recs if a)
    supp = [r for x in res for r in x.rows if not r.shown]
    ar = rng_for("audit", 0)
    aud = [r for r in supp if ar.random() < 0.1]
    est = estimate_recall(acc, len(supp), len(aud), sum(1 for r in aud if r.opp.useful))
    p = precision_from_log(recs)
    true_rec = acc / (acc + sum(1 for r in supp if r.opp.useful))
    print("  precision from the log %.3f | recall (accepted over accepted plus useful but suppressed): true %.3f, estimated from %d audited %.3f | F1 %.3f"
          % (p, true_rec, len(aud), est, f1(p, est)))
    lo, hi = wilson(acc, sum(1 for s, a in recs if s))
    print("  acceptance rate %.3f, 95%% Wilson interval %.3f to %.3f (n=%d shown)" % (p, lo, hi, sum(1 for s, a in recs if s)))
    t3 = {"GPT-4o": (0.9811, 0.4815, 0.6460), "Qwen2-7B-Proactive": (1.0000, 0.4978, 0.6647)}
    for k, (rc, pr, ff) in t3.items():
        print("  check of a published row (%s): F1 from recall %.4f and precision %.4f is %.4f; the paper's table says %.4f" % (k, rc, pr, f1(pr, rc), ff))

    _h(9, "Next action prediction with a confidence score (made-up habits, exact match instead of a judge)")
    tot = Counter(); n = 0
    for sd in range(5):
        seq, _ = habit_world(sd)
        r = nap_eval(seq, seed=sd)
        for k in ("pass1", "passk", "sel1", "all_conf"):
            tot[k] += r[k] / 5
        n = r["n"]
    print("  5 users, %d test steps each, 20 samples per step: pass@1 %.3f, pass@20 %.3f, pass@1 among the top 10%% by agreement %.3f, mean agreement %.2f"
          % (n, tot["pass1"], tot["passk"], tot["sel1"], tot["all_conf"]))

    _h(10, "A consent ledger (made-up scopes)")
    L = Ledger()
    L.grant("screen", "observe", 0)
    L.grant("calendar", "draft", 0, ttl=120)
    L.grant("shop", "act_reversible", 0)
    L.grant("bank", "act_irreversible", 0)
    for scope, lvl, t, cf in [("screen", "observe", 10, False), ("screen", "suggest", 10, False), ("calendar", "draft", 100, False),
                              ("calendar", "draft", 130, False), ("shop", "act_reversible", 50, False), ("shop", "act_irreversible", 50, False),
                              ("bank", "act_irreversible", 50, False), ("bank", "act_irreversible", 50, True), ("mail", "suggest", 50, False)]:
        print("  t=%3d %-9s %-17s confirm=%-5s -> %s (%s)" % (t, scope, lvl, cf, *L.authorize(scope, lvl, t, cf)))
    L.revoke("screen", 200)
    print("  after revoke at t=200:", L.authorize("screen", "observe", 210)[0])
    ok1 = L.verify()
    L.log[1] = (L.log[1][0], "grant screen act_irreversible", L.log[1][2], L.log[1][3], L.log[1][4])
    print("  log entries %d, chain valid %s; after editing one entry, chain valid %s" % (len(L.log), ok1, L.verify()))


if __name__ == "__main__":
    demo()

Run the demo

$ python proactive_agents_toolkit.py
== 1. Triggers over one Monday, with active hours, an idle-only job and NO_REPLY (made up)
  09:00  daily 09:00 brief  brief at 09:00
  09:00  on file change     changed src/a.py
  09:08  on file change     changed src/a.py
  09:30  inbox check        2 new item(s)
  12:00  inbox check        2 new item(s)
  15:00  on ci hook         ci says build red
  17:00  inbox check        1 new item(s)
  runs per job: {'inbox check': 18, 'daily 09:00 brief': 1, 'on file change': 2, 'on ci hook': 1} | silent runs (NO_REPLY): 15
  routing: ci hook from a trusted source -> main | an email event -> isolated_reader

== 2. A webhook gate (made-up token)
  disabled                   -> (404, 'disabled')
  wrong token                -> (401, 'unauthorized')
  first                      -> (200, 'queued')
  same text again            -> (200, 'coalesced')
  second                     -> (200, 'queued')
  third (queue is full)      -> (503, 'full')
  same as last, full queue   -> (200, 'coalesced')
  drained: ['a', 'b']

== 3. A small user model: propose, audit, revise, ask, retrieve (made-up observations)
  o1 screen blocked=False added=2 revised=0 pending=[]
  o2 mail   blocked=False added=1 revised=0 pending=[]
  o3 screen blocked=True  added=0 revised=0 pending=[]
  o4 mail   blocked=False added=0 revised=1 pending=[]
  o5 mail   blocked=False added=0 revised=0 pending=['User is on vacation']
  the user answers no to: User is on vacation -> True
  retrieve 'paper deadline wedding' two days later:
    conf 1.00  staleness 0.2/day  User is invited to a wedding
    conf 1.00  staleness 0.3/day  User is writing a paper
    conf 0.90  staleness 0.5/day  User has an impending deadline

== 4. Act, ask or wait: thresholds from the costs (made up)
  break (focus 0.1)        c_fp 0.195  wait below 0.089, ask up to 0.164, act above | at p=0.25: act, p=0.45: act, p=0.70: act, p=0.90: act
  admin (focus 0.3)        c_fp 0.285  wait below 0.130, ask up to 0.208, act above | at p=0.25: act, p=0.45: act, p=0.70: act, p=0.90: act
  deep work (focus 0.9)    c_fp 0.555  wait below 0.252, ask up to 0.291, act above | at p=0.25: wait, p=0.45: act, p=0.70: act, p=0.90: act
  send an email (c_fp 2.0) c_fp 2.000  wait below 0.182, ask up to 0.750, act above | at p=0.25: ask, p=0.45: ask, p=0.70: ask, p=0.90: act
  same numbers as an interrupt rule: eu(interrupt)=-0.089, eu(not)=-0.150 at p=0.3

== 5. Policies over 30 simulated days (made up; costs, fatigue and the model's scores are all invented)
  policy                          net/day   shown  accepted  precision  recall  per h
  never                            -20.96     0.0       0.0       0.00    0.00    0.0
  always                           -43.89   139.3       7.3       0.05    0.18    8.8
  fixed 0.5 on raw scores           -4.07    56.0      16.2       0.29    0.38    3.5
  expected utility, raw scores     -28.99   113.5       9.9       0.09    0.24    7.2
  expected utility, calibrated     -13.96    82.5      13.5       0.16    0.32    5.2
    + rate limit and quiet hours   -13.72    40.0       9.7       0.24    0.23    2.5
    fatigue-aware                   -4.07    56.5      15.7       0.28    0.37    3.6
  fixed 0.5 on calibrated           -0.86    33.9      15.7       0.46    0.38    2.1

== 6. A threshold sweep on calibrated scores, and how well calibrated the raw scores are
  threshold 0.2  net/day -13.33  shown  80.9  precision 0.17  recall 0.33
  threshold 0.3  net/day  -4.07  shown  56.0  precision 0.29  recall 0.38
  threshold 0.4  net/day  -0.86  shown  33.9  precision 0.46  recall 0.38
  threshold 0.5  net/day  -0.86  shown  33.9  precision 0.46  recall 0.38
  threshold 0.6  net/day  -6.35  shown  17.7  precision 0.61  recall 0.26
  threshold 0.7  net/day -12.16  shown   8.6  precision 0.72  recall 0.15
  raw scores:        Brier 0.183  ECE 0.145   base rate of useful 0.30
    scores 0.0 to 0.2: n  567, mean score 0.10, useful 0.04
    scores 0.2 to 0.4: n 1187, mean score 0.31, useful 0.13
    scores 0.4 to 0.6: n 1409, mean score 0.50, useful 0.31
    scores 0.6 to 0.8: n  760, mean score 0.68, useful 0.58
    scores 0.8 to 1.0: n  257, mean score 0.89, useful 0.82
  after recalibration: Brier 0.160  ECE 0.006
  (the recalibrator was fitted on days 0 to 9 and is scored here on days 10 to 39)

== 7. What the governor refused (one run of 30 days, calibrated expected-utility policy)
  checked 2476 suggestions that the policy wanted to show: {'cap': 383, 'duplicate': 536, 'ok': 1199, 'quiet': 316, 'rate': 42}

== 8. Logs: precision is observable, recall needs audited labels (made-up logs from section 5)
  precision from the log 0.164 | recall (accepted over accepted plus useful but suppressed): true 0.708, estimated from 158 audited 0.790 | F1 0.271
  acceptance rate 0.164, 95% Wilson interval 0.150 to 0.179 (n=2476 shown)
  check of a published row (GPT-4o): F1 from recall 0.9811 and precision 0.4815 is 0.6460; the paper's table says 0.6460
  check of a published row (Qwen2-7B-Proactive): F1 from recall 1.0000 and precision 0.4978 is 0.6647; the paper's table says 0.6647

== 9. Next action prediction with a confidence score (made-up habits, exact match instead of a judge)
  5 users, 900 test steps each, 20 samples per step: pass@1 0.076, pass@20 0.492, pass@1 among the top 10% by agreement 0.182, mean agreement 0.22

== 10. A consent ledger (made-up scopes)
  t= 10 screen    observe           confirm=False -> allow (covered by a grant)
  t= 10 screen    suggest           confirm=False -> deny (grant is below suggest)
  t=100 calendar  draft             confirm=False -> allow (covered by a grant)
  t=130 calendar  draft             confirm=False -> deny (no live grant)
  t= 50 shop      act_reversible    confirm=False -> allow (covered by a grant)
  t= 50 shop      act_irreversible  confirm=False -> deny (grant is below act_irreversible)
  t= 50 bank      act_irreversible  confirm=False -> ask (irreversible: needs a yes each time)
  t= 50 bank      act_irreversible  confirm=True  -> allow (confirmed for this action)
  t= 50 mail      suggest           confirm=False -> deny (no live grant)
  after revoke at t=200: deny
  log entries 15, chain valid True; after editing one entry, chain valid False

Exercises

1. A weekday brief. Add a Job that fires at 17:30 on weekdays with Cron("30 17 0-4"). Run Scheduler over a Monday (range(0, DAY)) and a Saturday (range(5 * DAY, 6 * DAY)); times count minutes from Monday 00:00. Which rows fire, and how does the three-field cron differ from the five-field form in the docs [4]?
2. A window across midnight. Test in_window with (22*60, 6*60) at 21:59, 22:00, 05:59 and 06:00. Which edges are inside, and which rule from [4] does that follow?
3. The debounce. Change the FileChange debounce from 5 to 0 and to 30 minutes and count the runs on the demo's file events. A leading-edge debounce fires on the first event. What would a trailing-edge one do differently, and which is better for a "tell me when I stop typing" job?
4. The full queue. Make a WebhookGate with a capacity of 2 and send five different texts and then a repeat of the last. Which answers do you get, and what does the docs' rule about not evicting accepted events protect [5]?
5. A poisoned observation. Feed the user model an observation that tries to lower a confident proposition (for example repeated "vacation ooo leave" text from a "mail" source). What does revise do, and what rule would you add so that an out-of-character change needs the user's yes first [1]?
6. Staleness. Change the staleness rate of one rule and compare the order that retrieve returns after 1, 3 and 10 days. Which reading of the paper's decay score (2.4) did you choose?
7. The cost where asking disappears. Sweep c_ask from 0 to 1 for the deep-work costs and find where thresholds returns two equal numbers. What does that say about when a question is worth asking [9]?
8. Overconfidence. Use sharpen with gamma 2 to make the scores overconfident. Compare Brier, ECE and the net value of the expected-utility policy before and after fit_calibrator. Which of the two fixes the problem, and why does a threshold on raw scores suffer less than the expected-utility rule?
9. Fatigue. Set Costs(fatigue=1.0) so that ignoring costs nothing later, and rerun section 5. Does the expected-utility rule now beat a fixed threshold? Then try 0.7. What does this say about the lab's main result?
10. Which refusal matters. Remove one check at a time from the Governor (quiet hours, the cap, the duplicate check, the token bucket) and report net value and the number shown each time. Which check does the work when the policy is the expected-utility rule, and which refusals move to other checks?
11. How big an audit? Run estimate_recall with 20, 158 and 500 audited cases, over ten different seeds, and report the spread of the estimate against the true recall. How many do you need for an error under 0.05 [12]?
12. Habit strength. Change the strength in habit_world from 0.2 to 0.05 and to 0.5. How do pass@1, pass@20 and the top-tenth accuracy change? Compare with the paper's 17.1%, 36.3% and 25.9% [2], and say what you cannot conclude.
13. The ledger. Add a rule that a grant cannot last more than 30 days, and write a test for it. Then change one log entry, run verify, and explain what a person who has only the final hash can prove [1].

The tests: test_proactive_agents_toolkit.py

The 165 tests check the helpers and the three-field cron parser (ranges, steps, lists and errors); the triggers (the cron match, the interval anchor, the one-shot, the webhook source filter and the debounce edges); the window rule (start inclusive, end exclusive, midnight wrap, a whole-day window); the scheduler (active hours, idle-only coalescing, the minimum gap and the silent reply); the webhook gate (disabled, wrong token, queued, coalesced, full and drained) and the routing rule; the text helpers and the audit gate; the user model (proposing, the negation rule, noisy-or revision, the confirm line, retrieval order, recency, diversity and staleness); the decision rule (the utilities, the crossing points, the order of the thresholds, the ask zone and the rule from [1]); calibration (Brier, reliability bins, ECE, sharpen, the calibrator's smoothing and its empty bins); the token bucket and the governor (the order of checks, the counts and the duplicate window); the ledger (levels, expiry, revocation, the irreversible case and the hash chain); the simulation (the day's blocks, the number of opportunities, the user's acceptance and fatigue, the net value of each policy and the study summary); the next action model (agreement, pass@k and the risk-coverage curve); the log metrics (precision, the recall estimate, F1, the acceptance rule and the Wilson interval); a set of pinned numbers; and a check that the demo runs and prints ten sections.

Mutation check. This was a light, sampled run, not an exhaustive one (the toolkit has about 555 places where such a change can be made, and I sampled 120 of them at random in each of two runs, with a fixed seed of 0). The first run, on the first version of the code (554 places), caught 87 of 120. For the 33 misses I looked at each one, rewrote two functions so that their parts could be tested on their own (the retrieval score and the diversity pick), and added about 22 tests that pin numbers for the real gaps (the thresholds of the decision rule, the retrieval score and its decay, the next action numbers and the governor's defaults). The second run, on the rewritten code (556 places), caught 110 of 120, and its 10 misses were recorded. I then re-ran the misses from both runs against the final tests, each at every place on its line where the change applies. That gave 44 changes. For four real gaps I added tests (the waiting option with a non-zero baseline, the spread of the simulated scores and the range of benefits, the count of one per duplicate refusal, and the default number of bins in the calibration functions), and 38 of the 44 were then caught. The 6 that survive are on 6 lines. Three are default values of an argument that every caller passes explicitly (in the "never", "always" and "fixed" policies), one is the fallback of 1.0 for an average length of zero, which cannot happen with a non-empty text, one is the cap of 1.0 on a confidence, which no rule reaches, and one is the strict comparison of a time in the token bucket, which refills by zero when the times are equal. I judge all six to be equivalent or harmless, which is my judgement and not a proof, and I did not pin them. That makes 165 tests in all. I did not run the whole sample again after the last tests were added, apart from the re-run of the misses.

$ python -m unittest -v test_proactive_agents_toolkit
test_allow_and_block (test_proactive_agents_toolkit.TestAudit.test_allow_and_block) ... ok
test_allowed_category (test_proactive_agents_toolkit.TestAudit.test_allowed_category) ... ok
test_categories (test_proactive_agents_toolkit.TestAudit.test_categories) ... ok
test_brier (test_proactive_agents_toolkit.TestCalibration.test_brier) ... ok
test_calibrator (test_proactive_agents_toolkit.TestCalibration.test_calibrator) ... ok
test_calibrator_empty_bin_midpoint (test_proactive_agents_toolkit.TestCalibration.test_calibrator_empty_bin_midpoint) ... ok
test_ece (test_proactive_agents_toolkit.TestCalibration.test_ece) ... ok
test_fit_calibrator_monotone_enough (test_proactive_agents_toolkit.TestCalibration.test_fit_calibrator_monotone_enough) ... ok
test_reliability (test_proactive_agents_toolkit.TestCalibration.test_reliability) ... ok
test_reliability_bin_edges (test_proactive_agents_toolkit.TestCalibration.test_reliability_bin_edges) ... ok
test_sharpen (test_proactive_agents_toolkit.TestCalibration.test_sharpen) ... ok
test_bad (test_proactive_agents_toolkit.TestCron.test_bad) ... ok
test_fields (test_proactive_agents_toolkit.TestCron.test_fields) ... ok
test_matches (test_proactive_agents_toolkit.TestCron.test_matches) ... ok
test_minute_field_alone (test_proactive_agents_toolkit.TestCron.test_minute_field_alone) ... ok
test_star_covers_range (test_proactive_agents_toolkit.TestCron.test_star_covers_range) ... ok
test_start_step (test_proactive_agents_toolkit.TestCron.test_start_step) ... ok
test_attention_cost (test_proactive_agents_toolkit.TestDecisions.test_attention_cost) ... ok
test_cross_parallel (test_proactive_agents_toolkit.TestDecisions.test_cross_parallel) ... ok
test_decide_ties_prefer_less_intrusive (test_proactive_agents_toolkit.TestDecisions.test_decide_ties_prefer_less_intrusive) ... ok
test_eu_formulas (test_proactive_agents_toolkit.TestDecisions.test_eu_formulas) ... ok
test_from_costs_defaults (test_proactive_agents_toolkit.TestDecisions.test_from_costs_defaults) ... ok
test_gumbo_eu_pinned (test_proactive_agents_toolkit.TestDecisions.test_gumbo_eu_pinned) ... ok
test_interrupt_rule_equals_wait_act_crossing (test_proactive_agents_toolkit.TestDecisions.test_interrupt_rule_equals_wait_act_crossing) ... ok
test_no_ask_zone_collapses_to_act_threshold (test_proactive_agents_toolkit.TestDecisions.test_no_ask_zone_collapses_to_act_threshold) ... ok
test_thresholds_match_brute_force (test_proactive_agents_toolkit.TestDecisions.test_thresholds_match_brute_force) ... ok
test_thresholds_pinned (test_proactive_agents_toolkit.TestDecisions.test_thresholds_pinned) ... ok
test_deterministic_and_ascii (test_proactive_agents_toolkit.TestDemo.test_deterministic_and_ascii) ... ok
test_key_lines (test_proactive_agents_toolkit.TestDemo.test_key_lines) ... ok
test_published_row_identities (test_proactive_agents_toolkit.TestDemo.test_published_row_identities) ... ok
test_sections (test_proactive_agents_toolkit.TestDemo.test_sections) ... ok
test_coalesce_only_with_last (test_proactive_agents_toolkit.TestGate.test_coalesce_only_with_last) ... ok
test_disabled_by_default (test_proactive_agents_toolkit.TestGate.test_disabled_by_default) ... ok
test_drain (test_proactive_agents_toolkit.TestGate.test_drain) ... ok
test_full_refuses_without_evicting_but_still_coalesces (test_proactive_agents_toolkit.TestGate.test_full_refuses_without_evicting_but_still_coalesces) ... ok
test_token (test_proactive_agents_toolkit.TestGate.test_token) ... ok
test_hhmm (test_proactive_agents_toolkit.TestHelpers.test_hhmm) ... ok
test_mean_clip (test_proactive_agents_toolkit.TestHelpers.test_mean_clip) ... ok
test_rng_for (test_proactive_agents_toolkit.TestHelpers.test_rng_for) ... ok
test_authorize_basic (test_proactive_agents_toolkit.TestLedger.test_authorize_basic) ... ok
test_authorize_is_logged (test_proactive_agents_toolkit.TestLedger.test_authorize_is_logged) ... ok
test_expiry_and_start (test_proactive_agents_toolkit.TestLedger.test_expiry_and_start) ... ok
test_irreversible (test_proactive_agents_toolkit.TestLedger.test_irreversible) ... ok
test_levels_order (test_proactive_agents_toolkit.TestLedger.test_levels_order) ... ok
test_log_chain (test_proactive_agents_toolkit.TestLedger.test_log_chain) ... ok
test_log_chain_detects_wrong_link (test_proactive_agents_toolkit.TestLedger.test_log_chain_detects_wrong_link) ... ok
test_max_level_takes_highest_live (test_proactive_agents_toolkit.TestLedger.test_max_level_takes_highest_live) ... ok
test_revoke (test_proactive_agents_toolkit.TestLedger.test_revoke) ... ok
test_unknown_level (test_proactive_agents_toolkit.TestLedger.test_unknown_level) ... ok
test_estimate_recall (test_proactive_agents_toolkit.TestLogMetrics.test_estimate_recall) ... ok
test_f1 (test_proactive_agents_toolkit.TestLogMetrics.test_f1) ... ok
test_lu_accept (test_proactive_agents_toolkit.TestLogMetrics.test_lu_accept) ... ok
test_precision_from_log (test_proactive_agents_toolkit.TestLogMetrics.test_precision_from_log) ... ok
test_wilson (test_proactive_agents_toolkit.TestLogMetrics.test_wilson) ... ok
test_agreement (test_proactive_agents_toolkit.TestNextAction.test_agreement) ... ok
test_model_counts_and_sample (test_proactive_agents_toolkit.TestNextAction.test_model_counts_and_sample) ... ok
test_nap_eval_invariants (test_proactive_agents_toolkit.TestNextAction.test_nap_eval_invariants) ... ok
test_nap_eval_k (test_proactive_agents_toolkit.TestNextAction.test_nap_eval_k) ... ok
test_nap_eval_pinned (test_proactive_agents_toolkit.TestNextAction.test_nap_eval_pinned) ... ok
test_nap_eval_top_share (test_proactive_agents_toolkit.TestNextAction.test_nap_eval_top_share) ... ok
test_risk_coverage (test_proactive_agents_toolkit.TestNextAction.test_risk_coverage) ... ok
test_smoothing_allows_unseen (test_proactive_agents_toolkit.TestNextAction.test_smoothing_allows_unseen) ... ok
test_world (test_proactive_agents_toolkit.TestNextAction.test_world) ... ok
test_world_strength (test_proactive_agents_toolkit.TestNextAction.test_world_strength) ... ok
test_calibrator_edges (test_proactive_agents_toolkit.TestPinned.test_calibrator_edges) ... ok
test_confirm_only_touches_own_negation (test_proactive_agents_toolkit.TestPinned.test_confirm_only_touches_own_negation) ... ok
test_defaults_of_binned_functions (test_proactive_agents_toolkit.TestPinned.test_defaults_of_binned_functions) ... ok
test_ece_default_bins_pinned (test_proactive_agents_toolkit.TestPinned.test_ece_default_bins_pinned) ... ok
test_estimate_recall_one_audited (test_proactive_agents_toolkit.TestPinned.test_estimate_recall_one_audited) ... ok
test_eu_policy_strictly_above (test_proactive_agents_toolkit.TestPinned.test_eu_policy_strictly_above) ... ok
test_eu_wait_with_nonzero_baseline (test_proactive_agents_toolkit.TestPinned.test_eu_wait_with_nonzero_baseline) ... ok
test_every_one_minute_allowed (test_proactive_agents_toolkit.TestPinned.test_every_one_minute_allowed) ... ok
test_f1_small_values (test_proactive_agents_toolkit.TestPinned.test_f1_small_values) ... ok
test_governor_counts_one_per_refusal (test_proactive_agents_toolkit.TestPinned.test_governor_counts_one_per_refusal) ... ok
test_governor_default_dup_window_is_240 (test_proactive_agents_toolkit.TestPinned.test_governor_default_dup_window_is_240) ... ok
test_governor_default_duplicate_threshold (test_proactive_agents_toolkit.TestPinned.test_governor_default_duplicate_threshold) ... ok
test_jaccard_one_side_empty (test_proactive_agents_toolkit.TestPinned.test_jaccard_one_side_empty) ... ok
test_mmr_diversity_weight (test_proactive_agents_toolkit.TestPinned.test_mmr_diversity_weight) ... ok
test_mmr_select (test_proactive_agents_toolkit.TestPinned.test_mmr_select) ... ok
test_nap_all_conf_pinned (test_proactive_agents_toolkit.TestPinned.test_nap_all_conf_pinned) ... ok
test_opp_text_variants (test_proactive_agents_toolkit.TestPinned.test_opp_text_variants) ... ok
test_per_hour_uses_span_not_sum (test_proactive_agents_toolkit.TestPinned.test_per_hour_uses_span_not_sum) ... ok
test_retrieve_default_k_is_three (test_proactive_agents_toolkit.TestPinned.test_retrieve_default_k_is_three) ... ok
test_risk_coverage_order_and_default_cuts (test_proactive_agents_toolkit.TestPinned.test_risk_coverage_order_and_default_cuts) ... ok
test_rule_table (test_proactive_agents_toolkit.TestPinned.test_rule_table) ... ok
test_scored_bm25_pinned (test_proactive_agents_toolkit.TestPinned.test_scored_bm25_pinned) ... ok
test_scored_decay_pinned (test_proactive_agents_toolkit.TestPinned.test_scored_decay_pinned) ... ok
test_scored_term_frequency_and_empty (test_proactive_agents_toolkit.TestPinned.test_scored_term_frequency_and_empty) ... ok
test_sharpen_underflow (test_proactive_agents_toolkit.TestPinned.test_sharpen_underflow) ... ok
test_signal_spread_and_benefit_range (test_proactive_agents_toolkit.TestPinned.test_signal_spread_and_benefit_range) ... ok
test_bucket (test_proactive_agents_toolkit.TestRateAndGovernor.test_bucket) ... ok
test_bucket_capacity_cap_and_time_backwards (test_proactive_agents_toolkit.TestRateAndGovernor.test_bucket_capacity_cap_and_time_backwards) ... ok
test_governor_daily_cap_per_day (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_daily_cap_per_day) ... ok
test_governor_duplicate_threshold (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_duplicate_threshold) ... ok
test_governor_duplicate_window (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_duplicate_window) ... ok
test_governor_no_limits (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_no_limits) ... ok
test_governor_quiet_first (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_quiet_first) ... ok
test_governor_rate_and_token_not_spent_on_refusals (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_rate_and_token_not_spent_on_refusals) ... ok
test_governor_refused_not_recorded (test_proactive_agents_toolkit.TestRateAndGovernor.test_governor_refused_not_recorded) ... ok
test_route (test_proactive_agents_toolkit.TestRoute.test_route) ... ok
test_active_hours_drop (test_proactive_agents_toolkit.TestScheduler.test_active_hours_drop) ... ok
test_events_by_t_and_two_jobs (test_proactive_agents_toolkit.TestScheduler.test_events_by_t_and_two_jobs) ... ok
test_handler_none_delivers_name (test_proactive_agents_toolkit.TestScheduler.test_handler_none_delivers_name) ... ok
test_idle_only_coalesces (test_proactive_agents_toolkit.TestScheduler.test_idle_only_coalesces) ... ok
test_idle_only_defers_to_first_idle_minute (test_proactive_agents_toolkit.TestScheduler.test_idle_only_defers_to_first_idle_minute) ... ok
test_last_updates_only_on_run (test_proactive_agents_toolkit.TestScheduler.test_last_updates_only_on_run) ... ok
test_min_gap (test_proactive_agents_toolkit.TestScheduler.test_min_gap) ... ok
test_min_gap_boundary_is_inclusive (test_proactive_agents_toolkit.TestScheduler.test_min_gap_boundary_is_inclusive) ... ok
test_no_reply_is_silent_but_logged (test_proactive_agents_toolkit.TestScheduler.test_no_reply_is_silent_but_logged) ... ok
test_pending_dropped_when_window_closes (test_proactive_agents_toolkit.TestScheduler.test_pending_dropped_when_window_closes) ... ok
test_c_fp (test_proactive_agents_toolkit.TestSimulation.test_c_fp) ... ok
test_costs_defaults (test_proactive_agents_toolkit.TestSimulation.test_costs_defaults) ... ok
test_day_blocks_tile (test_proactive_agents_toolkit.TestSimulation.test_day_blocks_tile) ... ok
test_day_custom_range (test_proactive_agents_toolkit.TestSimulation.test_day_custom_range) ... ok
test_eu_policy_aware (test_proactive_agents_toolkit.TestSimulation.test_eu_policy_aware) ... ok
test_eu_policy_benefit_and_miss (test_proactive_agents_toolkit.TestSimulation.test_eu_policy_benefit_and_miss) ... ok
test_eu_policy_threshold (test_proactive_agents_toolkit.TestSimulation.test_eu_policy_threshold) ... ok
test_fatigue_counts_only_ignored_shown (test_proactive_agents_toolkit.TestSimulation.test_fatigue_counts_only_ignored_shown) ... ok
test_fatigue_window_boundary (test_proactive_agents_toolkit.TestSimulation.test_fatigue_window_boundary) ... ok
test_focus_at (test_proactive_agents_toolkit.TestSimulation.test_focus_at) ... ok
test_governor_in_run (test_proactive_agents_toolkit.TestSimulation.test_governor_in_run) ... ok
test_ignored_useful_costs_fraction (test_proactive_agents_toolkit.TestSimulation.test_ignored_useful_costs_fraction) ... ok
test_opps (test_proactive_agents_toolkit.TestSimulation.test_opps) ... ok
test_opps_per_hour_and_base (test_proactive_agents_toolkit.TestSimulation.test_opps_per_hour_and_base) ... ok
test_opps_rate_and_signal_means (test_proactive_agents_toolkit.TestSimulation.test_opps_rate_and_signal_means) ... ok
test_policy_gets_recent_ignored_count (test_proactive_agents_toolkit.TestSimulation.test_policy_gets_recent_ignored_count) ... ok
test_policy_ordering_in_the_simulation (test_proactive_agents_toolkit.TestSimulation.test_policy_ordering_in_the_simulation) ... ok
test_prob (test_proactive_agents_toolkit.TestSimulation.test_prob) ... ok
test_result_empty (test_proactive_agents_toolkit.TestSimulation.test_result_empty) ... ok
test_result_metrics (test_proactive_agents_toolkit.TestSimulation.test_result_metrics) ... ok
test_run_policy_utilities (test_proactive_agents_toolkit.TestSimulation.test_run_policy_utilities) ... ok
test_simple_policies (test_proactive_agents_toolkit.TestSimulation.test_simple_policies) ... ok
test_study_and_summarize (test_proactive_agents_toolkit.TestSimulation.test_study_and_summarize) ... ok
test_user_draws_deterministic_and_seeded (test_proactive_agents_toolkit.TestSimulation.test_user_draws_deterministic_and_seeded) ... ok
test_jaccard (test_proactive_agents_toolkit.TestTextHelpers.test_jaccard) ... ok
test_tokens (test_proactive_agents_toolkit.TestTextHelpers.test_tokens) ... ok
test_at (test_proactive_agents_toolkit.TestTriggers.test_at) ... ok
test_base_trigger (test_proactive_agents_toolkit.TestTriggers.test_base_trigger) ... ok
test_cron_trigger_needs_tick (test_proactive_agents_toolkit.TestTriggers.test_cron_trigger_needs_tick) ... ok
test_event_get (test_proactive_agents_toolkit.TestTriggers.test_event_get) ... ok
test_every (test_proactive_agents_toolkit.TestTriggers.test_every) ... ok
test_file_debounce_measured_from_last_fire (test_proactive_agents_toolkit.TestTriggers.test_file_debounce_measured_from_last_fire) ... ok
test_file_no_debounce (test_proactive_agents_toolkit.TestTriggers.test_file_no_debounce) ... ok
test_file_prefix_and_debounce (test_proactive_agents_toolkit.TestTriggers.test_file_prefix_and_debounce) ... ok
test_webhook_source (test_proactive_agents_toolkit.TestTriggers.test_webhook_source) ... ok
test_below_confirm_lowers_target (test_proactive_agents_toolkit.TestUserModel.test_below_confirm_lowers_target) ... ok
test_blocked_by_audit (test_proactive_agents_toolkit.TestUserModel.test_blocked_by_audit) ... ok
test_confirm_at_boundary (test_proactive_agents_toolkit.TestUserModel.test_confirm_at_boundary) ... ok
test_confirm_yes_and_no (test_proactive_agents_toolkit.TestUserModel.test_confirm_yes_and_no) ... ok
test_observe_adds (test_proactive_agents_toolkit.TestUserModel.test_observe_adds) ... ok
test_propose_confidence (test_proactive_agents_toolkit.TestUserModel.test_propose_confidence) ... ok
test_propose_nothing (test_proactive_agents_toolkit.TestUserModel.test_propose_nothing) ... ok
test_retrieve_basic_and_empty (test_proactive_agents_toolkit.TestUserModel.test_retrieve_basic_and_empty) ... ok
test_retrieve_decay_formula (test_proactive_agents_toolkit.TestUserModel.test_retrieve_decay_formula) ... ok
test_retrieve_k_and_mmr_diversity (test_proactive_agents_toolkit.TestUserModel.test_retrieve_k_and_mmr_diversity) ... ok
test_retrieve_no_future_age (test_proactive_agents_toolkit.TestUserModel.test_retrieve_no_future_age) ... ok
test_retrieve_recency (test_proactive_agents_toolkit.TestUserModel.test_retrieve_recency) ... ok
test_retrieve_skips_zero_confidence (test_proactive_agents_toolkit.TestUserModel.test_retrieve_skips_zero_confidence) ... ok
test_revise_keeps_smaller_staleness (test_proactive_agents_toolkit.TestUserModel.test_revise_keeps_smaller_staleness) ... ok
test_revise_noisy_or (test_proactive_agents_toolkit.TestUserModel.test_revise_noisy_or) ... ok
test_target_goes_to_zero_below_five_percent (test_proactive_agents_toolkit.TestUserModel.test_target_goes_to_zero_below_five_percent) ... ok
test_equal_means_whole_day_and_later_days (test_proactive_agents_toolkit.TestWindow.test_equal_means_whole_day_and_later_days) ... ok
test_none_is_always (test_proactive_agents_toolkit.TestWindow.test_none_is_always) ... ok
test_plain (test_proactive_agents_toolkit.TestWindow.test_plain) ... ok
test_wrap (test_proactive_agents_toolkit.TestWindow.test_wrap) ... ok

----------------------------------------------------------------------
Ran 165 tests in 2.399s

OK

5. Discussion questions

Use these in a study group or a class. Try to answer with an example and a source. Give each about 5 to 10 minutes.

1. A wrong answer to a request and a wrong unrequested suggestion both cost something. List what each costs, and say why the lecture's default for a proactive agent is silence [9] [10].
2. Horvitz's rule has three outcomes and two thresholds. Pick a real feature (a calendar suggestion, a code review nudge, a "you might want to reply" prompt) and say what it would do at p = 0.1, 0.4 and 0.9, and what costs you assumed [9].
3. The interrupt threshold is c_fp / (B + c_fn + c_fp). What happens to it when the cost of a missed chance goes to zero, and what would that mean for an agent that is never blamed for staying quiet (2.2)?
4. Adamczyk and Bailey found that timing matters, and Iqbal and Horvitz found that recovery is slow. Which findings would you trust for a modern agent that posts in a chat window, and which would you want to measure again [10] [11]?
5. The GUM paper keeps propositions with a confidence and a staleness rate. Name a fact about a person that should decay in hours, one in months and one never, and say how an agent could tell them apart [1].
6. OpenClaw's USER.md is written by the user, and a general user model infers its statements. Which would you want on your own computer, what would each get wrong, and could you combine them [1] [8]?
7. The audit step in the GUM paper flagged 7 of 180 propositions. Is that number high or low, and what else would you need to know before you decided [1]?
8. A spam email can edit a user model, and OpenClaw's webhook page separates trusted wake text from untrusted content. Design the smallest rule that would stop a poisoned email from changing a proposition, and say how it could still fail [1] [5].
9. LongNAP gets 17.1% at one try and 36.3% in twenty. Describe a product that could use a predictor with those numbers, and one that could not [2].
10. In the lab a plain threshold beat the expected-utility rule. List three reasons that could be true of real users, and three that are only true of the toy, and say how you would find out which side you are on (2.8).
11. ProactiveBench uses a model as a stand-in for the user, and most models scored near 100% recall and under 50% precision. What would you need to believe about the reward model before you used its scores to choose a product [12]?
12. A team reports "our proactive agent has an acceptance rate of 80%". List four questions you would ask before you were impressed, and say which of them an audited sample of suppressed opportunities would answer (2.11).
13. Several sources here are written by people who built the system or are close to it: an instructor co-wrote both main papers, a donor is named on the OpenClaw README, and a vendor wrote the ambient agents post. How should a student weigh a builder's account of its own work, and what independent evidence is in this pack [1] [2] [3] [13]?

6. Quiz (10 questions)

Pick one answer for each question, then press Check answers. Check answers 0 of 10 answered.

1. OpenClaw's webhooks page describes what happens when a wake request arrives and the session's queue is already full. What does it say [5]?
   • The oldest queued event is dropped to make room for the new one, and the new wake is accepted, so the call never fails
   • It ignores the new wake and still returns 200, so the caller cannot tell that the event was lost along the way
   • It accepts the wake and runs it at once in a second session, so that no two events ever wait for each other
   • It returns HTTP 503 with an actionable error, and it does not evict an event that the queue had already accepted
2. In Horvitz's LookOut example, which three outcomes does the system choose between for each message [9]?
   • Do nothing, ask the user in a dialog, or act, chosen from the probability of the goal and the utilities
   • Delete the message, keep it, or forward it, chosen by a rule that depends only on who sent the message
   • Act now, act later, or never act, chosen from a fixed schedule that does not depend on the goals of the user at all
   • Schedule, summarize or flag, chosen by a classifier that does not use any estimate of cost or benefit
3. Use the threshold p* = c_fp / (B + c_fn + c_fp). A benefit of 1.0, a cost of 0.5 for an unwanted interruption and a cost of 0.5 for a missed chance give which threshold?
   • 0.20, from 0.5 / (1 + 0.5 + 0.5 + 0.5), which counts the cost of an unwanted interruption twice
   • 0.33, from 0.5 / (1 + 0.5), which leaves out the cost of a missed chance from the bottom of the fraction
   • 0.25, from 0.5 / (1 + 0.5 + 0.5), the cost of an unwanted interruption over the sum of all three
   • 0.50, because the probability only has to be higher than the cost of an unwanted interruption to pay off
4. What did Iqbal and Horvitz's field study of 27 users report about alerts and recovery [10]?
   • Users went back to the interrupted task within about a minute, because the window of the task stayed open on their screens
   • Users lost nearly 10 minutes to alert switches and 10 to 15 more before returning, and 27% waited over two hours
   • Email alerts caused no switches at all, and only instant messages made users leave the task that they were working on
   • Recovery took longer after an immediate response than after a delayed one, by a wide and significant margin
5. In the General User Model paper's email study, what separated the full model from its ablations [1]?
   • Accuracy, where the full model was far ahead of both ablations and the intervals did not overlap at all
   • The number of propositions produced, since the ablations produced none and so could not be judged by the participants
   • Calibration alone, because the three variants had the same win rate and the same accuracy in the ranked setting
   • Participants' rankings (win rates 0.618 against 0.463 and 0.420), while the accuracy of the variants was close
6. Why did the General User Model paper add a limit of one suggestion per minute to Gumbo's interruption rule [1]?
   • The rule judges each suggestion alone, as if actions were independent, so it ignores the user tiring of many in a row
   • The model could not write more than one suggestion per minute on the graphics cards that the authors used for the study
   • Participants asked for exactly one suggestion each minute in their interviews, so the authors followed their request
   • The audit step lets only one observation through per minute, so more suggestions would have nothing to work from
7. How does the next action prediction paper get a confidence for a prediction [2]?
   • It asks the model to write a number from 1 to 10 after each prediction, and it uses that number as it is without a check
   • It takes the probability of the first word of the answer and treats that probability as the chance of being right
   • It samples 20 predictions and measures how closely their embeddings agree, with low variance meaning high confidence
   • It asks the user to rate each prediction afterwards and averages the ratings over the month of the study
8. In the lab, the reliability diagram of the raw scores lies below the diagonal in every bin. What does that mean, and what follows for an expected-utility rule that uses them?
   • The scores are underconfident, so the rule stays too quiet and misses many of the useful chances
   • The scores are overconfident, so too many cases pass the threshold and the rule interrupts too often
   • The scores are calibrated, so the rule is optimal and no change to the scores could make it any better
   • The scores carry no information, so the rule does no better than a coin that is flipped at each opportunity
9. Table 3 of the ProactiveBench paper shows seven of eight models with recall near 100% and every model with precision under 50%. What does the pattern mean [12]?
   • The models offer help for nearly every event, so they catch real needs and also make many unwanted offers
   • The models rarely offer help, so they miss many needs but are right almost every time that they do speak up
   • The models are well calibrated, so they offer help exactly as often as the users in the test set want it
   • The reward model is too strict, so it rejects tasks that real users would have accepted in most of the cases
10. A team logs every suggestion that its agent showed and whether the user accepted it. Which quantity can it not compute from that log alone?
    • The acceptance rate, which is the number accepted divided by the number shown in the log, a ratio of two counts
    • An interval for the acceptance rate, which needs only the number accepted and the number shown
    • The suggestions per hour, which is a count of the rows of the log in each hour of the day, summed up by hour
    • Recall, the share of useful opportunities that were shown, because the suppressed ones are not in the log

Answer key

7. Graded assignment and rubric

Title: Build a small proactive agent for a toy user, tune it on some days, test it on others, and report it honestly.
Time: about 3 hours. Total: 100 points. Work: solo.

The task

Extend the lab code and write a short memo (two pages at most). Everything is a toy: no real model, no real person's data, no screen capture and no network. Pick your own domain, such as a study helper, a code review nudger or a travel planner, with its own opportunities, costs and fatigue. Every cost, probability and limit is yours and must be labelled as made up.

Part A. Triggers and gates (25 points)

• Add one new trigger or gate to the engine (for example a trailing-edge debounce, a "calendar is busy" gate, or a limit on runs per hour) and write tests for it, including a case that should be refused.
• Show one simulated day with your jobs: when each ran, which were silent, which were delayed and which were dropped. Say which rule in the docs of [4] or [5] each of your gates follows, or say that it is your own.
• Say what a wrong setting of your gate costs the user.

Part B. Decide, calibrate, govern (25 points)

• Make your own opportunities (with scores and a base rate) and your own costs. Compute the act, ask and wait thresholds for two user states (2.2, 2.13).
• Fit a calibrator on 10 days, and compare on 30 other days: a fixed threshold, the expected-utility rule on raw scores, on calibrated scores, with a governor, and a fatigue-aware version. Report net value, number shown, precision and recall, with the mean and the spread across days.
• Say how you chose your threshold, and whether you chose it on the days that you report. If you did, say that the number is optimistic (2.8).

Part C. User model or logs (25 points)

Do one of the two.

• User model. Add at least five rules and one sensitive category to the audit. Write a poisoned observation that changes a proposition, show that the model is fooled, add a defense (for example a confirm step for out-of-character changes), and test both the attack and the defense. Say how the defense could still fail [1].
• Logs and consent. Estimate recall from audited samples of 20, 100 and 500 suppressed cases over at least 20 seeds, and report the spread against the true value. Then add one new rule to the ledger (for example a maximum life of a grant) with a test, and explain what a hash chain does and does not prove [12].

Part D. The memo (25 points)

• Write a memo of two pages at most that answers: what does your toy measure, which sources in this pack does each design choice follow, what are three ways it could mislead a reader, and what would you need to see before you believed a real-world claim based on a toy like it. Name every author or vendor overlap in the sources you cite.
• State what in your results is built into the toy by construction.
• State three limits of your toy and three limits of your memo.

What to hand in

• Your proactive_agents_toolkit.py (or your own file names), your own code and tests. All tests must pass.
• A memo in PDF or Markdown with the tables from Parts A to C and the memo from Part D.
• A sources list for any reading you cite.

Rubric

Criterion	Full marks	Partial	Points
A. Triggers and gates	A new trigger or gate with tests including a refused case, a day's run table with silent, delayed and dropped runs, and the cost of a wrong setting.	No refused case, no table of a day, or no cost discussed.	25
B. Decide, calibrate, govern	Thresholds for two states, a calibrator fitted and tested on separate days, five policies compared with spread, and an honest account of how the threshold was chosen.	Tuning and testing on the same days without saying so, or means without spread.	25
C. User model or logs	One of the two done fully, with an attack and a defense that are both tested, or with recall estimates over many seeds and a ledger rule, and a reasoned closing answer.	No test of the attack, a single seed, or a claim that goes beyond the data.	25
D. The memo	All four questions answered, overlaps named, built-in effects stated, and three limits of each kind.	A memo that repeats the lab, or missing limits.	25
Total			100

Grade bands: 90 to 100 excellent. 75 to 89 good. 60 to 74 needs work. Under 60: redo and resubmit. Plain writing is rewarded: short sentences, no filler. Label your own arithmetic as your own.

8. Glossary

Term	Meaning
Abstention	Choosing not to answer or act when unsure, to raise accuracy at the cost of coverage (2.7).
Acceptance rate	Suggestions accepted divided by suggestions shown. A precision-like number that a log gives you (2.11).
Active hours	A time window in which a scheduled job may run. In OpenClaw the start is inclusive, the end exclusive, and overnight windows work [4] (2.9).
Ambient agent	An agent that listens to a stream of events and is not (solely) started by a person's message, and several can run at once [13] (2.1).
Audit (in a user model)	A gate that decides whether an observation should be used, built on contextual integrity [1] (2.4, 2.5).
Brier score	The mean squared difference between a probability and the 0-or-1 outcome. Lower is better (2.7).
Calibration	The match between a score and the observed frequency: of cases scored 0.7, about 70% should be right (2.7).
Coalesce	To merge repeated or missed triggers into one run or one queue entry [4] [5] (2.9).
Contextual integrity	The idea that information flows have norms that depend on context. The audit step of [1] builds on it (2.5).
Cron	A schedule written as fields for minute, hour and so on. OpenClaw takes 5-field or 6-field expressions with an optional time zone [4]; the lab uses three fields (2.9).
Debounce	Ignoring a repeat of an event within a set time. The lab's file trigger fires on the first event and drops repeats for 5 minutes (2.9).
ECE (expected calibration error)	The average gap between score and observed frequency across bins, weighted by bin size (2.7).
Expected utility	The average value of an option over the chances that the goal is wanted or not. The basis of the wait, ask or act rule [9] (2.2).
Fatigue	The user's falling willingness to accept suggestions after ignoring some. In the lab a made-up factor (2.8).
General user model (GUM)	A set of plain-language propositions about a user, with confidences, built from observations of computer use [1] (2.4).
Governor	A layer after the decision that applies quiet hours, a cap, a duplicate check and a rate limit (2.8).
Gumbo	The proactive assistant built on a general user model in [1]: it generates suggestions, decides by expected utility, limits its rate and can act (2.4).
Heartbeat	A periodic wake-up of an agent to check for work. In OpenClaw's current docs it is an ordinary automation job [6] (2.9).
Idle only	A job setting that waits for foreground work to finish before running [4] (2.9).
Interruption cost	The cost of an unwanted interruption, which depends on timing and on the user's focus and includes time to recover [10] [11] (2.3).
LongNAP	The next action predictor of [2], trained with a policy-gradient method and retrieval of its own past reasoning (2.6).
Mixed initiative	An interface in which the user and an automated service each take initiative at the right moments [9] (2.2).
Next action prediction (NAP)	Predicting what a user will do next from recent multimodal history [2] (2.6).
NO_REPLY	OpenClaw's silent result for a check with nothing to report [6] (2.9).
Notify, question, review	Three patterns of involving a person: tell, ask, or approve an action [13] (2.10).
p* (threshold)	The probability above which acting beats waiting. With the lab's costs, c_fp / (B + c_fn + c_fp) (2.2).
pass@k	The chance that at least one of k tries is good enough (2.6).
Precision and recall	Of the suggestions shown, the share that were wanted; and of the wanted opportunities, the share that were shown. The paper of [12] defines both in its own way (2.11).
ProactiveBench	The benchmark of [12]: 233 real-world test events and a reward model that stands in for the user (2.11).
Proposition	One statement in a user model, with a confidence, a staleness rate and its evidence [1] (2.4).
Quiet hours	Hours in which the agent may not show suggestions (2.8).
Reliability diagram	A plot of observed frequency against score, in bins (2.7).
Reward model	A model trained on human labels to judge whether a suggestion would be accepted [12] (2.11).
Selective prediction	Answering only the cases where confidence is high, and reporting the trade of coverage and accuracy (2.7).
Standing intent	An event-conditioned instruction stored by OpenClaw, with a cooldown, a fire limit and an expiry [8] (2.9).
Token bucket	A rate limiter that adds tokens at a fixed rate up to a capacity, and spends one per action (2.8).
Webhook	An HTTP endpoint that lets an outside service start a run. In OpenClaw it is off by default and needs a bearer token [5] (2.9).
Wilson interval	A confidence interval for a proportion that behaves well for small counts and for rates near 0 or 1 (2.13).

9. References

Entries marked (required) or (additional reading) are the course's readings. Entries marked (added) are my own additions. The course site and the slide folder are listed without a mark. Each entry says how much of the source I read. A source by the tool's own authors, or by a company that sells it, is a claim by an interested party, and I say so where it matters.

• [1] Shaikh, O., Sapkota, S., Rizvi, S., Horvitz, E., Park, J. S., Yang, D., and Bernstein, M. S. "Creating General User Models from Computer Use." UIST 2025. arXiv:2505.10831 (submitted 16 May 2025; version 3, 21 September 2025). Diyi Yang is an instructor of this course; Eric Horvitz also wrote [9]. https://arxiv.org/abs/2505.10831 (I read the full text; the authors' own studies, with 18 and 5 participants) (required)
• [2] Shaikh, O., Teutschbein, V., Gandhi, K., Chi, Y., Haber, N., Robinson, T., Ram, N., Reeves, B., Yang, S., Bernstein, M. S., and Yang, D. "Learning Next Action Predictors from Human-Computer Interaction." arXiv:2603.05923 (submitted 6 March 2026). A preprint. Diyi Yang is an instructor of this course. https://arxiv.org/abs/2603.05923 (I read the full text; the authors' own data, with a model as judge) (additional reading)
• [3] Steinberger, P., and the OpenClaw project. "OpenClaw" (repository README on GitHub). The schedule lists "OpenClaw: Open-source proactive AI agent" (2025). The repository was created on 24 November 2025 and has an MIT licence, according to GitHub's API; the README names a foundation as steward and OpenAI as a donor. https://github.com/openclaw/openclaw (I read the whole README, as it is on 29 October 2026 (the current revision, not a 2025 snapshot); I did not run the software; the project's own claims) (additional reading)
• [4] OpenClaw documentation. "Automation" and "Automation schedules" (the Markdown versions of the pages). https://docs.openclaw.ai/automation/cron-jobs/schedules (I read the overview and, on the schedules page, the schedule-type table, the monitoring-policy table and the stream and pacing sections, not every section; project documentation, current on 29 October 2026. The overview is at https://docs.openclaw.ai/automation) (added)
• [5] OpenClaw documentation. "Inbound webhooks" (a section of the automations pages). https://docs.openclaw.ai/automation/cron-jobs/webhooks (I read the authentication, wake, queue and untrusted-content passages by searching the page, not every section; project documentation) (added)
• [6] OpenClaw documentation. "Heartbeat migration." https://docs.openclaw.ai/gateway/heartbeat (I read the whole page; project documentation) (added)
• [7] OpenClaw documentation. "Gateway architecture." https://docs.openclaw.ai/concepts/architecture (I read the whole page; project documentation) (added)
• [8] OpenClaw documentation. "Standing intents" and "User model." https://docs.openclaw.ai/concepts/standing-intents (I read the standing intents page in full, and the directive and file-choice sections of the user model page at https://docs.openclaw.ai/concepts/user-model; project documentation. The pages cite TriggerBench, ProEvent, PrefEval and HorizonBench, which I did not read) (added)
• [9] Horvitz, E. "Principles of Mixed-Initiative User Interfaces." CHI 1999. doi:10.1145/302979.303030. Sole author. Horvitz is also a co-author of [1] and [10]. https://erichorvitz.com/chi99horvitz.pdf (I read the whole paper (the author's PDF)) (added)
• [10] Iqbal, S. T., and Horvitz, E. "Disruption and Recovery of Computing Tasks: Field Study, Analysis, and Directions." CHI 2007. doi:10.1145/1240624.1240730. https://doi.org/10.1145/1240624.1240730 (I read the whole paper; a field study of 27 users in 2007) (added)
• [11] Adamczyk, P. D., and Bailey, B. P. "If Not Now, When?: The Effects of Interruption at Different Moments Within Task Execution." CHI 2004. doi:10.1145/985692.985727. https://doi.org/10.1145/985692.985727 (I read the whole paper; a lab study with 16 subjects) (added)
• [12] Lu, Y., Yang, S., Qian, C., Chen, G., Luo, Q., Wu, Y., Wang, H., Cong, X., Zhang, Z., Lin, Y., Liu, W., Wang, Y., Liu, Z., Liu, F., and Sun, M. "Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance." arXiv:2410.12361 (submitted 16 October 2024; version 3, 3 December 2024). https://arxiv.org/abs/2410.12361 (I read the full text and Table 3; a reward model stands in for the user; the text and the table disagree in two places) (added)
• [13] Chase, H. "Introducing ambient agents." LangChain blog, 14 January 2025. Vendor-authored. https://www.langchain.com/blog/introducing-ambient-agents (I read the whole post; a vendor's own description of its direction, with no data) (added)
• [14] GUM project: repository "GeneralUserModels/gum" on GitHub and the project page. https://github.com/GeneralUserModels/gum (GitHub's API shows an MIT licence and a creation date of 10 April 2025; I did not read the code or the project page in detail. The project page is at https://generalusermodels.github.io/) (added)
• [15] Stanford CS329Z: Engineering AI Agents, course site and schedule (Diyi Yang, Michael Ryan, John Yang). https://cs329z.stanford.edu/ (the course's own page, read on 29 October 2026; it lists "Proactive Agents" on Wednesday, December 2, with one reading and two additional readings, and a finals week, December 7 to 11, titled "Open Problems & Final Demos" with a demo day whose time and place are "TBA")
• [16] CS329Z slide folder and the addresses lecture12.pdf and lecture18.pdf to lecture21.pdf, each tried once on 29 October 2026 at about 10:05 PM Pacific time. https://web.stanford.edu/class/cs329z/slides/lecture18.pdf (the addresses returned "not found" (HTTP 404), and the folder index listed lecture01.pdf to lecture11.pdf only, so I did not see the slides)
• [17] Grok Bot "Week 8b: Agent Safety" (this series, on the privacy, injection and human-oversight sections). week8b-agent-safety.html (re-used by link; its sources (PrivacyLens and others) were read for that pack and not re-read for this one) (added)