The first time an AI agent writes to a system your team depends on, such as a project in Asana or a deal in HubSpot, the question stops being whether it can do the job. It becomes what happens when it does the job wrong, or does it twice, and who answers for it.
I built the AI service delivery coordinator to answer that in code. It is a portfolio project of mine and not a client engagement. A client request becomes a project plan. A person edits and approves the plan. Only then does the project appear in Asana and the client and deal in HubSpot. Afterwards the system watches delivery and proposes fixes, and a person approves each one before anything is written. A three-minute walkthrough shows it running.
This article is the design, as rules for any human-in-the-loop AI agent that touches real tools. The repository is private, so the snippets below are simplified illustrations of the ideas, labelled as such, and not copies of the code.
The model interprets and drafts, and code decides
The coordinator has four agents: Orchestrator, Planning, Monitoring and Communication. The Orchestrator uses no model at all. It is plain code deciding which job follows which event and whether the project is allowed to take that step.
The Planning agent reads a client request and lists its assumptions and the information it is missing. It then proposes milestones, tasks, dependencies, roles and durations in working days. Its output is a typed object in a strict structured-output mode, validated before anything uses it. A draft with a dependency cycle or a reference to a task that does not exist is rejected with a fixed code, and the model gets one attempt to repair it.
What the model does not do is dates. The prompt tells it not to compute start or due dates, and plain code schedules every task, working backwards from the target date over working days. A model can do date arithmetic and still get it wrong now and then, and you cannot write a test that proves it never will. A function can be tested.
def add_working_days(day, count):
step = 1 if count >= 0 else -1
remaining = abs(count)
while remaining > 0:
day += timedelta(days=step)
# Monday to Friday only
if day.weekday() < 5:
remaining -= 1
return dayTwo caveats. The model does read a deadline stated in the request, such as “in three weeks”, and turns it into a calendar date from today’s date, so that date goes in front of the person for checking. And the scheduler counts Monday to Friday with no holidays, which is a documented limit, not an oversight. Impact analysis and forecasts, meaning which milestones a slipped task puts at risk, are plain code too.
Approval names the person and the exact version
A person edits the plan freely, and each edit is a new version. The system checks the edited plan and warns about problems such as dates out of order, cycles, or tasks that run past their milestone. It never overrides an edit.
Approval then records who approved which version. It is one of four separate records kept for every consequential step: what the AI reasoned, what it proposes, who approved it, and what the side effect did. Keeping them apart is what lets you answer “who approved this, and what did it cause?” without reconstructing it from logs.
Four rules sit behind the approve button.
- Only the latest version can be approved. An attempt on an older one is refused with a message telling the person to reload, because they were looking at something that has since changed.
- A repeat approval is safe. Approving a version that is already approved returns the stored approval, so a retry after a lost response does nothing new.
- Approval and edit happen one at a time. The project is locked while the approval runs, so the two cannot interleave.
- The job that does the writing is created in the same transaction. There is no approval without its job, and no job for an approval that rolled back. An edit after approval returns the project to planning.
def approve(project, version, user):
with transaction(), lock(project):
latest = latest_version(project)
if version != latest.id:
raise StaleVersion("reload")
existing = approval_for(latest)
if existing:
# a retry is safe
return existing
approval = record(latest, user)
# in the same transaction
enqueue_sync_job(approval)Each external write happens once
“Exactly once” is not something a network gives you. You build it from three things: a key that is the same on every attempt, a record written before each call, and a look at the vendor before any retry.
- A deterministic key. Every action gets a key made from the approval it belongs to, the operation and the subject, so the same input produces the same key on every run.
- Write ahead. Each attempt is committed to the database before the HTTP call, and its outcome is saved after.
- Look before retrying. A timeout, or a worker killed mid-call, leaves the outcome unknown. The system searches the vendor for the object it may have created, in Asana by a marker stored in the object, and in HubSpot by the same marker or, for a contact, by email. One match means that is the object. No match means the write never happened and it is tried again. Several matches stop the run for a person.
key = f"{approval}:{op}:{subject}"
# committed before the call
attempt = save_attempt(key)
try:
obj = create(payload, marker=key)
except Timeout:
# look, then decide
matches = find_by_marker(key)
# 1: use it, 0: retry, 2+: stop
obj = decide(matches)
save_outcome(attempt, obj)A write that fails three times, or is refused for good, stops the run as needs_human, and the screen says what to check. Nothing is reported as synced unless every action succeeded. A company or contact that already exists in HubSpot is linked and never changed.
There is a limit worth stating. A vendor object deleted by hand is not recreated. The sync stops and says what to check, because guessing that someone meant to delete it would be worse.
Webhooks arrive twice, late and out of order
When Asana tells the coordinator a task changed, the message is treated as a hint, not a fact. The webhook receiver verifies the signature over the raw bytes, stores each event as pending without duplicates, and queues its processing in the same transaction. A malformed event is stored as ignored with a reason, and its neighbours carry on.
Processing compares the event with the stored state of the task. A duplicate or out-of-order event therefore changes nothing it should not. The job system underneath delivers at least once, and the claim on each job makes a duplicate delivery a no-op.
Webhooks are the fast path. A periodic reconcile reads Asana’s own listing for every synced project, and that listing is the source of truth. A webhook that has died is replaced. If you take one idea from this section: treat the vendor’s push as a prompt to look, and keep a slower loop that does the looking regardless.
Spend is capped in code
An agent that loops can spend without anyone choosing to. A line in the prompt asking the model to be brief does not prevent that. In the coordinator every model call reserves tokens first, its prompt estimate plus its output cap, and is refused if the reservation would take its run, or all runs over the rolling 24 hours, past a cap. After the call, the reservation is settled with what was actually used. If a worker dies before settling, the reservation keeps counting in full.
def reserve(run, prompt, max_output):
need = prompt + max_output
with ledger_lock():
if used(run) + need > RUN_CAP:
raise BudgetExhausted("run")
if used_24h() + need > DAY_CAP:
raise BudgetExhausted("day")
record(run, need)There are two caps, one per run and one per day, and both are settings rather than constants in the code. A call that would cross either is refused before it is made. The live-model evals run under their own cap for the whole invocation.
The model gets no tools and no trust
Client requests, task comments and CRM data are untrusted, and that includes everything in a prompt. They reach the model only quoted inside the user message, with the request encoded as a JSON string so its own quotes cannot end the quotation. The system prompt carries the rules and nothing from the request. The agents have no tools, so nothing a model says can do anything except through code that sits behind an approval.
What this costs and what it does not cover
A human gate costs time: every plan waits for a person. That is deliberate, but it is a cost you should price in. The demo runs with stand-in vendors and recorded agent answers, and the real adapters are exercised against faked HTTP transports, not against a customer’s workspace. Known gaps include revised project titles and target dates not yet reaching Asana or HubSpot, and corrective options that occasionally offer a task that cannot help on a strict chain of tasks.
Seven questions to ask before an agent gets write access
Use these on any agent design.
- What may the model decide, and what must pass through code you can test? Dates, money and permissions belong in the second group.
- Who approves, and what exactly are they approving? If the thing can change after they click, they approved nothing in particular.
- What happens if the same approval arrives twice?
- What happens if a vendor call times out? Do you look before you retry?
- What happens if a webhook arrives twice, late, or never?
- What is the most a runaway loop can spend, and which line of code enforces it?
- Which text in your prompts can a stranger write?
If you want an agent wired into Asana, HubSpot or another tool your team already uses, my AI agent development service builds it with this approval boundary in place. You can send me a brief to start. The full design and the measured numbers are in the AI service delivery coordinator case study.