Describe a task in plain English. Get an AI workflow you can check.
- Builds the workflow from small, single-job AI specialists
- Refuses to run a plan that's wired wrong
- Stops for a person before anything leaves it
- Learns a team's unwritten rules from their corrections
Every number below comes from a test set written separately from the engine, and can be re-run.
One engine, several workflows
- A shared library of single-job specialists does the general work: sort, score, match, check, act.
- A workflow adds its own specialists only where the work is specific, like reading a fax or a health plan's billing rules.
Each column is a workflow with a recorded run. See them run.
How it works
A shared library
15 single-job specialists for general judgments: sort, match, check, extract, act. A workflow configures them, never copies them. Domain work is new code: the healthcare demo uses from the library and written for it (reading scans, plan rules, the claim).
Checked before it runs
Every wire must lead to something that exists and has already run. Any step that acts outside the engine must sit behind a human gate.
Model where judgment is needed, code where it isn't
Reading a scan or a message: the model. Plan rules, the claim form and the safety sentence: plain code, the same every time.
How it learns
No fine-tuning. Each workflow keeps its own corrections, and nothing crosses between workflows or customers.
1 · Instructions
What the step is for: its categories, criteria, any written policy. With no corrections, that's all it has.
2 · A rulebook it writes
From 8 corrections on, it writes down the rules they imply, and rewrites them as corrections come in.
3 · The closest examples
Each call also sees the 6 past corrections most like the input, so the rules come with cases.
On a clinic's messages it went from 59.3% → 97.3% as staff corrections accumulated, with no fine-tuning. Watch it learn
The workbench
One page to make, keep, run and correct workflows: python -m engine.app
- Describe a task; the composer drafts it; adjust the steps in a form
- Every save is a new version
- Runs take pasted text, a CSV, a folder, a webhook, or a folder on a schedule
What it can't do yet
- No real sending: no email, no claims portal, no eligibility check with a payer.
- No general image reader: scans are read only in the healthcare demo.
- One machine, no login.
- Learning beats a long prompt on size, not accuracy, at a few hundred corrections.
- Tests and engine both use Claude, so they may share blind spots. All test data is synthetic; no person has checked the labels.
Run it yourself
git clone https://github.com/mtichikawa/workflow-engine && cd workflow-engine python3 -m venv .venv && source .venv/bin/activate pip install -e '.[live]' python -m engine.app # the workbench: http://localhost:8602 python -m engine.live serve # the doula intake, live: http://localhost:8600 python -m engine.cli compose "sort support emails by type and draft a reply" python evals/blind.py validator # re-run a sealed evaluation
The honest frame. The parts aren't new: orchestration, few-shot examples and human review all exist. What's here is a working whole, built from scratch, with every number measured and re-runnable. Why it's built this way, with the costs.
Tested
Every number on this site comes from a test set written separately from the engine. How:
- A separate Claude agent, with no access to the code, wrote the test set.
- A second one labelled it again; only items both agreed on were kept.
- Sealed with a hash before the first score; scored once.
- A change to the engine gets a new set, never a re-score.
- Every result has its sample size, a baseline and a 95% interval.
Method and raw results: docs/blind-evals.md · evals/blind-results
It learns rules nobody wrote down
- A clinic routes patient messages by a 15-rule protocol the engine is never shown; it knows only the 6 queue names.
- Staff correct it as it works. Drag the slider to add corrections and watch wrong answers come right.
- Scored on 150 messages it never saw.
The clinic's real rules (never shown to it)
Rules it wrote from 291 corrections
The first six of each, as written.
- Its rulebook
- 97.3% on 150 messages it never saw
- Real rules, by hand
- 100%
- One long prompt
- with all 291 corrections pasted in: just as accurate
- So the rulebook buys
- a prompt about 40% the size, and rules a person can read, diff and lock
Try to break it
Three things an AI workflow must not get wrong, each tested on a set written by a separate agent that never saw the code. Every item is listed, the misses too.
A patient app's assistant must send anything medical to a person and never answer it.
On a message that mixes paperwork with a symptom, the patient gets an answer to the paperwork, a fixed safety message, and a nurse is alerted. See it in the doula intake, under “Reply to the patient”.
Asked for a job it can't do, does it say so, or fake a plan?
How it got there: each change, measured on a fresh set written after it
| Set | The change | Doable → planned | Impossible → faked |
|---|
Before anything runs, does it catch a workflow that's wired wrong?
Every kind of fault, before and now
| Kind of fault (in the outside list's words) | Before | Now |
|---|
What the validator says about five of them
More measurements
Smaller or less independent than the four above, so they aren't headlines.
Bugs its own tests found (5)
- A step reached by both a "pass" and a "fail" edge was always skipped, so the triage workflow's approval step never ran. Found by a seeded-fault test; fixed, with a regression test.
- The validator crashed on an edge to a step that doesn't exist. Fixed.
- The new validator caught the composer's own mistake: in the demo, its draft wired a list into a field that takes text. The composer now repairs it before running.
- An approval step that came after the action. Driving the workbench in a browser showed a gated file step writing its file first and asking second; a rejected item had already written. Outside actions now wait for approval before they run.
- Every item wrote the same file, and a saved workflow's corrections went into the shared example set. Staged files are never overwritten now, and each workflow learns in its own folder.
Use cases
The same engine and library in four workflows. Each has a recorded run you can open and click through.
A doula intake, from fax to claim
- In
- A new client's paperwork: a faxed referral, phone photos of the insurance card, an intake form, a visit log and a message from the patient
- Out
- An insurance claim and a reply to the patient, held for a person to approve. Nothing is sent.
- Read the documents and pull out what the claim needs
- Check it against the health plan's billing rules and flag anything missing
- Triage the patient's message: anything medical goes to a nurse, and the AI never answers it
- Draft the claim and the reply
- Stop for a person to approve
The exact request it was given
- Click a box to open it: what it did in this run
- Hover a box or a line to trace it
- Under each name: the specialist, model or code
- Rows: reads on the left, produces on the right
Press Play the run. It stops here, at the approval step, before anything leaves.
The drafts you'd be approving are below.
What a healthcare team would ask
Design facts, not compliance claims.
Medical content goes to a person
A written policy decides what's medical, mental-health warning signs included. The assistant never answers it; a nurse is alerted.
Replies stay inside the line
Replies cover coverage and scheduling only. The safety sentence is fixed text; a second model checks every draft.
A record of every decision
The packet a person approves holds the claim, the reply, the nurse handoff, the open items, and where every field came from.
Model calls go to Anthropic's Claude; documents and results stay on the machine running the engine. It has not been tested on real patients or real payer systems.