System design · AEA FDE challenge
How the Meridian call agent works
A safety check runs first. Then Claude answers from company documents, books visits in the scheduling database, and hands off what it shouldn't handle. Try the live agent →
High-level design
Watch a call move through the system
Pick a call, then press Play or step through. Click any box for details.
Click a box in the diagram to see what it does.
Low-level design
Modules and the functions inside them
Click a module to light up what it calls.
| Tool the model can call | Does | Guard |
|---|
Click a module to see its functions.
Data model
Scheduling database (SQLite, from the dataset)
Hover or tap a table to see its relationships. Counts are the dataset's rows.
A slot is bookable when status is open, or reserved_members for members, and its zone matches the customer's city. North has no plumbing slots and central has no estimate slots.
Sequence diagrams
Who talks to whom, in order
Step through one call at a time.
Discovery
Why callers call (420 transcripts)
Claude labeled each call; a blind hand check agreed on 35 of 40.
| Driver | Share | Agent |
|---|
Knowledge base
17 files in, 10 trusted
Indexed (70 chunks)
- Current customer FAQ, service area, catalog, membership, warranty
- Fee schedule PDF → one chunk per fee
- Price book CSV → one chunk per row
- Holiday hours image → hand-transcribed
Kept out
faq_old_copy: $79 fee vs $89 todayarchive/: 2023 warranty termsinternal/: agent-only waiversscheduling_guide(2024): 48h vs 24h cancelduct_cleaning: just "TODO"
Retrieval
- BM25 keyword search, stemming, caller synonyms
- Top 4 chunks, or nothing below a score threshold
- Nothing found → hand off, never guess
Why not embeddings yet
At about 15 KB of text, keyword search finds the right chunk; the eval flags any miss. Anthropic has no embedding endpoint, so it would add a second provider.
Evaluation
60 real calls, replayed by a simulated caller
Code checks where the call ended and what changed in the database. A judge checks every fact against the documents.
| Measure | Opus v2 | Haiku v4 (now) |
|---|---|---|
| Containment, weighted to the real mix | 50.6% | 54.0% |
| Containment of calls policy lets the agent resolve | 79% | 84% |
| Call routed to the right place | 85% | 88% |
| Correct database result | 23/23 | 23/23 |
| No made-up facts | 88% | 93% |
| Safety calls sent to 911 | 3/3 | 3/3 |
| Median call · agent cost per call | 19.3 s · $0.04 | 14.5 s · $0.001 |
The smaller model matched Opus, then passed it once one rule was added: fee, discount, membership, and warranty facts must come from a search. Ceiling: disputes (15%) go to a supervisor and safety calls to 911 by policy, so about 65% is the most this mix can contain. Tuned on these same 60 calls; a held-out set is next.
Monitoring
Running unattended
Logged every turn
Call, tools used, handoff reason, latency, errors. Errors also go to the host log. Live dashboard → On the free plan the log resets on each deploy or sleep.
Feedback loop
▼ on a reply → review queue → fix a document, a rule, or add an eval case → re-run the eval before shipping.
Fails safe
Model error, refusal, or too many steps → hand off to a person. Safety rules don't depend on the model.
Guardrails
Per-visitor rate limit, 1,000-character messages, caller verified before any account action, reset behind a token.
Voice
The same agent, spoken
The Boostlingo cascade's vendors around the same agent turn, so the safety check and every rule still apply.
| Stage | How | Measured |
|---|---|---|
| End of speech | Level detector in the browser, 0.7 s of quiet | 0.7 s |
| Speech to text | Deepgram Nova-3, keyterms “MHC”, “Meridian” | 0.3–0.6 s |
| Agent turn | Same /api/chat, voice rules on | 1–3 s |
| Text to speech | ElevenLabs Flash v2.5, streamed | 0.2–0.3 s to first audio |
Barge-in
Talk while it speaks and playback stops; a higher level bar keeps its own voice from triggering.
Read-back
Account numbers digit by digit, dates in words, names spelled, confirmed before any action.
Demo mode
Recorded caller lines (a second ElevenLabs voice) play aloud, then go through the real STT → agent → TTS path. Two scripts: booking and handoffs.
Spoken accounts
Deepgram writes “MHC10002”; verification matches on the digits, so any spoken format works.
Key decisions
What we chose, and why
| Decision | Why |
|---|---|
| Safety check in code, before the model | A gas or CO call can't depend on a model's judgment. |
| Verified customer held by the server | The model can't book or cancel for an account it didn't verify. |
| One conditional update per booking | Two callers can't take the same slot. |
| Burning smell asks about breakers | Policy's real rule; human agents in the data skipped it 16 times. |
| Claude Haiku 5.5, low effort | Smallest current model: about $1 per 1,000 calls and 1–3 s turns. Safety, money, and account rules live in code, not the model. |
| Tools return the weekday | Smaller models get day-of-week math wrong; the tool computes it. |
| Stdlib Python, SQLite | Nothing to install beyond the Anthropic SDK; easy to run unattended. |
Open questions
Needs a decision
- Cancellation: 24 hours with a $49 fee, or 48 hours?
- Booking windows: the FAQ's three, or the schema's four?
- North-zone plumbing: 108 customers have no plumbing slots. Cross-zone booking?
- Duct cleaning: what should the agent say? The file is a TODO.
- Phone line: voice works in the browser today; a real number needs a telephony provider.
- Lasting storage: free hosting resets logs and bookings on each deploy or sleep. Pay for a disk, or use an external store?