Some background first. Earlier this year, I started building an open-source project called EverLingo (记了么). The numbers as of today:

• Development period: 74 days
• Commits: 460+
• Code size: src/ is about 19k lines of Python, plus a React frontend and a Chrome extension
• Tests: roughly 1,150 test cases, 20k lines of test code—more than the source itself
• Releases: from v0.0.1-rc.1 to v0.1.3, 20+ tags

The team roster: me, and one agent.

The whole project was my first time building something to “real project” standards with a Coding Agent (OpenCode). Not a demo, not a weekend project—an open-source product deployed for family and friends.

This post is a retrospective on three things:

1. Working with a Coding Agent long-term for the first time: how my workflow took shape (docs, diagrams, Skills);
2. The project itself is an AI Agent Harness—how its multi-agent architecture evolved, and where it broke;
3. How the product grew from a little WeChat bot into a multi-user containerized product.

Before We Start: Meet EverLingo in Two Minutes

The rest of this post keeps referring to certain product and architecture concepts, so let me set the scene with one small scenario.

Imagine you’re reading an English web page and you aren’t sure what “structural” means in context. You open a chat with Nori (小记) in WeChat—the product’s persona is a hamster 🐹. You paste in the whole paragraph and ask. Nori explains it with the context taken into account—and because you told it earlier that you’re a programmer, it naturally reaches for engineering examples. Then it tells you: “I’ve saved this knowledge point for you.”

Behind the scenes, an invisible chain of events has just run: the queried passage, source URL, and your question get assembled into a memory entry and handed to a background Agent, which calls an LLM to merge the new knowledge with what’s already in your notebook into a markdown note; the notebook automatically rebuilds its full-text and vector indexes and periodically commits backups via git. Three weeks later you’ve forgotten the word; you come back and ask Nori “did I ever save anything about structural?”, and it retrieves the note—and even knows which article you met that word in.

That’s the product’s core loop: query → answer → memory settles in the background → the more you use it, the better it knows you. Traditional dictionary tools do “query → get answer → done”; the query leaves nothing behind. EverLingo’s stance: looking something up is not remembering it—a query deserves a second act.

One table for all the concepts that will appear below:

One minimal mental picture is enough: two LLM Agents—the Chat Agent talks on stage, the Memory Writer writes backstage—sharing one user’s markdown memory vault. What follows is how this picture evolved step by step.

Part One: My First Real Project Built with a Coding Agent

1.1 Starting Out: A Repo Containing Nothing but Docs

The project’s very first commit (e015fb1 "init") contained no code at all, only 153 lines of documentation: PRODUCT.md and a phase 0.1 design spec.

This wasn’t deliberate performance art—it was instinct born of necessity: faced with an Agent that cannot read minds, the only thing you can do is prepare the context it needs. The next few days of commit messages recorded the process quite honestly:

271a0df day1
6d8fcb6 day2 before first gen
e384f47 before intent analysis became an Agent

“before first gen”, “before intent analysis became an Agent”—write the spec first, then let AI generate. That habit held until the very end. Looking back after 74 days, the whole project is really a combination of three layers of things:

• Documentation written for the Agent (specs, ADRs, style guides)
• Rules imposed on the Agent (the mandatory loops in AGENTS.md)
• Code written by the Agent

1.2 How to Feed Docs: Layering + Enforced Loops

At first I figured that since an agent was writing the code, docs could be casual scribbles. That proved unworkable fast: every new session starts the agent with amnesia, and documentation is its memory. Worse, in the AI era documentation rots several times faster than in manual development—code gets generated so quickly that once docs fall behind, the next round of Agent generation proceeds on stale knowledge, and errors snowball.

After iterating through several versions over two months, the doc system stabilized into these layers:

AGENTS.md            # agent code of conduct: test commands, code style, entry point to collaboration rules
PRODUCT.md           # what the product is, who it serves, core capabilities
DOMAIN.md            # domain glossary: the single consensus my agent and I share for every term
ARCHITECTURE.md      # single entry point for architecture: overview diagram + component table + links to all design docs
docs/impl-spec/**    # implementation specs, organized by domain (agents/ channels/ vision/ ...)
docs/ADR/**          # architecture decision records
docs/archived/**     # archive of deprecated designs—never deleted, kept for traceability
TASKS.md             # work log: what changed, why, how it was verified

A few battle-tested lessons.

DOMAIN.md restraint. The discipline of “only document objects perceivable by end users” exists to prevent doc bloat—internal implementation details sink into impl-spec, and the domain doc stays thin forever. An Agent’s attention is a scarce resource, and every document competes with every other document for tokens.

AGENTS.md’s enforced loop. Docs alone aren’t enough; the Agent must also be obligated to maintain them. I wrote a few rules into AGENTS.md:

1
2
3
4
5
6
7

- Generally speaking, write tests alongside generated code
- No new dependencies without approval
- If any documentation seems badly ambiguous or contradictory, ask for confirmation before executing
- Present the design plan first; implement only after user confirmation
- If the implementation diverges from design docs, ask whether the docs need updating
- Major architectural changes require recording an ADR
- Any change to source code requires updating TASKS.md

The essence of these rules is upgrading “manual Code Review” into “manual Doc Review”—I don’t read code line by line, but I watch for drift between docs and implementation. Once drift appears, either the code is wrong or the docs need updating, and I need to know either way.

ADRs are the highest-return investment. For every discussion at the level of “should we delete this Agent?” or “should we switch protocols?”, I had opencode write down motivation, alternatives, risks, and mitigations as an ADR. At the time the payoff wasn’t visible; two weeks later, an agent in a fresh session would read it once and fully restore decision context, no re-explaining needed on my part. Here’s a real risk table from one ADR (taken from the one about removing the Memory Extract Agent—that story arrives in Part Two):

Filling out that table is the design review. Plenty of “surely this works” ideas got killed while filling it in.

Deprecate docs into an archive; don’t delete them. Git history tells you when something was deleted; archived docs tell you why it existed in the first place. Agents sometimes propose an approach that was rejected months ago—with docs/archived/ around, one sentence (“that approach was killed because X, see the archive”) saves a whole round of re-litigation. (Part Two contains a live example.)

TASKS.md as a work log. After every source change, have the agent append an entry in a mandatory format: date-time, what was done, how it was verified. It looks tedious, but git log is a stream organized by commits while TASKS.md is a narrative organized by tasks—when debugging, the latter is far more effective.

In late August I ran a full “doc checkup” and caught, among other things, a 49-line ARCHITECTURE.md—in its early days it was almost purely a list of links; tier-one subjects like the Indexer process and the multi-user architecture were completely absent. Only after rewriting did it deserve the title “single entry point for architecture”.

1.3 Diagrams as Code: From draw.io to d2 / mermaid

This was the tooling switch that taught me the most.

In early July I drew several architecture diagrams in draw.io and exported SVGs into the repo. The human experience was great, but about ten days in I hit an awkward fact: only I could see these diagrams—my agent couldn’t.

Three specific pain points:

• The agent can’t read draw.io source files. I couldn’t tell it “check this module’s docs against the architecture diagram”; to the agent, the diagram was a black box;
• When the architecture changed, updating the diagram fell to me. The agent would finish changing code and often forgot to remind me the diagram had gone stale;
• SVGs don’t go through git diff; during review you can’t tell which edge of a diagram actually changed.

So that diagram froze in mid-July and went unmaintained for over a month—not for lack of will, but because opening a GUI and hand-aligning shapes cost too much while the code changed daily.

The turning point came with the late-August documentation overhaul. I wrote this decision into an ADR:

All architecture diagrams must use d2 text embedded in markdown, compile-verified via the d2 CLI.

d2 is a DSL for describing diagrams in plain text, embedded directly in markdown. The logic behind it is simple: Coding Agents can write text; they can’t drive GUIs. After migrating, the benefits were tangible:

• Agents can read them and write them. “Update this d2 diagram against the latest gateway code” becomes a directly executable instruction;
• Text formats play nicely with git diff; adding one edge is a one-line change;
• The d2 CLI compiles and validates; syntax errors can be caught even in CI;
• Docs and code are reviewed in the same place, which greatly relieves staleness.

Results were immediate: on switchover day, I had the Agent add diagrams to 28 specs in one pass, and rewrote ARCHITECTURE.md with three embedded overview diagrams. Unthinkable in the draw.io era.

The experience eventually condensed into one rule: structure diagrams (what connects to what) in d2; flowcharts (sequences, state transitions) in mermaid—d2’s elk layout is more comfortable for topologies, mermaid is the most ergonomic for flows, and GitHub renders it natively. This isn’t aesthetic preference: when you have a collaborator reading your repo around the clock, “machine-readable” stops being a bonus and becomes mandatory.

A real example: here’s the core dataflow diagram currently in ARCHITECTURE.md (conversation and memory consolidation):

This diagram is itself evidence for this article’s thesis: it is maintained by the Agent, living in the repo alongside the code. Incidentally, every figure in this post was produced the same way. Eating our own dog food.

1.4 Skills: Baking SOPs into the Repo

Any workflow repeated more than twice becomes a skill—essentially an instruction document placed under .agents/skills/, auto-loaded by the agent when context matches.

The most concrete example is releasing. This project releases extremely frequently (20+ tags), and a single release touches a frightening number of places: bilingual README, deployment docs, __init__.py, frontend pages, pyproject.toml, plus version constants scattered across services… One hard-won lesson got baked verbatim into the skill doc:

The version field in a Chrome extension’s manifest.json supports only 1–4 dot-separated integers, not semver pre-release suffixes. Any version containing a hyphen is rejected at Web Store upload with Invalid value for 'version'.
Therefore 0.1.2-rc.4 must be rewritten as "version": "0.1.2.4" in the manifest, while package.json keeps the original semver.

Checklists like this will inevitably leak if kept in a human brain. As a skill, releasing is a single /releasing, and the agent runs the whole procedure and handles such special cases automatically.

Another skill is more domestic: round-corners-of-images, which batch-rounds the corners of images referenced in a given markdown file—built for writing blog posts.

Both skills are the same pattern: a skill’s value isn’t saving those few minutes; it’s turning “I remember we need to do this” into “the repo remembers we need to do this”. Human memory is unreliable. Repos are reliable.

Part Two: The Evolution Story of the Project’s Own Agent Harness

If Part One was “how to develop with Agents,” this part is “how to develop Agents.” Everlingo’s main tech stack is itself an AI Agent Harness: one Chat Agent with tools at the front, and a self-running memory system behind it.

This section walks through several major architecture changes in chronological order. Each one covers: why it changed, how it changed, and what pitfalls got hit.

First, today’s full architecture:

Two processes, two LLM Agents, one MCP-based storage-and-retrieval layer. Looks pretty clean—but it didn’t start out this way.

2.1 Early Days: chat.py Rules Everything, Then Breaks

Day one held no surprises. Single-file chat.py: call the LLM once, print the reply.

The first real requirement crushed it: users alternate between word lookups and small talk, so intents had to be separated. Rule-based branching quickly became unwritable, so it changed to letting the LLM judge the intent branch itself—which was also my first taste of the “LLM decides, code routes” pattern.

Then came the WeChat integration. The thread model of message passing and the async model of LLM calls tangled together in the same file, and chat.py turned into something nobody could parse. That produced the first proper refactor: Gateway / Session / Agent / Channel, four abstractions each finding their seat—Channel owns only platform messaging details, Session strings together one user’s conversation, Agent does nothing but think.

After that refactor I fixed a process I kept until the end: every structural change requires the agent to update the corresponding design doc and run the full unit-test suite before it counts as done. Docs first, tests as safety net—that’s what made every later big change safe to attempt.

2.2 Birth of the Three-Stage Pipeline, and the Exit of the Extract Agent

When the core capability—“automatic note-taking”—shipped (late June), we designed a textbook three-stage pipeline: the Chat Agent handles conversation; a separate Memory Extract Agent (daemon thread + queue + LLM structured output) decides “does this exchange contain knowledge worth saving” and extracts structured entries; then the Memory Writer Agent merges those entries into markdown notes.

Textbook multi-agent division of labor, right? Reality quickly taught me otherwise.

Pitfall one: headword dedup fails. The old design fed the last 20 exchanges wholesale to the LLM each round, deduplicating within a session via a session_seen_headwords string set. The problem: headwords are LLM-generated, and even temperature=0 cannot guarantee verbatim consistency—the same history got scanned repeatedly and two passes extracted different titles, making dedup effectively decorative. The fix was hard isolation on the input side: split messages into new_messages (this round’s additions, the sole extraction source) and context_messages (background only), eliminating duplicate scanning at the root.

Pitfall two: temperature=0.7 caused field drift. The Extract Agent initially reused the chat LLM config. Extraction demands structured, deterministic output; a temperature of 0.7 brings field drift. So it got its own dedicated LLM factory with temperature locked at 0.

The shared lesson from both pitfalls: anywhere you want the LLM to produce precisely consistent output, you will be disappointed. Deduplicate with clean code-side input splits; fill factual fields with code; don’t hope the LLM produces the same word twice.

Then, over the following month, the Extract Agent’s duties were moved away piece by piece:

• The “worth extracting?” judgment moved to the Chat Agent—it holds the fullest conversational context;
• Input isolation became cursor-based slicing; Extract only consumes pre-cut data;
• Structured output slimmed down until only three fields remained; all other system fields are filled in by code.

When the cutting was done I stared at it for a long while: all the work left to this Agent was one additional LLM call to re-format the same semantics the Chat Agent had just produced. Same-source data, duplicated judgment, extra cost, another failure point, another set of data structures and tests. And the benefit? Approaching zero.

Thus came one of the project’s earliest ADRs: “Remove the Memory Extract Agent.” In the new shape, the Chat Agent declares “this is worth remembering” mid-conversation via a request_memory_extraction(entries=[...]) tool; drafts accumulate in memory; when invoke() ends, code fills in the system fields before enqueueing to the Writer. Fields the LLM might fabricate are locked down with pydantic Literal enums:

1
2
3
4
5
6
7
8

class _MemoryEntryDraft(BaseModel):
    item_type: Literal["vocab","phrase","grammar","pragmatics","others"]
    why_want_to_save_memory: Literal[
        "User explicitly asked to remember this",
        "Correction item",
        "Chat Agent judgment",
    ]
    title: str

Note that the system fields (entry_id, timestamp, message slices) are not in the schema—not fabricated by the LLM, but completed by tool-binding code.

This refactor established a principle that has been validated again and again since: whenever you’re about to add a new Agent, ask first—can its job become a tool call on an existing Agent? An Agent isn’t an ordinary class; each one adds scheduling, state, and failure complexity. The default answer should be “no.” The same principle shows up everywhere in this project:

• events log appends don’t go through the LLM (“poor value for money, and it raises hallucination/format-error risk”—pure code assembly);
• entry_id and timestamps are always code-generated; the LLM never touches them;
• System fields get Literal enum backstops against fabrication;
• The Vision Service discussed later only perceives and never solves problems—same logic.

Incidentally, I didn’t delete the Extract Agent’s spec; it lies in docs/archived/. A month later an agent once proposed restoring a standalone extraction Agent; I just threw the archive link back at it.

2.3 NoticeSink: How a Background Thread Raises Its Hand

After the pipeline went async, a new UX problem surfaced—when the Writer finishes a note, nobody knows.

Concretely: the Memory Writer is a global singleton daemon thread consuming the queue asynchronously and writing to the vault, while each chat Session is bound to an asyncio event loop. The Writer diligently finishes a note and the chat window stays silent—the user has no idea Nori just saved something for them. The sentence “Saved xxx for you” must travel from a background thread back into the foreground conversation stream.

It sounds simple, but doing it reveals a classic cross-thread communication problem with a special constraint: notifications must not be blindly forwarded to the user—some operations are worth announcing and some are noise, and judging requires understanding context, which is, once again, an LLM’s job.

The solution introduced the NoticeSink Protocol and a SystemNotice event (accompanied by one big commit refactoring Sessions into a unified event-queue pattern). The Protocol comment is blunt:

1
2
3
4
5

class NoticeSink(Protocol):
    """Protocol for injecting session-bound notices from background agents.

    Memory Writer Agent etc. hold a reference to this protocol
    and call notify() after successful async operations."""

How it works:

Three design details worth mentioning:

1. The Writer holds no Session references whatsoever. It knows only the notice_sink.notify() interface; the Gateway registers itself as the implementer, notifications route by session_id, and cross-thread safety is guaranteed by call_soon_threadsafe. The Writer can therefore run standalone in any UI-less scenario.
2. Loss-tolerant semantics: notifications enter the Session’s unified event queue (the same pipe as user messages and channel events); if the session no longer exists, drop + log—no persistence or compensation. Background notifications aren’t on the critical path; better to lose one than carry complexity.
3. Asymmetric design: synchronous operations (delete/edit) return results directly via futures, not via notify; only async create needs it. Besides, delete/edit are initiated by the users themselves—nothing worth announcing there. Drawing a clear line between “when to shout and when to shut up” matters more than the mechanism itself.

The essence of this pattern is solving “how do background components participate in foreground conversation in a controlled manner”: interface isolation (Protocol) + unified pipe (event queue) + layered decisions (code gatekeeps; the LLM judges whether to speak). Afterward the Writer stopped being a mute file-writing worker; it can re-enter the conversation stream in the form of a standard event—whether and how to relay it to the user is left to semantic judgment.

2.4 Envelope: A Protocol Forced Into Existence by Multi-Client Input

Mid-July, the next feature was the Chrome extension: select a word on a web page, a side-panel chat pops up, and Nori answers using the selected text, its surrounding paragraph, and the page URL. Right behind it on the roadmap: a PDF plugin and an iOS word-selection service.

This requirement exposed a deeply buried assumption. The Channel abstraction at the time:

1
2

class Channel(ABC):
    async def recv(self) -> str | None: ...

Plain text. Four fatal problems:

1. Structured context (selected paragraph, URL, device info) had nowhere to go;
2. Source differences (web? extension? WeChat?) got flattened away;
3. No schema versioning mechanism, so protocol evolution was a non-starter;
4. Most insidious of all: ambiguity—if the convention were “JSON embedded directly in text”, then a user typing {"name": "mark"} by hand in a terminal would be parsed as structured input.

Enter the second early ADR: “Envelope.” User input from all channels is uniformly wrapped into a UserInputEnvelope, serialized as <envelope>{json}</envelope> and injected into the prompt. The schema looks roughly like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

{
    "schema_version": 1,
    "task": "translate",
    "chat": { "message": "" },
    "chat_context": {
        "resource_contexts": [
            {
                "kind": "selected_text",
                "text": "structural",
                "paragraph_text": "The embedding of the steel rods..."
            }
        ]
    },
    "source": {
        "kind": "chrome_ext",
        "url": "https://example.com/article",
        "title": "Some Article",
        "surface": "sidecar"
    }
}

Four design decisions I consider worth stealing:

Zero intrusion at the Agent layer. The Chat Agent’s interface signature didn’t change at all; the Session layer renders everything to text before passing it in. There was some hesitation at the time: wouldn’t letting the Agent receive structured objects directly be more type-safe? But then the Agent would couple to the protocol—every added field would ripple into the Agent signature and a pile of tests. In hindsight this decision paid off enormously: the envelope later survived two schema refactors (selection+context merged into tagged-union resource_contexts[]; splitting out a standalone chrome_ext source), touching a dozen-plus files across frontend and backend—without changing a single line at the Agent layer.

task is a preference, not a command. task=translate in the envelope merely expresses user preference; the LLM may freely decide whether to follow it—for example, if the user translates a word and then follows up with “why does ‘bank’ appear here?”, the Agent should translate first, then explain. The final arbiter of intent is always the LLM; the protocol layer doesn’t overstep.

Tagged JSON, not markdown. The LLM sees a JSON block with explicit boundaries and can reference fields like source.url precisely—none of the lossy reverse-parsing out of markdown.

Unknown kinds fail loudly; no silent fallback. source.kind is a discriminated union; unrecognized values raise ValidationError. Silent-fallback bugs surface three weeks later, wearing the weirdest possible disguise.

The Envelope later proved to be an extraordinarily high-return investment: image attachments arrived—drop SHA256 references into chat.attachments[] (references only; bytes never cross the protocol); the Vault editor scenario arrived—add one tagged union to resource_contexts[]. Define the protocol firmly once, and every new client interface turns from a design question into a fill-in-the-blank exercise.

2.5 Vision Service: The Border Between Perception and Action

August’s new requirement was images: a user sends a screenshot of a homework problem asking “how do I solve this?”, or photographs a vocabulary book page for Nori to explain. WeChat and the Chrome extension both needed support too.

The easiest approach would have been stuffing images into the Chat Agent’s multimodal messages. Instead we split out a standalone Vision Service, for reasons learned through earlier scars: perception and reasoning sharing one context makes prompts balloon uncontrollably, and image-analysis results can’t be cached or reused.

The Vision Service’s design has exactly two boundaries, but each one is hard:

Perception boundary: the spec’s first rule states—Vision answers only “what’s in the image”, outputting OCR text + business-semantic structure (together called ImageAnalysis), and never outputs answer/explanation. Problem-solving is done by the Chat Agent based on Vision’s returns. Example: the user sends a screenshot of a multiple-choice question; Vision outputs only the textual structure of stem and options; “the answer is B because XXX” belongs to the Agent. How strict is this ban? Later, when analyze_image gained an instruction parameter (users can phrase their understanding requests in words), the spec spelled out explicitly where it may and may not be used:

✅ "What's the circled word", "Describe in English the story shown in the picture", "Stem and options of question 1"
❌ Solving/explaining tasks (e.g., "pick the correct answer") must not be written here—the Agent does that based on returned results

Access boundary: the Agent holds no raw image bytes and never receives analysis results directly. It holds only a src_resource_sha256 reference; the sole path of access is the analyze_image tool, whose return value enters message history as a standard ToolMessage—no XML-injection-style side doors.

Why bother with all this? Cost engineering. Image analysis is expensive and slow, so the Vision Service built three layers of protection internally:

1. On successful upload, fire-and-forget triggers analysis immediately (Eager Warm), pre-warming the cache during the gap between “upload” and “press send”;
2. Analysis results land in an LRU+TTL cache keyed by image sha256, model name, and prompt version;
3. Concurrent requests merge via asyncio.Future—no matter how many callers, the vision model runs at most once per image.

Of course there were pitfalls too; two memorable ones:

Pitfall one: event-loop binding. Futures inside in_flight bind to the event loop that created them. If Eager Warm and analyze_image run on different loops, you get “awaiting future bound to a different loop”. Fix: run all image-analysis coroutines uniformly on one event loop. A classic asyncio trap.

Pitfall two: cache key missing a segment. The designed cache key was src + model + prompt_version; during implementation we discovered the purpose parameter also affects the prompt and hence the result—leaving it out of the key causes cache cross-contamination. Here I left a self-correction record I’m quite fond of—the spec gained an implementation note:

Implementation note (divergence from the letter of the ADR): the actual implementation appends an extra purpose segment. Reason: purpose affects prompt → affects analysis result; excluding it from the key would cause cache collisions.

When implementation and docs disagree, admit it and record the delta—it beats leaving the next person reading the doc (or the next session’s agent) confused for half a day.

The module’s biggest payoff showed up in extensibility: in August, Web shipped image chat first; days later WeChat cloned it (ImageStore/VisionService are process-level singletons co-located in the gateway process, so they are naturally shared, zero backend changes needed); then the Chrome extension added screenshot Q&A—still zero changes. The original “standalone Service” decision repaid itself, principal plus interest, on the day the third client plugged in.

2.6 Retrospective: Two Threads

Looking back at these evolutions, two patterns keep repeating:

Thread one: the LLM does semantic judgment only; all mechanical work sinks into code. The Extract Agent’s exit, pure-code events appends, Literal-guarded system fields, Vision barred from solving—all variants of it. It cuts the other way too: when should you reach for an LLM? Semantic questions with no fixed answer—“should the user be told?”, “how should it be phrased?"—hand those over confidently.

Thread two: inter-Agent communication keeps getting protocol-ized. From plain-text Channels, to the schema-versioned Envelope, to the thread-safe event queue + NoticeSink reverse channel, to trace ids threaded end-to-end (next chapter). The “conversation” between Agents gradually matured into formal communication with contracts, thread guarantees, and observability. The maturity of multi-agent collaboration isn’t measured by Agent count—it’s measured by the quality of communication between them.

Part Three: Tracing: Installing a Dashcam on a Multi-Agent System

While paying down engineering debt in late August, I finally did something I’d been putting off forever: adding tracing to this multi-agent system. In hindsight it was the highest-return debt repayment of all—it turned every architecture discussed above (async pipelines, cross-thread cooperation, system notifications) into something visible.

Why Tear It Down and Rebuild

The project previously used langfuse’s Python SDK for LLM tracing, with three problems:

• Traces scattered. Tracing initialization hung off every LLM instance, and each LLM construction produced an independent trace—“one conversation turn” simply didn’t exist at the observation layer; LLM thinking, replies, and tool calls lay scattered across isolated records;
• Locked to a single backend. Both the data model and export channel were langfuse-specific; swapping backends or adding a Collector layer: no chance;
• Even the dependency declaration lied: pyproject said langfuse>=2.0.0 while the code used the 4.x API.

So one ADR was written: OpenTelemetry as the sole tracing protocol layer. Conveniently, Langfuse v3+ servers natively accept standard OTLP/HTTP, so one OTel stack serves both Langfuse and any OTel backend—protocol neutrality ended up protecting the langfuse option.

Three Key Design Points

One trace per conversation turn. At the Session layer, each turn opens a root span chat.turn carrying metadata like session_id, channel, target_lang; LLM calls and tool calls in the LangChain runtime are auto-instrumented by instrumentation libraries following GenAI semantic conventions (model, token usage, tool_calls). OTel context lives in contextvars and propagates automatically within the same asyncio task—business code changes nothing.

Cross-thread propagation. This is the part echoing 2.2 / 2.3: the Memory Writer runs in a daemon thread, and by the time it consumes an entry the root span has usually ended. The approach: at the moment drafts are enqueued, serialize current context into a W3C traceparent stored on entry.trace_carrier; when the Writer consumes, extract, restore, and attach—the on-stage and backstage Agents reunite in the trace tree. Two error-prone points worth recording: attach must happen before asyncio.run() (the Runner snapshots context at startup; too late and it’s lost); parent-already-ended is a legal state—backends assemble trees by parent_id, and late children display normally.

Fail-open degradation. The principle is simple: tracing is icing on the cake; no failure may ever affect chat. Unconfigured means no-op spans (pure in-memory objects, negligible overhead); when the backend goes down, BatchSpanProcessor retries silently on a background thread and drops oldest when the queue fills; initialization exceptions degrade globally to no-op plus a warning log. Not a single synchronous network call sits on the request path.

A Memorable Pitfall: The Missing LLM Costs

Once traces flowed, a strange thing appeared: every LLM generation carried complete token usage, but billing info was empty across the board.

Investigation revealed Langfuse costs arrive along two paths: ingested cost (reported with the span, takes priority) and inferred cost (server-side inference from a model price table). The inferred path requires model names to match price definitions, and self-host instances’ price tables don’t recognize OpenRouter-style model names (like deepseek/deepseek-v4-flash-0731), so inference failed; meanwhile instrumentation forwarded only token counts, discarding the real dollar cost present in the response. More troublesome still: costs are computed only once at ingest; historical traces never backfill.

The fix came in two steps: request-side, enable OpenRouter usage accounting (extra_body={"usage": {"include": True}}—note this is an OpenRouter-private parameter; it returns 400 against official OpenAI endpoints, hence conditional injection); span-side, wrap the conversion function inside the instrumentation to write the response cost into a gen_ai.usage.cost attribute.

There’s a fine-grained lesson here: attributes must land on the LLM generation span created by the instrumentation itself. Initially I wrote attributes from a business-side callback, but by trigger time the current span was already the outer runnable span—and Langfuse only bills generation-type observations; attaching to the wrong place equals not attaching at all.

The Self-Host Evaluation Setup

For the server side I chose the Langfuse v4 + OTel Collector docker compose setup. My favorite bit of its division of labor: authentication converges on the Collector—EverLingo configures only one keyless intranet endpoint; Basic Auth lives entirely on the Collector; business config contains no secrets:

1
2
3
4

sys_setting:
  tracing_setting:
    tracing_service: otlp
    otlp_endpoint: "http://<host>:4318/v1/traces"

Restart the gateway, chat a “translate hello”, and seconds later you can watch that chat.turn trace unfold in the Langfuse UI: the LLM’s intermediate thinking and final reply, per-tool-call latency, token usage, plus the slightly-late-arriving memory.writer.process_entry span—the §2.3 “background note-writing” process, visible to the naked eye for the first time.

Analyzing Traces with opencode

Finally, my favorite part of this closed loop: using an AI Agent to analyze an AI Agent application’s observability data.

The method is humble. langfuse ships a CLI and an official Agent Skill; install them and the coding agent can query production traces directly:

1
2
3
4
5
6

npm i -g langfuse-cli
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_BASE_URL="http://<host>:3000"

langfuse api observations list --trace-id <id>

Then just tell opencode: “Check whether billing info exists on this trace; the app uses deepseek/deepseek-v4-flash-0731 via OpenRouter.” It will pull observations itself, check costDetails, and deliver a verdict. That cost pitfall above was diagnosed in exactly this fashion.

Agent writes the code, agent runs in production, agent reads its own traces to debug—at this point, the loop is finally turning.

Part Four: How the Product Grew

Beyond tech, a quick pass over the feature timeline, because it reflects prioritization. Month-by-month, coarse-grained:

Three observations:

Channel expansion validated the abstractions. After WeChat launched I was a real user every day—using foreign languages, looking things up, being remembered; nearly every product requirement came from my own genuine frustrations. And from the second channel onward, each new platform’s integration workload was visibly smaller than the previous one—the Channel abstraction and Envelope protocol compressed “platform-specific stuff” down to a minimal set; the Chrome extension went from design to usable in just over a week.

The multi-user conversion was a thorough act of self-cloud-native-ing. The scheme: “one host = one auth reverse proxy (WS-Router) + one container orchestrator (WS-Master) + N per-user containers”, with each container running the original single-user whole kit. That big lump (database, CLI, Internal API, docker lifecycle, reverse proxy) landed as a few consecutive PR slices in a very short time—the precondition was spending an entire day beforehand writing the design doc and phased plan. When the design is detailed enough, writing code really can be extremely fast.

i18n is not as simple as translating copy. Interface language and learning-target language are two independent dimensions, and even vault templates and event formats have language ownership attached. Complexity got underestimated; several supplementary specs later, things finally hung together.

One more feeling: engineering hygiene is debt, and it comes due sooner or later. The first two months barely touched observability and doc consistency; late August was concentrated repayment: tracing foundation, doc checkup, full text-ification of architecture diagrams. The good news: with ADRs and TASKS.md underneath, agents rebuilt context quickly during repayment; the bad news: this should have been done earlier.

Part Five: For Those About to Set Off

After 74 days, if only a few pieces of advice could survive:

1. Write specs before generating code. Documentation is the agent’s context; writing docs is part of programming. That the repo’s first commit contained only docs was no coincidence. I spent no less time on docs than on reading code, and the return is that every new session starts with the agent carrying full decision background. Conversely, wherever docs are vague, the agent tends to fill gaps in a “plausible but wrong” way.

2. Docs need enforced maintenance loops. Docs alone accomplish nothing; rules must bind behavior: change code → update TASKS.md, implementation diverges from design → ask, major change → write an ADR. You are architect and Reviewer, no longer typist.

3. Turn every asset into text. Diagrams in d2/mermaid, not GUI tools; procedures in SKILL.md, not in brains; decisions in ADRs, not chat history. Same essence: move the collaboration interface from “human relay” to “repo as source of truth”. Only text can diff, enter git, and be read and maintained by your Agent.

4. Subtract from your Agent architecture. When you find an Agent whose residual value no longer covers its cost, merge it away or sink it into code decisively. Very often what you wanted was simply one more tool call, or one rule line in a system prompt. Every extra Agent adds communication, failure, and observability costs.

5. LLMs judge semantics; code owns facts. Anything requiring the LLM to stay consistent across two outputs is unreliable: dedupe on the input side with code, fill factual fields with code, lock enums with Literal. Let LLMs do what they’re good at: parsing ambiguity, weighing trade-offs, organizing language.

6. Keep records of deviations. When implementation and design docs disagree, don’t secretly edit one of the two—write an “implementation note” in the spec. Your self of three months from now, and the agent reborn every day, will thank you for those lines.

Finally, a plug: EverLingo is open-source and usable now, with WeChat, Web/PWA, Chrome extension, and terminal integrations all available. You handle the learning; Nori (a hamster 🐹) handles the remembering.

Back to the opening line: this was my first complete project built with a coding agent. The biggest takeaway after 74 days isn’t empty phrases like “efficiency multiplied several times over,” but a shift in where the work concentrates—I went from being the person who writes code to being the person who designs systems and maintains context. The agent wrote the code, but every “why is it designed this way” had to be answered by me, and the more I answered, the clearer it became that these answers themselves need careful safekeeping.

The place that keeps them is this project’s documentation system.

References

• EverLingo GitHub repository (every ADR and spec cited here can be verified inside)
• ADR 20260719-remove_memory_extractor_agent — full decision record for removing the Extract Agent
• ADR 20260719-envelope — Envelope protocol design
• Vision Service image-understanding design doc
• Observability design doc / ADR 20260824-otel-tracing-base
• Langfuse / OpenTelemetry
• d2 language / mermaid
• OpenCode