Architecture

Flint is one Go binary that serves both the web UI and a JSON/streaming API, stores everything in SQLite, and talks to a local Ollama for the models. There's no build step, no CDN and no other service to run.

browser (htmx + Alpine.js + Pico.css, all vendored)
   │  same-origin HTTP: server-rendered pages + /api/*
   ▼
flint (Go, net/http)
   ├── SQLite (modernc.org/sqlite, pure Go, WAL, one connection)
   ├── attachment files on disk
   ├── Ollama HTTP API (chat, embeddings, model management;
   │   a `:cloud` model is forwarded by Ollama to ollama.com)
   └── Brave Search API, only on an explicit `@web` query

The bet#

A well-guided 3-4B local model isn't an inferior model, just an under-scaffolded one. Flint's work goes into the scaffolding around it: careful context management, a structured tool loop with human approval, and durable history. It avoids anything that only adds weight, which is why there's no bundled RAG or embedding stack.

An orchestration layer, not just a chat UI#

Calling Flint a "chat UI" is technically incomplete. A standard chat UI is a passive passthrough: it collects your message history and forwards an array of JSON objects directly to Ollama's /api/chat. When a chat grows long or a model attempts complex tool interactions, a raw endpoint degrades or overflows.

Flint is an active orchestration engine that sits between the human, the local filesystem, and Ollama, coordinating four essential subsystems:

• Context & budget orchestration: Instead of naively sending every message, Flint enforces a strict token budget per request. It preserves the user's latest prompt verbatim, injects structured folder anchors, and runs background layered summarization so long-running chats fit inside small model windows without overflow.
• Multi-agent decomposition orchestration (@agent): Small models struggle to hold a whole codebase and execute a multi-step plan simultaneously. Flint plans a task decomposition, provisions worker sub-agents with clean, isolated context windows, coordinates parallel tool execution, and synthesizes findings into a single coherent answer.
• Tool & safety orchestration: Flint brokers tool execution by checking preconditions (ensuring files and binaries exist), enforcing a hard safety shield against destructive commands (like rm -rf ~ or fork bombs), and managing an interactive approval state machine (Approve, Deny, Reply instead).
• Hardware & model lifecycle orchestration: To keep local VRAM free for the primary chat model, Flint dynamically requests embedding models (like nomic-embed-text) for @web ranking and immediately instructs Ollama to unload them once search results are scored.

Why not generic agent CLIs or frontier wrappers?#

A natural question is: "If Ollama provides a local endpoint, why not just point Claude Code, Aider, or another agent CLI at it?"

Existing agent CLIs are built and prompt-tuned for frontier models (Claude 3.7 Sonnet, GPT-4o) that boast 128k–200k token windows, deep instruction-following, and flawless JSON tool schema generation. When pointed at a 3B–4B local model, they break down in practice:

• Context bloat: Frontier CLIs dump thousands of lines into the context—full file listings, repository trees, git diffs, and verbose shell outputs. While a 200k-token model parses this easily, a 3B model experiences catastrophic attention degradation past 4k–8k tokens, losing track of instructions or looping repeatedly.
• Tool syntax and hallucination: Small models frequently hallucinate non-existent CLI flags, guess file names that do not exist, or emit invalid tool call JSON when confronted with sprawling tool schemas. Flint provides focused, single-tool guidance (such as instructing models to use grep -rni with single short words rather than guessing paths) and validates commands before human review.
• Monolithic vs. micro-agent reasoning: Frontier models can hold an entire project in context and execute an end-to-end plan in one turn. Small models cannot. Flint's @agent divides large questions into micro-subtasks, each executed in an isolated context window, so each small model instance only reasons about what it can comfortably fit.
• Human-in-the-loop safety in the browser: Terminal CLIs often encourage "auto-approve fatigue" where users hit Enter repeatedly without reading shell commands. Flint puts approvals in an explicit visual interface with clear diffs and output previews.

Resource footprint & philosophy#

Flint is designed for personal hardware—such as a laptop with 8–16 GB of RAM and a 6 GB GPU running 3–4B models. Every architectural choice reflects this constraint:

• About 24 MB container, Alpine included (about 11 MB to download), vs. multi-GB stacks: Popular local LLM web UIs (such as Open WebUI or LibreChat) bundle Python runtimes, PyTorch, LangChain, vector databases (ChromaDB), and Redis. Their Docker images frequently exceed 2–4 GB and consume 1–2 GB of system RAM just idling before a single model is loaded.
• Pure Go & SQLite: Flint compiles to a single static binary with no external dependencies (using pure-Go SQLite without cgo) and a 23.5 MB Alpine Docker image. It idles at ~15 MB of RAM, leaving every possible megabyte of CPU and GPU memory available for Ollama and model weights.
• Zero build step & zero CDN dependencies: The frontend uses vendored htmx, Alpine.js, and Pico.css with zero build tools (no Node.js, no webpack) and zero external network calls.

Repository layout#

backend/            Go module; run from here (asset paths are ../frontend/...)
  main.go           config from env, routes, graceful shutdown, /healthz
  auth.go           signup, login, logout, sessions, requireAuth
  recovery.go       security-question password recovery
  ratelimit.go      in-memory sliding-window limiter for auth endpoints
  handlers.go       conversations, messages, streaming turns, tool approval
  context.go        request budget and history fitting (see context-management.md)
  summary.go        background layered summaries
  memory.go         `@memory` save, draft, recall, folder memories, memory API
  agent.go          `@agent` plan call and validation, limits, run and transcript API
  agentrun.go       running agents: concurrency, inputs, commands, budgets, results
  ablate.go         FLINT_ABLATE: switch scaffolding off for the benchmark
  ollama.go         Ollama client: chat (streaming and not), models, embeddings
  tools.go          the run_shell tool definition and reasoning nudge
  fileops.go        read_file, write_file, edit_file: folder check, diff, apply on approve
  staging.go        a cloud model's staged edits and the end-of-turn review
  grounding.go      file names in replies checked against the folder; the anchor's file list
  shield.go         hard block list for catastrophic commands
  preconditions.go  cheap checks before a command is offered for approval
  folder.go         folder manifest and per-turn anchor header
  images.go         attachment upload, storage and image token estimate
  websearch.go      Brave Search, then re-ranking via Ollama embeddings
  views.go          server-rendered pages
  db.go             schema, migrations, all SQL, chat search
frontend/
  templates/        html/template pages (base, chat, settings, login, signup, recover)
  static/js/        chat.js (streaming chat UI, markdown/math setup), settings.js, app.js, copy.js,
                    vendored htmx, Alpine.js, marked, DOMPurify, Temml
  static/css/temml/ Temml's stylesheet and math font (local only)
  static/css/       flint.css on top of Pico.css
bench/              scored benchmark: run.py, tasks.py, fixture/
docs/               this documentation
Dockerfile, docker-compose.yml (Linux), docker-compose.desktop.yml (Mac/Windows)
.github/workflows/  docker.yml: publishes the image on version tags

Data model#

Deleting a conversation cascades to its messages, attachments, commands, summaries and agent runs (with their agents and transcripts); a memory saved from it stays and only loses its link. Deleting a user removes everything they own. Rows are never shared between users.

Schema changes that CREATE TABLE IF NOT EXISTS can't make on an existing database (new columns, the FTS index) are applied in migrate at startup. Each step is idempotent.

A chat turn#

1. POST /api/conversations/{id}/messages takes the conversation's lock (lockConversation), so overlapping requests to one conversation run in arrival order.
2. The message and any attachments are validated as a whole, then saved. An @web query runs its search first. An @memory recall adds the matching memories first, and @memory save is handled on its own with no chat turn, and so is @compact (see features.md).
3. streamAssistantTurn builds the budget and the fitted history, appends the tool nudge and folder anchor to the last message when a folder is attached (the nudge is left off when the user's message says not to run commands, see experiments.md E17), and streams the model's reply. Without a folder no tool is offered; if the message asks to run or check something, a note tells the model to give the command for the user to run instead of inventing output (E19).
4. A tool call is checked by the shield, then by the preconditions, then saved as a pending command. The stream ends, and the user approves or denies it with a separate request, which runs the command and continues the same turn. Reply instead denies it without continuing (deny?reply=1), and the user's reply is sent as an ordinary message.
5. After a final text reply come the stats line, then title generation on the first message (skipped if the reply failed), then a background summarization pass.

An @agent run#

1. Plan. @agent <task> makes one call to the chat's model with a JSON schema and the chat's own num_ctx (another size would reload the model, E24). It needs an attached folder. Go validates the plan (folder files only, no web queries from the model, inputs or exploration) and saves it as a planned run; the stream ends with <<<AGENT_PLAN>>>. Nothing enters the chat history. A web search is only added by the user on the card, and checked again at Run.
2. Run. POST /api/agent-runs/{id}/run takes the conversation's lock, re-checks the run is still planned, validates the edited plan again, and streams <<<AGENTS>>> lines until every agent ends. At most "max agents" run at once, in plan order; the rest wait.
3. One agent. Go reads its files (a big file keeps its start and end, sized at 1.5 chars per token so the prompt really fits) or runs its search, and builds a fresh history: instructions and the subtask with its inputs. An agent given no files or search, on a tool-capable model, may call the shell tool without a JSON format (a format suppresses tool calls, E25), with the tool nudge on the last message; one given inputs works from them only. Each call is checked by the shield and preconditions and waits for the user in the main chat; the decision arrives through POST /api/agent-runs/{id}/commands/{cmdId}/{approve|deny}, which doesn't take the conversation's lock. Then one call with the result schema gives {"answer","found"}, which Go checks and bounds. The whole transcript goes to agent_messages, never to the chat.
4. Combine. On the same stream: the task is saved as a user message (linked from the run's message_id, so the chat shows it once), then the agents' bounded results as a system message, then one call with no tools and a clean window streams the answer, saved as the assistant message, followed by the stats line, title generation and a summary pass, as after any turn. With no result at all there's no call and nothing is saved.
5. Stop. Closing the run's request (Stop, a closed tab) cancels every agent, every waiting command and the answer; what was shown is kept.

Streaming protocol#

Replies stream as text/plain: the model's tokens, plus marker lines the client consumes:

Fixed decisions#

The reasons behind the settled choices (pure-Go SQLite, a single connection, cookie sessions instead of JWT, 404 instead of 403, no CORS, native tool calls, the nudge placement and so on) are described where they apply: security.md for accounts, sessions and the shell tool, context-management.md for prompts and the window, and experiments.md for the measurements behind them.

Where the ideas come from#

Each borrows from research, scaled down to what a small local tool can honestly claim:

• The retry budget is a bounded verifier loop, after Snell et al. on test-time compute. A shell exit code is a free, high-precision verifier, so every tool result states success or failure explicitly ([exit code: N] or [FAILED, exit code: N]; a silent failure used to look like success; empty output says (no output), and a grep with no match suggests a shorter word), and after 3 steps that needed your approval (or 8 in all) the model has to answer in text. A small retry budget beats one greedy attempt without a bigger model, while each approved retry still costs you a decision; reads and refused proposals cost you nothing, so only the total bounds them (E30).
• The shield takes the idea of shielding (Alshiekh et al.): block unsafe actions before they run. Unlike that work it's a pattern list, not a proof. A shell is Turing-complete, so it's a floor under human approval, not a guarantee.
• Preconditions are the ToolGate / Hoare-triple idea at its smallest: check that a command can work before asking you to approve it.
• Decay by observed signals. How fast old tool output fades depends on what can actually be seen (a failed result fades 3x slower, an exploring command's output 2x faster), not on a belief filter whose transition probabilities would have to be invented, with no data to learn them from.

Deliberately not built#

• A bundled RAG or embedding stack. It's the weight Flint exists to avoid. @web re-ranking uses Ollama's own /api/embed instead.
• Sampling several commands to detect confabulation (semantic entropy, Farquhar et al.). It triples the model calls per tool decision, and grouping shell commands by syntax gives false alarms, because many different commands do the same thing (find, ls -R and git ls-files all count the same files).
• Conformal abstention by embedding distance to the folder manifest. With no calibration set, any cutoff would be arbitrary presented as a bound. The score doesn't track hallucination either: a correct "this code doesn't have that" is far from the manifest, and a wrong answer that name-drops real files is close to it.

Neither should come back without solving the problem that stopped it.