SUV validates startup business ideas. A founder submits an idea. The system researches the market, models the financials, critiques the plan, and writes the report. That is the visible job.
The second job is invisible. The same system ships its own upgrades — features built, tested, deployed, and proven live by agent loops. One harness does the business intelligence. The same harness manufactures itself.
This article opens the machine. Every claim below names the real file, the real tool, and the real number.
A report does not come from one prompt. It comes from a crew of specialist agents, each with a bounded toolset. The pipeline lives in suv-reportgen/app/workers/report_tasks.py and the agents live in suv-reportgen/agents/.
The crew, in order:
| Agent | File | Tools it may call |
|---|
| Researcher | agents/researcher.py | OpenAI search, mem0 memory, data validation, calculator, market sizing |
| Financial analyst | agents/financial_analyst.py | Calculator, financial calculator, market standards validator |
| Financial critic | agents/financial_critique.py | Calculator, financial calculator, SaaS benchmarks, growth simulator, market standards validator |
| Market strategist | agents/market_strategist.py | OpenAI search, calculator |
| Writer | agents/writer.py | Chart creator |
| Deck crew | agents/deck_crew.py | None — reads section titles, returns slide JSON |
The critic is the honest part. Its job is to attack the financial model, not to approve it. It compares projections against real industry benchmarks — conversion rate, churn, CAC, LTV — keyed by business model and stage (agents/tools/saas_benchmarks.py). A B2C prosumer app gets checked against B2C prosumer data, not enterprise averages. The benchmark table is concrete: conversion 2-5%, monthly churn 5-8%. The model either matches its bracket or the critic says why.
The deck crew has a fail-closed rule: max 12 slides, max 5 bullets per slide. If the agent fails, a deterministic outline built from the section titles renders instead. The deck never fails to render.
Every agent carries a spending cap. The cap is real — it counts actual tool executions, not prompt iterations. The ToolBudget class in agents/tools/budget.py wraps every CrewAI tool in a BudgetedTool that passes every call through one gate.
| Tier | Tool-call limit |
|---|
| Free | 8 |
| Advanced | 20 |
| Pro | 40 |
| Super-pro / Copilot | Unlimited |
When the researcher burns its budget, the wrapper does not raise. It returns BUDGET_EXHAUSTED_MESSAGE and logs once. The run continues with the evidence already gathered. No partial report, no crash, no silent overspend.
Not every question deserves the same compute. A complexity_classifier (app/services/complexity_classifier.py) sorts each chat turn into LOW, MEDIUM, or HIGH.
The rules are deterministic first — a pure function, no LLM in the decision path. “Name my coffee shop” is LOW. “Analyze this market and give me a go-to-market plan” is HIGH. The LLM is consulted only when the rules conflict.
The measured difference is real: a LOW turn answers in 2.7 seconds, a HIGH turn takes 44.3 seconds and spends the deeper budget. The cheap question never pays for the expensive answer.
The idea chat is a stateless interview agent (app/services/idea_chat_agent.py). It captures a business idea into the broker’s 10 fields — title, overview, category, offering type, target audience, business model, pricing strategy, target market, customer segments, tags — one question at a time.
The agent replies with a machine-readable block per turn: FIELD_JSON {"field": "value"} plus optional questionnaire markers. The endpoint parses the block into structured fields. No free-form text guessing.
The action panel is where the agent gets hands. Four tools, one transport (app/services/action_events.py):
| Tool | What it does |
|---|
edit_idea | Changes report data — target market, pricing, category |
rerun_report | Regenerates the report with new context |
write_code | Writes a script into the user’s sandbox |
run_code | Executes it, with full sandbox controls |
web_search backs the panel with live data. The agent prompt is explicit: if the report lacks a market fact, search for it and cite the source before answering.
Every action emits a structured event: {"tool": "edit_idea", "status": "running|done|failed", "summary": "..."}. The SSE endpoint frames each event as event: action, and the UI renders a live panel — the user watches the agent work, step by step.
run_code executes real Python. It does so inside a fail-closed sandbox (app/services/code_sandbox.py). Every control is a hard limit:
- Path containment. Any user-supplied path must stay inside the per-user sandbox root, or
SandboxPathError fires.
- Isolated interpreter.
python -I — no PYTHONPATH, no user site, no env inheritance.
- Scrubbed environment. Only
PATH=/usr/bin:/bin and HOME=<sandbox>. The pod’s secrets never reach the child process.
- Resource limits. 256 MB address space, 10 seconds CPU, 64 open file descriptors — set before exec.
- Wall clock. 30-second hard timeout. A timed-out child is killed and surfaced as a timed-out result.
- Output cap. stdout + stderr truncated to fixed byte and line ceilings.
- No background.
subprocess.run only. No Popen, no detach, no daemon.
The contract is test-locked: the BAR for the chat-agent gauntlet required “no silent background processes,” and the suite enforces it.
The harness remembers. Four layers hold different kinds of knowledge:
| Layer | Store | Holds |
|---|
| 1 | Built-in memory | Durable facts, user profile, session history |
| 2 | Skills | Procedures — exact commands, pitfalls, verification steps |
| 3 | Knowledge graph | 259 entity files in OKF SPO triples — people, systems, projects, skills |
| 4 | Hindsight + Mnemosyne | External runtime memory with LLM extraction, shared across agents |
Layer 4 is the newest. Hindsight runs as a daemon with embedded PostgreSQL. Mnemosyne runs in-process with SQLite. One provider is active at a time; the switch is one config command. The shared bank — one graph, one ID, reachable at 172.16.0.112:8888 — lets two agents on two hosts share one brain. A retain on one side becomes a recall on the other.
The report agents use the same pattern at the product level. Mem0 keeps per-user memory in a shared PostgreSQL + pgvector store, swapped from Chroma with a config change. A semantic cache holds LLM responses and Tavily queries — repeat questions skip the expensive call.
None of the above appeared by hand. It shipped through factory gauntlets — the AFK (Away From Keyboard) software factory pattern popularized by Matt Pocock’s Sandcastle — run with stricter discipline.
A gauntlet is a named loop with a plan, an acceptance bar, and a driver process. One builder owns a repo at a time. The loop:
- Plan — PLAN.md breaks the wave into atomic tasks; BAR.md names the bar.
- Build — each tick completes 3-6 tasks; tests ship with code.
- Critic — a fresh-context subagent inspects the real artifact, runs the tests, names the biggest gap.
- Fix — the builder fixes and resubmits; max three rounds, then DEFERRED with analysis attached.
- Gate — canonical suites must pass; status file flips State.
- Deploy wave — the four repos push in order; Kaniko builds, MicroK8s deploys, Argo Rollouts canaries.
- Prove + EXIT — live proofs against the deployed cluster with a throwaway user. Pushed state equals tested state.
The discipline rules are what make it trustworthy:
- Deferred push. Builders commit locally. Code pushes once, at the deploy wave, in order.
- Owner labels. A task tagged to another agent is never claimed. Two agents share one repo without collision.
- False-exit guard. A loop exits only on the full output match. A stray line cannot fake a finish.
- Flock single-flight. Launchers hold a lock; manual trigger and cron cannot double-spawn a driver.
The chain is cron-driven. Each loop’s exit flips a gate; a launcher polls it every five minutes and starts the next driver. The 2026-08-07 wave ran the chain end to end:
| Loop | Scope | Result |
|---|
| B9 report-copilot | Context-resolved edits, session persistence | 14/14 tasks, EXIT |
| B11-B14 account wave | Notifications bell+email, NY-legal ToS/privacy, account functions, API docs | 18/18 tasks, 23/23 live proofs, EXIT |
| B10 Google login | OAuth 2.0, account linking | RUNNING — chained auto-start |
CrewAI provides the agent framework the report crews run on (CrewAI docs). Everything else — the budgets, the sandbox, the gates, the chain — is custom, tested, and proven live.
The harness is not a demo. It is the production path. The account wave shipped 18 tasks, four pipelines, and 23 live proofs in a day, and the next loop started itself behind it.
The same discipline runs inside the product. The researcher spends a bounded budget. The critic attacks the model against real benchmarks. The code the agent runs is sandboxed to 256 MB and 30 seconds. Nothing runs unbounded, nothing runs unobserved, nothing runs unproven.
That is the difference between an agent and a harness. An agent tries. A harness budgets, critiques, gates, deploys, and proves — then starts the next loop.
