You Don't Need Cloud AI for Everything. Route It Instead of Burning It.
Your company is paying $25 per million output tokens for a model that answers the same question 10,000 times a month. The answer never changes. The bill does.
Wendell of Level1Techs made the full case in You Don’t need to use Cloud AI! Switchyard and Nemotron 3.5 Lightning. The argument is blunt: frontier inference for every task is setting money on fire. The fix is a local model router with a feedback loop. NVIDIA shipped both halves this month.
The trap is the default
Section titled “The trap is the default”The trap is not just the token price. The trap is turning judgment over to the model.
When a company buys a frontier subscription and turns it loose, it loses the ability to answer three questions. What did the AI do? Why did it take that step? What information did it use? Without those answers, a failure teaches nobody anything (Level1Techs, 2026).
Software engineers want labor augmentation, not delegation. They want supervision. They want structure. They want to know what problem they are solving. A router that logs every decision gives them that. A blank chat window does not.
Switchyard: the router with a paper trail
Section titled “Switchyard: the router with a paper trail”Switchyard is NVIDIA’s open-source supervision architecture for routing user requests across AI models (NVIDIA Developer Blog).
A request arrives. The router sends it to a specialized local worker, a small model, a customized model, or a tool-calling agent. Each step produces an observable trace. The router accepts the result or escalates it to a stronger model. Human supervision is a first-class component, not an afterthought.
The design gives you three concrete advantages:
| Advantage | What it buys you |
|---|---|
| Cost | A $5-per-million-token local model handles what Opus-class models were doing at $25 |
| Observability | You know which component did what, where, in your organization |
| Organizational learning | Every routing decision and escalation becomes a dataset |
That dataset is the real prize. It shows how your people actually use AI, where they get stuck, and which workflows repeat. It is institutional knowledge, not telemetry (Level1Techs, 2026).
Nemotron 3.5 Lightning: built to be customized
Section titled “Nemotron 3.5 Lightning: built to be customized”Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model with 3 billion active parameters (NVIDIA NIM model card). It is fast, it is local, and it is explicitly pitched as a model you are supposed to customize (NVIDIA Developer Blog).
Customization does not mean full retraining. NVIDIA ships the LoRA recipes, the supervised fine-tuning setup, the reinforcement learning config, and the training data. You freeze the base model and train a small set of additional weights for your specific job.
NIM takes it one step further. It keeps one base model resident and dynamically loads and unloads LoRA adapters while serving (NVIDIA Developer Blog).
Accounting gets the accounting adapter. Software engineering gets the code-review adapter. Support gets the support adapter. That undocumented internal product from 2017 gets the adapter containing the dark knowledge known only to Gary. Gary can finally take a vacation.
One base model, many specialists, no retraining, no cloud round-trips.
The flywheel: measure, curate, promote
Section titled “The flywheel: measure, curate, promote”NVIDIA’s data flywheel blueprint makes the loop explicit (NVIDIA Developer Blog):
- Instrument the AI application and log production traffic.
- Build evaluation and fine-tuning datasets from those logs.
- Evaluate smaller models against the data.
- Customize the ones that work.
- Promote them and measure again.
You cannot improve what you do not measure. The flywheel saves tokens because the small local model is cheaper on every request. It saves sanity because every escalation is a recorded decision, not a guess.
The benchmark backs it up. On Humanity’s Last Exam, an NVIDIA-orchestrated system scored 37.1% versus 35.1% for GPT-5, at 30% of the cost and 2.5 times faster (Artificial Analysis, Wikipedia). The small model beat the frontier model at the task because it was inside a tool-calling ecosystem, not because it tried to know everything.
Why the cloud providers are nervous
Section titled “Why the cloud providers are nervous”Wendell’s sharpest observation is the dark pattern. Recent Codex CLI updates surface less reasoning detail in the UI, and there are open issues about reasoning summaries missing from the session log (Level1Techs, 2026).
High-quality outputs and task traces are exactly the raw material you need to train a cheaper specialized model. OpenAI even sells model distillation around that idea. There is an economic incentive for frontier providers to keep useful internal signals from being trivially exportable.
That is the argument for owning your loop. The company that routes and logs its own AI traffic stops renting intelligence and starts accumulating it.
What to do Monday
Section titled “What to do Monday”- Route by task class. Send simple, repeated tasks to a small local model. Send only the hard edge cases to the frontier.
- Capture every decision. Log what the model did, why, and what the human corrected. That log is your training data.
- Customize what repeats. If users ask the same question 10,000 times, give the small model a LoRA adapter that answers it from your own data.
The era of using a frontier model for every little task is ending. The high water mark for cloud token spend is here. The machines that replace it are smaller than you think, and they sit on a desk.