Course · Pi · Models & config

Route the cheap work to the cheap model

Here’s the backlog sitting in front of you on stash right now, and it’s deliberately lumpy:

Re-extract every URL saved before the readability fix. A few hundred rows: fetch the page, run it through the (now-corrected) extractor, overwrite the stored text. Mechanical. Every row is the same shape of work.
Redesign how the extractor handles the sites it still mangles. One judgment call with real branches - which signals to trust when a page has no clean <article> tag, how far to fall back before giving up and storing the raw text. Get it wrong and you’re back here next month with the same bug.

Run both through the same pinned model at the same reasoning depth and you’re either overpaying on the sweep or underthinking the redesign. Pi actually gives you two separate levers to avoid that, and they’re worth naming separately because they compound.

Here’s the claim this lesson rederives by hand, on the sweep, before it touches a single JSON file: pinned to the expensive corner, four rows of “fetch, strip tags, save” cost about fifteen times what the same four rows cost at the matched corner - for output that’s identical either way. Not fifteen times more thinking. Fifteen times more bill.

(This chapter’s dial is Pi’s own settings.json and models.json; the Foundations model-selection chapter has the spec-level view of how all six tools in this site’s scope handle the same choice.)

The two dials

Which model answers - set by defaultModel (and defaultProvider) in settings.json, or picked mid-session:

/model opens the full model picker.
Ctrl+L opens the same selector as an overlay, without leaving what you’re typing.
Ctrl+P cycles through whatever’s in your enabledModels allow-list - the curated shortlist from the last lesson, not every model every provider exposes.

How hard it thinks - Pi calls this the thinking level, and it’s a first-class setting, not a hidden knob: off | minimal | low | medium | high | xhigh | max, set globally via defaultThinkingLevel, cycled mid-session with Shift+Tab, and - if a provider’s defaults don’t fit - retuned per level with thinkingBudgets (a custom token allocation for each rung of the scale).

Neither dial cares what the other is set to. That’s the point: a cheap model at high thinking and a frontier model at off are both real, useful, and completely different tools. The re-extraction sweep wants a cheap model at low or off - there’s nothing to reason about, only rows to process. The extractor redesign wants your strongest model at high - a wrong call here is a rewrite, same as it would be anywhere else in this course.

Trace the sweep by hand

Take four rows off the re-extraction backlog - small enough to add up in your head, and standing in for the “few hundred” that make up the real chore. Give each row a toy spend unit of 1 at the cheapest matched corner: a light model, thinking off. Nothing about the row changes what it costs to process; four mechanical rows cost four units, at that corner, by definition.

Guess before the next paragraph: if you ran those same four rows through your strongest model at its highest thinking level instead, would you guess the bill doubles? Triples? Something bigger?

Now trace it. A capable model bills roughly five times what a light model does, per token. High thinking generates roughly three times the tokens a low-thinking pass does, on the same request - reasoning tokens the sweep has no use for, since there’s nothing in “fetch, strip tags, save” to reason about. Multiply the two: 5 x 3 = 15. Four rows at the expensive corner cost 4 x 15 = 60 toy units, against 4 units at the matched corner. Fifteen times the spend. Zero extra output. That’s the number this lesson opened with, rederived from two multipliers you can multiply yourself.

Route the other task - the extractor redesign - through the cheap corner and the trap flips shape instead of disappearing. The redesign has a real branch in it: which signals to trust when a page has no clean <article> tag. A light model at low thinking doesn’t do a shallower version of that job; it’s more likely to get the branch wrong, and a wrong call here isn’t a bad paragraph, it’s a design decision you’re now shipping. Catch it in review and you pay for the light model’s attempt, then pay again for a capable model to redo it properly - which comes out more expensive than routing it correctly the first time. Verdict: wrong-cheap costs more than right-expensive.

The 5x and 3x multipliers above are the same ones the widget below uses, on a small set of task types rather than one toy sweep - deliberately coarse, chosen so the ratios are the point, not exact pricing. Set every task’s dials to the corner you’d have picked out of habit, then to the corner each task’s own verdict recommends, and watch which one racks up the redo tax:

A lumpy day: four kinds of task, and two dials on each - which model answers, and how much effort it spends thinking. Everything starts where most people leave it: pinned to the expensive corner. Re-dial each task and watch what the day costs.

presets

first runs495 unitsredo tax0 unitsvs the matched day1.9×

Everything ships - no failures, no redo tax - and the day still costs 1.9× what it should. That’s the quiet leak of a pinned dial: the mechanical work bills like hard work, five and three times over. Dial it down to the cheapest corner that ships it; the hard problems keep their budget.

Numbers are illustrative - the ratios are the point (capable ≈ 5× per token, high effort ≈ 3× the tokens). The redo tax counts an underpowered task’s failed attempts plus the escalation you’d run anyway, not the hour lost to confident wrong answers.

Actually adding the cheap model

enabledModels and defaultModel only help once a cheap option is defined for Pi to route to. That’s models.json, and it’s usually easier to have Pi do the editing than to hand-write the provider block yourself - it can read its own docs and figure out the shape a new provider needs. Say you want to add a low-cost coding model behind an OpenRouter-style API key, for the sweep work specifically:

> add a custom provider to models.json for a cheap coding model I can
  route the stash re-extraction backlog to, then tell me where to paste
  the API key

Pi identifies that this isn’t a login-subscription provider the way OpenAI or GitHub Copilot are - it needs a models.json entry with its own baseUrl, api shape, and an apiKey field, plus (depending on how the key resolves) a matching entry in auth.json:

{
  "providers": {
    "cheap-coder": {
      "baseUrl": "https://openrouter.ai/api/v1",
      "api": "openai-completions",
      "apiKey": "$OPENROUTER_API_KEY",
      "models": [
        { "id": "some-cheap-coding-model", "name": "Cheap coder", "reasoning": true, "contextWindow": 128000 }
      ]
    }
  }
}

apiKey resolves three ways - a literal string, a $ENV_VAR interpolation, or a !command whose stdout it uses - so you’re not forced to put the key in the file at all. Paste the key yourself; don’t ask Pi to read the file back and confirm it took. Agents that can read files can also print them, verbatim, to whatever terminal you’re sitting in - including one you might be sharing or recording. If a key is ever echoed back to you in plaintext, treat it as burned and rotate it, on principle, whether or not you think anyone else saw it.

Once the provider’s live, /model shows it in the picker and Ctrl+P cycles it if it’s in enabledModels. Point the re-extraction sweep at it, low thinking, and let it run through the backlog for a fraction of what the frontier model would have billed for the same mechanical rows. Keep the extractor redesign on your strongest model at high - that one still earns every token.

That reconciles the opening claim: the toy trace put the ratio at 15x for the sweep specifically, because the sweep has no real use for the extra reasoning it would otherwise be billed for. A different task mix will land on a different multiple, but the mechanism is the same one you just multiplied by hand - model price times thinking-generated tokens - and it’s checkable on any task the same way: pick the toy size, pick the corner, multiply.

When routing doesn’t help: a one-off task you’ll type once and never repeat isn’t worth a /model switch - whatever the switch would save costs you more attention than it’s worth, and attention is the thing this whole exercise is supposed to protect. Routing also doesn’t help when you can’t actually tell, before you start, whether a task is mechanical or has a branch hiding in it - misjudging that costs more either direction than not routing at all. Route the backlog chores you’ve done before and recognize on sight; leave anything genuinely new on your default model until you’ve seen its shape once.

One thing routing doesn’t give you: a ceiling. Nothing in settings.json or models.json caps total spend or warns you before a runaway loop bills the frontier model all night - Pi’s core product doesn’t manage token spend for you, by the same minimal-core choice that keeps its file count small. The community has filled that gap with a third-party extension or two rather than Pi shipping one; if a hard spend cap matters to you, check your provider’s own dashboard for a spend alert, because Pi’s config surface won’t stop you before the fact.

Routing between hosted models is one lever. There’s a more extreme version of “cheap” available too: a model that costs nothing per token because it’s running on the machine in front of you. See what that actually buys you - and where it runs out of room.