Frontier prices only for frontier work.

01 · Measured, not promised

Tested on 50 tasks it had never seen.

Fifty English tasks (40 Arena-Hard prompts and 10 requests with four asks each), judged pair by pair, twice with the order swapped, by a model from a third provider (Gemini 3.1 Pro). Costs are the ones the gateway reported for both sides, with Jev at its list price.

• Sonnet-class answers for much less, on equal footing. Basic and standard tasks cost 88% less than Sonnet 5, and were as good or better in 77% of them.
• Labelling customer messages (first benchmark): 95% cheaper than Sonnet 5, with the same accuracy (96 of 100 right on both sides, against the true labels).

Fifty tasks give a direction, not a precise figure. Method, raw data and how to rerun it: the full report.

Where it loses

• Against Opus on hard prompts. Taskpenny saves most of an Opus bill, but its cheap tiers were judged worse than Opus about half the time; critical tasks go to Opus itself and tied. If every answer must match Opus, Taskpenny is not there yet.
• The first full run. On 142 tasks against Sonnet 5, before the ceiling and the “split only when it pays” rule existed, Taskpenny was only 13% cheaper: critical tasks went to Claude Opus, a dearer model, and splitting did not pay at advanced level.
• Catalan. Our weakest set: in that first run, as good or better in 43% of 14 Catalan tasks.
• The judge likes long answers. Many of Taskpenny's losses on basic tasks were short, correct answers against longer ones, and Taskpenny's workers are told to show the key steps while the baselines get no instruction; Taskpenny also caps hidden reasoning (off on the two cheapest tiers, low on the other two) while the baselines use each model's default, so part of the saving came from that setting and not from routing. The equal-footing control gives the baseline both the same instruction and the same reasoning cap: that is why the headline is about 38% cheaper and as good or better in 74%, not 52.5% and 82%. The author also voted blind on 20 pairs and rated Taskpenny as good or better in 18, where the judge said 14; twenty votes from one person are too few to correct the judge, so the figures are the judge's.

02 · How it works

Four steps, and every one is on the receipt.

1. 01

   Decide

   Jev, TypeSafe AI's decision model, answers closed questions: should this request be split, which tier from 1 (basic) to 4 (critical) each part needs, and what form the answer should take.

2. 02

   Route

   Each task goes to the cheapest model of its tier. When a task is itself a choice (an option, a yes or no, a score), Jev answers it directly, with no language model.

3. 03

   Check

   Jev checks every result against the task's success criteria. A weak answer is repaired first, then moved up one tier, at most twice.

4. 04

   Receipt

   The parts are stitched into one answer and checked against the request once more. You get the answer, the full task tree and what every step cost.

Splitting uses more tokens in total. The saving is in money, because most tokens go to models that cost a fraction of a frontier model, and Taskpenny drops a plan when the parts would not go to clearly cheaper models: it dropped 5 of 11 plans in those 50 new tasks.

03 · Download and install

Free to download. You pay only for the models you use.

Taskpenny is free and open source, under the MIT license: no account, no subscription, and nothing added to what your model calls cost. It runs on your computer, and you pay the models with your own key.

pip install taskpenny

How to install it, in four steps

1. Check that you have Python

   Taskpenny needs Python 3.10 or newer. If you do not have it, get it from python.org.

   python3 --version

2. Install it and try it, with no key

   pip install taskpenny
   # replays a real run in your terminal: no key, no cost
   taskpenny demo

3. Add your key

   Real requests need one Vercel AI Gateway key: every decision is Jev's, and Jev is reached through Vercel AI Gateway. The same key reaches Claude, GPT and Gemini, and your own OpenAI, Anthropic or Gemini keys can be added next to it.

   export AI_GATEWAY_API_KEY=...
   # shows what your keys reach, without spending tokens
   taskpenny doctor

4. Send it real requests

   taskpenny run "Write a short email to move tomorrow's meeting to Thursday"
   # the live task tree in your browser
   taskpenny ui

   And taskpenny serve gives you an OpenAI-compatible endpoint, so any OpenAI client, or n8n, needs only a new base URL.

If you prefer Docker: in a copy of the repository, docker compose up starts the page and the API on your machine, at 127.0.0.1:8765.

04 · Who it is for

For people who send everything to one strong model.

05 · Compared

What the others do, and what Taskpenny adds.

From each project's own README and docs, September 2026. All of them are good at what they do, and several can sit next to Taskpenny: it speaks the OpenAI API.

06 · Questions

Frequently asked questions.