QIIP 0.2.0 has arrived, bringing a token leaderboard, smarter node provisioning, and 145 new commits to the QUADS Idle Inference Proxy. We’ve brought a gigantic set of features and fixes forward into this second big production release!
QIIP is the QUADS Idle Inference Proxy: it finds GPU capacity that would otherwise sit idle, serves it as an OpenAI-compatible endpoint, and tears nodes down ahead of scheduled work. In its first 13 days, QIIP handled 4.22 billion tokens across 33,699 requests from 22 active users, with an estimated cost of $13,061.61, about 40% below the premium list-price estimate of $21,769.34.
v0.2.0 is now tagged, with 145 commits over v0.1.0. This release is the direct result of feedback from the proof of concept: it addresses review findings across provisioning, GPU setup, llama.cpp artifacts, auth, and the new dashboard.
What’s new in v0.2.0
The headline feature is visibility. QIIP now keeps a token leaderboard and dashboard: per-user token counts, requests, share of usage, and estimated token cost, sorted by any column, with an admin toggle to include admins in the ranking. You can also copy your own API token straight from the pages.
Node provisioning got a big overhaul:
- Power-on new nodes via Redfish, with BMC credentials resolved per lab
- Reuse compatible GPU drivers and verified llama.cpp builds during setup
- Size automatic vLLM launches from available memory, and verify vLLM identity before routing
- Resume verified provisioning stages after transient failures
- Auto-rediscover model drift on self-setup nodes, and delete cached models from the admin UI
- RPM releases for Fedora for production and development releases
Auth picks up an opt-in local OIDC provider with the local login and admin sign-in paths unified. Release RPMs are linked from the release notes, and token cost estimates are priced at current Sonnet model rates.
By the numbers
From v0.1.0 to v0.2.0: 145 commits, 61 of them from release automation. First 13 days: 4.22B tokens across 33,699 requests, with an estimated cost of $13,061.61, about 40% below the premium list-price estimate of $21,769.34.
Try it and tell us
Source and docs live at github.com/quadsproject/qiip. If something misbehaves, file an issue with the timestamp and what you were doing; that feedback is what made this release.
