What Is billion-context and How to Use It: Reversible Compression for Billion Token AI Sessions

When you run long projects with an AI coding agent, context balloons and the session degrades or dies once it hits the window limit. billion-context is an open source context compression proxy whose motto is billion-context is all you need. Sitting between your agent and the model API, it compresses long chats into high fidelity summaries, so a 100K window can support month long sessions and save about 5x tokens without losing the conversation.

What Is billion-context

billion-context is a general purpose context compression proxy, positioned as billion-context is all you need. It runs as a standalone HTTP reverse proxy between any AI coding agent, such as Claude Code, Codex, Cursor, or Aider, and the model API. Through the acp-kernel compression engine it rewrites the model stream and injects four context tools: compress, decompress, search_context, and acp_status. The model itself decides when and what to compress into a high fidelity summary. It addresses a systemic problem in long human agent collaboration, keeping the agent coherent across months rather than solving a single model limit, and it does so without breaking the prefix cache that keeps latency low.

It sits between the client and the model, rewriting requests and responses so the application keeps its normal API shape while the effective context grows far beyond the window.

Features

• Incremental compression. Summaries are written in small blocks, reversible and restorable, not one shot truncation that throws history away.
• Cache friendly. Prefix caches stay intact, reaching 95 to 97 percent hit rates with compression enabled.
• Self managed tools. The model compresses, decompresses, and searches on its own through four built in tools, with no human in the loop.
• Long session support. A 100K window can run sessions of days to months, totaling billions of tokens over a single project.
• Broad client coverage. Native plugins, launchers, and URL rewriting cover more than 20 clients out of the box.
• Visible management. A Web UI dashboard and self update let you watch compression state and keep the proxy current.
• Transparent control. Compression can be searched and restored, so you can verify the summary missed no critical detail before trusting it.
• Model driven decisions. The model decides when to compress and which details to keep, rather than a fixed token threshold applied blindly.
• Local storage. State lives in node:sqlite on your machine, so compressed history stays under your control and survives restarts.

How to Use

1. Prepare the environment. You need Node 24 or later, then build with esbuild or tsup following the project docs.
2. Start the proxy. Run it as a standalone HTTP reverse proxy, listening on port 8787 by default.
3. Connect a client. Use the bili <client> launcher or bili plugin install <client> to add a supported client.
4. Rewrite addresses. Forward requests through the /bili/ prefix that embeds the upstream absolute URL for transparent routing.
5. Watch status. Open the Web UI dashboard to inspect acp_status and cache hits during a live session.
6. Run long term. Enable self update so the proxy tracks upstream versions without manual upgrades.
7. Add clients. Install more client plugins over time to bring your common agents under the same compression layer.

Supported Platforms

The proxy stores state in node:sqlite and organizes capabilities with the cordis plugin framework, which keeps the codebase modular as new clients land. Because it is an HTTP reverse proxy, you can place it in front of almost any client by rewriting the API endpoint rather than patching the client itself.

Pricing

billion-context is MIT licensed and free, with one extra attribution clause: products or services with a user visible interface must credit billion-context and acp-kernel and link to the repo. The code is public on GitHub and you can self host it on your own server without paying the author, which keeps sensitive conversation data inside your own network.

There is no hosted tier to subscribe to, and the binaries needed to run the proxy are part of the repository.

Use Cases

• Long running coding. Keep AI coding agents from losing history or switching sessions over months of continuous work.
• Cost reduction. Reuse cache under a smaller window and cut token spend sharply on large projects.
• Multi agent teams. Give every client uniform compression for a consistent, predictable experience.
• Research and prototypes. Replace hard truncation with reversible compression in long context experiments.
• Solo developers. Run the proxy on your own machine and avoid upgrading plans for long context. If you also bolster agent memory, see What Is claude-mem and How to Use It: Persistent Cross Session Memory for AI Agents.
• Codebase onboarding. Keep architecture decisions and earlier failed attempts available while an agent explores a large unfamiliar repository.
• Batch experiments. Apply one compression policy across many evaluation runs so results stay comparable instead of drifting with the window.

How It Works

billion-context is community maintained as a pluggable layer between agents and models. It is built on the acp-kernel compression core, organized with the cordis plugin framework, and stores state in node:sqlite. Released under MIT with attribution, it encourages free integration on compliant terms, with a roadmap toward higher compression and wider client coverage. Its value is making compression a pluggable layer rather than something hard coded into one agent, so any compatible client benefits. As long context costs stay in focus, its reach will grow, and for teams that lean on AI coding agents it is the most direct way to lower bills and keep history intact across very long sessions.

Because compression is reversible, the agent can request the original block again when a summary proves too thin for the current task.

Conclusion

If the not enough context problem keeps biting your AI coding agent, billion-context offers a reversible, cache friendly path. It uses engineering compression to bring long session costs down instead of stacking bigger windows. It does not replace larger windows, but lets small windows carry big tasks, and it is worth trying first for anyone who treats AI coding agents as a main tool. Project address https://github.com/ranxianglei/billion-context .

Try it on one long running project first, where the cache savings are easiest to measure.