What we are building, what broke, what we learned. Long-form technical writeups about LLM billing, multi-provider routing, observability, and the practical parts of running an OpenAI-compatible gateway in production.
The fields the SDK sends on the wire, the ones it deliberately does not, how cost is computed without seeing the prompt, and the GDPR / LGPD implications of a database that cannot leak content.
Read →Hold/capture/release, idempotency under streaming, orphan cleanup, and the four iterations it took to charge customers for upstream usage without double-spending or leaking holds.
Read →Subscribe to the GitHub repo to follow along, or get on the mailing list by creating a free account.