AI Engineering
Cost
The LLM Gateway: How We Cut AI Running Costs by 80% and Survived Multi-Hour Provider Outages
One layer in front of every AI request, choosing which model answers it. It kept 700+ employees working through outages at two providers and cut AI running costs by up to 80%. The architecture, the trade-offs and the lessons.
17 August 20268 min read