BricksLLM: AI Gateway For Putting LLMs In Production, Written in Golang

BricksLLM: AI Gateway For Putting LLMs In Production, Written in Golang

BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM.

BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases.

Use-cases BricksLLM

Here are some use cases for BricksLLM:

  • Set LLM usage limits for users on different pricing tiers
  • Track LLM usage on a per user and per organization basis
  • Block or redact requests containing PIIs
  • Improve LLM reliability with failovers, retries and caching
  • Distribute API keys with rate limits and cost limits for internal development/production use cases
  • Distribute API keys with rate limits and cost limits for students

Features

License

MIT License.

Resources & Downloads

GitHub - bricks-cloud/BricksLLM: 🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports Open…









Open-source Apps

9,500+

Medical Apps

500+

Lists

450+

Dev. Resources

900+