Overview
Upstash Ratelimit AI
It is a specialized rate limiting library designed for Large Language Model (LLM) API providers (OpenAI, Anthropic, Azure etc.) built on top of Upstash Redis. This connectionless library provides token-aware rate limiting with built-in support for request scheduling and analytics.
Quick Links
Github Repository
Have a look at the source code
Getting Started
Start using Ratelimit AI
Features
See the things you can do with Ratelimit AI
Features
Multiple Rate Limits
Support for RPM (Requests Per Minute), RPD (Requests Per Day), and TPM (Tokens Per Minute)
Token Counting
Automatic token counting for both prompts and responses
Analytics
Built-in analytics support with Upstash Console integration
Request Scheduling
Automatic request scheduling with QStash when rate limits are hit
Serverless First
Designed for serverless environments including Edge functions
Easy Integration
Simple integration with popular LLM providers and AI SDKs