> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/cheahjs/free-llm-api-resources/llms.txt
> Use this file to discover all available pages before exploring further.

# Understanding Rate Limits

> Learn how to work with rate limits across free LLM API providers

## Overview

Rate limits determine how many requests you can make to an API within a specific time window. Understanding and optimizing for these limits is crucial for building reliable applications with free LLM APIs.

## Types of Rate Limits

<CardGroup cols={3}>
  <Card title="Request-Based" icon="hashtag">
    Limits the number of API calls per time period (e.g., 20 requests/minute)
  </Card>

  <Card title="Token-Based" icon="coins">
    Limits the number of tokens processed per time period (e.g., 60,000 tokens/minute)
  </Card>

  <Card title="Quota-Based" icon="calendar">
    Total allowance over longer periods (e.g., 1,000 requests/month)
  </Card>
</CardGroup>

## Provider Rate Limits Comparison

### High Volume Providers

<Accordion title="Providers with 10,000+ Requests/Day">
  | Provider                     | Requests/Minute | Requests/Day | Tokens/Minute | Notes           |
  | ---------------------------- | --------------- | ------------ | ------------- | --------------- |
  | **Cerebras**                 | 30              | 14,400       | 60,000-64,000 | Varies by model |
  | **Google AI Studio (Gemma)** | 30              | 14,400       | 15,000        | Per model       |
  | **Groq (Llama 3.1 8B)**      | -               | 14,400       | 6,000         | -               |
  | **Groq (Llama Guard)**       | -               | 14,400       | 15,000        | -               |

  <Info>These providers are suitable for production applications with consistent traffic.</Info>
</Accordion>

### Medium Volume Providers

<Accordion title="Providers with 1,000-10,000 Requests/Day">
  | Provider                 | Requests/Minute | Requests/Day | Tokens/Minute | Notes            |
  | ------------------------ | --------------- | ------------ | ------------- | ---------------- |
  | **Mistral Codestral**    | 30              | 2,000        | -             | Code-focused     |
  | **Groq (Llama 3.3 70B)** | -               | 1,000        | 12,000        | Larger model     |
  | **Groq (Compound)**      | -               | 250          | 70,000        | High token limit |

  <Note>Good for development and small to medium applications.</Note>
</Accordion>

### Low Volume Providers

<Accordion title="Providers with <1,000 Requests/Day">
  | Provider                      | Requests/Minute | Requests/Day | Monthly Quota | Notes                    |
  | ----------------------------- | --------------- | ------------ | ------------- | ------------------------ |
  | **OpenRouter**                | 20              | 50           | -             | 1000/day with \$10 topup |
  | **Cohere**                    | 20              | -            | 1,000/month   | Shared across models     |
  | **Google AI Studio (Gemini)** | 5-15            | 20-500       | -             | Varies by model          |
  | **NVIDIA NIM**                | 40              | -            | -             | Per-minute only          |

  <Warning>These are best for prototyping and low-traffic applications.</Warning>
</Accordion>

## Token Limit Strategies

### High Token Throughput Providers

| Provider                      | Tokens/Minute | Tokens/Day | Tokens/Month  | Best For                           |
| ----------------------------- | ------------- | ---------- | ------------- | ---------------------------------- |
| **Mistral La Plateforme**     | 500,000       | -          | 1,000,000,000 | Long documents, content generation |
| **Google AI Studio (Gemini)** | 250,000       | -          | -             | Multimodal, analysis               |
| **Groq (Compound)**           | 70,000        | -          | -             | Fast inference                     |
| **Cerebras**                  | 60,000-64,000 | 1,000,000  | -             | Consistent high volume             |

<Tip>
  Token limits are often more important than request limits for applications processing large documents or generating long-form content.
</Tip>

## Handling Rate Limits

### 1. Exponential Backoff

Implement retry logic with exponential backoff:

<CodeGroup>
  ```python Python theme={null}
  import time
  import random
  from openai import OpenAI, RateLimitError

  def call_with_backoff(client, model, messages, max_retries=5):
      for attempt in range(max_retries):
          try:
              response = client.chat.completions.create(
                  model=model,
                  messages=messages
              )
              return response
          except RateLimitError as e:
              if attempt == max_retries - 1:
                  raise
              
              # Exponential backoff: 1s, 2s, 4s, 8s, 16s
              wait_time = (2 ** attempt) + random.uniform(0, 1)
              print(f"Rate limited. Waiting {wait_time:.2f}s...")
              time.sleep(wait_time)

  # Usage
  client = OpenAI(
      base_url="https://openrouter.ai/api/v1",
      api_key="your-api-key"
  )

  response = call_with_backoff(
      client,
      "meta-llama/llama-3.3-70b-instruct:free",
      [{"role": "user", "content": "Hello!"}]
  )
  ```

  ```javascript JavaScript theme={null}
  const OpenAI = require('openai');

  async function callWithBackoff(client, model, messages, maxRetries = 5) {
    for (let attempt = 0; attempt < maxRetries; attempt++) {
      try {
        const response = await client.chat.completions.create({
          model: model,
          messages: messages
        });
        return response;
      } catch (error) {
        if (error.status !== 429 || attempt === maxRetries - 1) {
          throw error;
        }
        
        // Exponential backoff with jitter
        const waitTime = (2 ** attempt) * 1000 + Math.random() * 1000;
        console.log(`Rate limited. Waiting ${waitTime/1000}s...`);
        await new Promise(resolve => setTimeout(resolve, waitTime));
      }
    }
  }

  // Usage
  const client = new OpenAI({
    baseURL: 'https://openrouter.ai/api/v1',
    apiKey: process.env.OPENROUTER_API_KEY
  });

  const response = await callWithBackoff(
    client,
    'meta-llama/llama-3.3-70b-instruct:free',
    [{role: 'user', content: 'Hello!'}]
  );
  ```
</CodeGroup>

### 2. Rate Limit Headers

Monitor rate limit headers in API responses:

```python theme={null}
response = requests.post(
    "https://api.groq.com/openai/v1/chat/completions",
    headers={"Authorization": f"Bearer {api_key}"},
    json={"model": "llama-3.3-70b-versatile", "messages": messages}
)

# Check rate limit headers
remaining = response.headers.get('x-ratelimit-remaining-requests')
limit = response.headers.get('x-ratelimit-limit-requests')
reset = response.headers.get('x-ratelimit-reset-requests')

print(f"Remaining: {remaining}/{limit}, Resets: {reset}")
```

<Info>
  Not all providers return rate limit headers. Check the provider's documentation for specifics.
</Info>

### 3. Request Queuing

Implement a queue to stay within rate limits:

```python theme={null}
import asyncio
from asyncio import Queue, Semaphore
import time

class RateLimiter:
    def __init__(self, max_per_minute=20):
        self.max_per_minute = max_per_minute
        self.semaphore = Semaphore(max_per_minute)
        self.requests = []
    
    async def acquire(self):
        # Remove requests older than 1 minute
        now = time.time()
        self.requests = [t for t in self.requests if now - t < 60]
        
        # Wait if at limit
        while len(self.requests) >= self.max_per_minute:
            await asyncio.sleep(1)
            self.requests = [t for t in self.requests if time.time() - t < 60]
        
        self.requests.append(time.time())
    
    async def call_api(self, client, model, messages):
        await self.acquire()
        return await client.chat.completions.create(
            model=model,
            messages=messages
        )

# Usage
limiter = RateLimiter(max_per_minute=20)  # OpenRouter limit
response = await limiter.call_api(client, model, messages)
```

### 4. Token Counting

Count tokens before sending requests:

```python theme={null}
import tiktoken

def count_tokens(text, model="gpt-3.5-turbo"):
    encoding = tiktoken.encoding_for_model(model)
    return len(encoding.encode(text))

def estimate_cost(prompt, max_tokens=100):
    input_tokens = count_tokens(prompt)
    total_tokens = input_tokens + max_tokens
    return total_tokens

# Check before calling
prompt = "Explain quantum computing"
if estimate_cost(prompt) > 4000:  # Stay under token limit
    print("Prompt too long, truncating...")
    prompt = prompt[:2000]  # Rough truncation
```

## Multi-Provider Load Balancing

Distribute requests across multiple providers:

```python theme={null}
from typing import List, Dict
import random

class MultiProviderClient:
    def __init__(self, providers: List[Dict]):
        """
        providers = [
            {
                'name': 'openrouter',
                'client': openrouter_client,
                'model': 'meta-llama/llama-3.3-70b-instruct:free',
                'limit_per_day': 50
            },
            {
                'name': 'groq',
                'client': groq_client,
                'model': 'llama-3.3-70b-versatile',
                'limit_per_day': 1000
            }
        ]
        """
        self.providers = providers
        self.usage = {p['name']: 0 for p in providers}
    
    def get_available_provider(self):
        # Filter providers under their daily limit
        available = [
            p for p in self.providers
            if self.usage[p['name']] < p['limit_per_day']
        ]
        
        if not available:
            raise Exception("All providers exhausted")
        
        # Weighted random selection based on remaining quota
        weights = [
            p['limit_per_day'] - self.usage[p['name']]
            for p in available
        ]
        return random.choices(available, weights=weights)[0]
    
    def call(self, messages):
        provider = self.get_available_provider()
        self.usage[provider['name']] += 1
        
        return provider['client'].chat.completions.create(
            model=provider['model'],
            messages=messages
        )

# Usage
from openai import OpenAI

providers = [
    {
        'name': 'openrouter',
        'client': OpenAI(base_url="https://openrouter.ai/api/v1", api_key=os.getenv("OPENROUTER_KEY")),
        'model': 'meta-llama/llama-3.3-70b-instruct:free',
        'limit_per_day': 50
    },
    {
        'name': 'groq',
        'client': OpenAI(base_url="https://api.groq.com/openai/v1", api_key=os.getenv("GROQ_KEY")),
        'model': 'llama-3.3-70b-versatile',
        'limit_per_day': 1000
    },
    {
        'name': 'cerebras',
        'client': OpenAI(base_url="https://api.cerebras.ai/v1", api_key=os.getenv("CEREBRAS_KEY")),
        'model': 'llama3.3-70b',
        'limit_per_day': 14400
    }
]

multi_client = MultiProviderClient(providers)
response = multi_client.call([{"role": "user", "content": "Hello!"}])
```

<Tip>
  **Pro Tip:** Use Vercel AI Gateway for automatic provider routing, caching, and observability.
</Tip>

## Monitoring and Alerts

Track your usage to avoid hitting limits:

```python theme={null}
import logging
from datetime import datetime

class UsageTracker:
    def __init__(self):
        self.daily_count = 0
        self.last_reset = datetime.now().date()
        self.logger = logging.getLogger(__name__)
    
    def track_request(self, provider, tokens_used):
        # Reset daily counter
        today = datetime.now().date()
        if today > self.last_reset:
            self.daily_count = 0
            self.last_reset = today
        
        self.daily_count += 1
        
        # Log usage
        self.logger.info(
            f"Provider: {provider}, "
            f"Daily requests: {self.daily_count}, "
            f"Tokens: {tokens_used}"
        )
        
        # Alert if approaching limit (e.g., 80% of OpenRouter's 50/day)
        if provider == 'openrouter' and self.daily_count >= 40:
            self.logger.warning(
                f"Approaching daily limit for {provider}: "
                f"{self.daily_count}/50 requests used"
            )

tracker = UsageTracker()
tracker.track_request('openrouter', 150)
```

## Best Practices

<CardGroup cols={2}>
  <Card title="Cache Responses" icon="database">
    Store and reuse responses for identical requests to reduce API calls
  </Card>

  <Card title="Batch Requests" icon="layer-group">
    Group multiple queries when possible to maximize token efficiency
  </Card>

  <Card title="Use Streaming" icon="water">
    Stream responses to provide faster perceived performance without extra requests
  </Card>

  <Card title="Implement Fallbacks" icon="shield">
    Have backup providers ready when primary provider hits limits
  </Card>
</CardGroup>

<AccordionGroup>
  <Accordion title="Optimize Prompts">
    **Reduce token usage:**

    * Use concise system messages
    * Avoid repeating context in conversations
    * Use smaller models when appropriate
    * Truncate long inputs intelligently

    ```python theme={null}
    # Bad: Repeating full context every time
    messages = [
        {"role": "system", "content": "You are a helpful assistant with extensive knowledge..."},
        {"role": "user", "content": "Question 1"},
        {"role": "assistant", "content": "Answer 1"},
        {"role": "user", "content": "Question 2"}
    ]

    # Good: Minimal context, reference previous answers
    messages = [
        {"role": "system", "content": "Helpful assistant."},
        {"role": "user", "content": "Q1"}
    ]
    ```
  </Accordion>

  <Accordion title="Time-Based Distribution">
    **Spread requests throughout the day:**

    For providers with daily limits (e.g., Cerebras with 14,400 req/day):

    * Max sustainable rate: \~10 requests/minute
    * Consider off-peak processing for batch jobs
    * Queue non-urgent requests

    ```python theme={null}
    from datetime import datetime

    def should_process_now(priority='normal'):
        hour = datetime.now().hour
        
        # Process high priority immediately
        if priority == 'high':
            return True
        
        # Queue normal requests during peak hours (9am-5pm)
        if 9 <= hour <= 17:
            return False
        
        return True
    ```
  </Accordion>
</AccordionGroup>

## Provider-Specific Tips

<Tabs>
  <Tab title="OpenRouter">
    **Rate Limit:** 20 req/min, 50 req/day (1000 with \$10 topup)

    **Optimization Strategies:**

    * Use the \$10 lifetime topup for 20x increase
    * Models share a common quota - choose wisely
    * Consider switching to Groq for high-volume workloads

    <Tip>OpenRouter's topup is one-time and never expires, making it excellent value.</Tip>
  </Tab>

  <Tab title="Mistral La Plateforme">
    **Rate Limit:** 1 req/sec, 500K tokens/min, 1B tokens/month per model

    **Optimization Strategies:**

    * Token limits are per-model - use multiple models
    * 1B tokens/month is \~33M tokens/day
    * Best for high-token workloads

    <Warning>Free tier requires opting into data training.</Warning>
  </Tab>

  <Tab title="Cerebras">
    **Rate Limit:** 30 req/min, 14,400 req/day, 1M tokens/day

    **Optimization Strategies:**

    * One of the highest daily request limits
    * Token limit shared across models
    * Great for production workloads
    * Consider for primary provider
  </Tab>

  <Tab title="Groq">
    **Rate Limit:** Varies by model (250-14,400 req/day)

    **Optimization Strategies:**

    * Use smaller models (Llama 3.1 8B) for 14,400 req/day
    * Compound models have highest token limits (70K/min)
    * Ultra-fast inference - ideal for real-time apps
    * Use model-specific limits strategically
  </Tab>
</Tabs>

## Next Steps

<CardGroup cols={2}>
  <Card title="Best Practices" icon="lightbulb" href="/guides/best-practices">
    Learn more optimization techniques and security practices
  </Card>

  <Card title="Provider Comparison" icon="scale-balanced" href="/guides/choosing-provider">
    Compare providers to find the best fit for your use case
  </Card>
</CardGroup>