Groq Rate LimitsSAFE

Model-agnostic agent-skills platform with a harness-free canonical layer, verified adapters, and the ccpi package manager. Explore at tonsofskills.com.

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: groq-rate-limits
description: 'Implement Groq rate limit handling with backoff, queuing, and header
  parsing.

  Use when handling rate limit errors, implementing retry logic,

  or optimizing API request throughput for Groq.

  Trigger with phrases like "groq rate limit", "groq throttling",

  "groq 429", "groq retry", "groq backoff".

  '
allowed-tools: Read, Write, Edit
version: 1.11.0
license: MIT
author: Jeremy Longshore <[email protected]>
tags:
- saas
- groq
- api
compatibility: Designed for Claude Code
---
# Groq Rate Limits

## Overview

Handle Groq rate limits using the `retry-after` header, exponential backoff, and request queuing. Groq enforces limits at the organization level with both RPM (requests/minute) and TPM (tokens/minute) constraints -- hitting either one triggers a `429`.

The workflow builds up in five composable layers: parse the rate-limit headers, wrap calls in retry-with-backoff, gate concurrency through a queue, monitor remaining capacity proactively, and fall back across models when one pool is exhausted. Read `SKILL.md` for the high-level flow, then drill into [the full implementation](references/implementation.md) for every code block and [the reference tables + worked examples](references/reference.md) for header definitions and composed clients.

## Prerequisites

- A Groq API key (`GROQ_API_KEY`) — get one at [console.groq.com](https://console.groq.com).
- `groq-sdk` installed: `npm install groq-sdk`.
- For queuing (Step 3): `p-queue` installed: `npm install p-queue`.
- Node.js 18+ (for native `fetch` and the SDK).
- Know your plan's limits — check [console.groq.com/settings/limits](https://console.groq.com/settings/limits).

## Rate Limits at a Glance

Groq applies **RPM, RPD, TPM, and TPD** limits simultaneously — you must stay under every one, and either RPM or TPM can trip a `429`. Every response (even a success) carries `x-ratelimit-*` headers describing remaining capacity and reset timing; `429` responses add a `retry-after` header. Full header and constraint tables: [reference.md](references/reference.md).

## Instructions

Compose these five steps into one client wrapper (queue → monitor → retry). Each step's complete, copy-pasteable code is in [implementation.md](references/implementation.md).

### Step 1: Parse Rate Limit Headers

Read the `x-ratelimit-*` headers off every response into a typed `RateLimitInfo` so downstream logic can reason about remaining capacity. Groq reports reset times as strings like `"1.2s"` or `"120ms"` — normalize them to milliseconds.

### Step 2: Exponential Backoff with Retry-After

Wrap each API call in a retry loop. Prefer Groq's `retry-after` header when present; otherwise back off exponentially with jitter, capped at `maxDelayMs`. Retry only `429` and `5xx` — other `4xx` errors are not retryable.

### Step 3: Request Queue with Concurrency Control

Gate all requests through a `p-queue` sized to your plan's RPM (`intervalCap` over a 60s `interval`) so bursts never exceed the limit in the first place.

### Step 4: Proactive Rate Limit Monitor

Track remaining requests/tokens from response headers and pause *before* hitting zero (`shouldThrottle()` → `waitIfNeeded()`), instead of reacting to `429`s after the fact.

### Step 5: Model-Aware Rate Limit Strategy

Different models draw from different limit pools. When the preferred model is throttled, fall back to another model to keep making progress without waiting for a reset.

## Output

Applying this skill produces:

- A **`withRateLimitRetry()`** wrapper that transparently retries `429`/`5xx` with `retry-after`-aware backoff.
- A **`RateLimitMonitor`** that surfaces live status (`getStatus()` → `"Requests: N remaining | Tokens: M remaining"`) and throttles proactively.
- A **`p-queue`-backed client** that caps throughput to your RPM so 100 fan-out calls complete without tripping the limit.
- Console diagnostics on every backoff/throttle event (e.g. `Rate limited (attempt 2/5). Waiting 1.4s...`).

The observable end state: sustained request volume that stays under RPM/TPM with zero unhandled `429`s.

## Error Handling

| Scenario | Symptom | Solution |
|----------|---------|----------|
| Burst of requests | Many 429s in quick succession | Use queue with `p-queue` interval limiting (Step 3) |
| Large prompts burn TPM | 429 on tokens, not requests | Reduce `max_tokens`, compress prompts |
| Free tier too restrictive | Constant 429s | Upgrade to Developer plan at console.groq.com |
| Multiple services sharing key | Cascading 429s | Use separate API keys per service |
| `retry-after` absent on 429 | Retries hammer too fast | Fall back to exponential backoff + jitter (Step 2) |

## Examples

Start from this minimal `429` handler, then graduate to the composed client (queue + monitor + retry) in [reference.md](references/reference.md):

```typescript
try {
  await groq.chat.completions.create({ model, messages });
} catch (err) {
  if (err instanceof Groq.APIError && err.status === 429) {
    const retryAfter = parseInt(err.headers?.["retry-after"] || "0");
    console.log(`Rate limited. retry-after says wait ${retryAfter}s.`);
    // -> feed retryAfter into withRateLimitRetry (Step 2)
  }
}
```

- **Full five-step implementation** (every code block, verbatim): [implementation.md](references/implementation.md)
- **Composed client + header/limit tables + a 100-request fan-out**: [reference.md](references/reference.md)

## Resources

- [Groq Rate Limits Documentation](https://console.groq.com/docs/rate-limits)
- [Groq Pricing / Plans](https://groq.com/pricing)
- [p-queue on npm](https://www.npmjs.com/package/p-queue)
- Full implementation: [references/implementation.md](references/implementation.md)
- Reference tables & worked examples: [references/reference.md](references/reference.md)

## Next Steps

For security configuration, see the `groq-security-basics` skill in this pack, which covers API key storage, rotation, and request signing to complement 

SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (0)

No findings outside the package's declared scope.

Gates applied: no_behavioural_pass.