Rate limits

Praxis rate limiting has two layers: the platform layer caps requests per second and concurrent requests per Key; the model layer caps requests per minute (RPM) and tokens per minute (TPM) per model. Every response — success or 429 — carries three x-ratelimit-* headers stating your current allowance and when it resets, so you never have to guess or spend a request to check.

The two layers

Both layers return the same three rate limit headers; which layer tripped is stated in the error body (Errors).

Platform layer: requests per second and concurrency

By default, each API key allows 5 requests per second and 4 concurrent in-flight requests. Streams occupy a concurrency slot until they finish. Exceeding either limit returns 429 with ratelimit-limit, ratelimit-remaining, ratelimit-reset and retry-after; when concurrency is full, let an existing request finish.

Configured limits may differ; use the actual response headers and concurrency error details.

Three response headers

x-ratelimit-limit-requests: 5
x-ratelimit-remaining-requests: 4
x-ratelimit-reset-requests: 850ms

The same three numbers are also available under the IETF names RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset (whole seconds). The default is 5 requests/second and 4 in flight; response headers reflect configured request-rate overrides.

Exceeding the per-second limit

The (N+1)th request within one second — N being the Key's per-second ceiling — is answered 429 with a retry-after header (whole seconds, usually 1). Rejected requests do not consume the allowance: x-ratelimit-remaining-requests stays 0, and the window simply resets.

Exceeding the concurrency limit

When the number of in-flight requests — including unfinished streaming responses, which hold their slot until the stream ends — exceeds the Key's concurrency ceiling, extra requests get 429 with retry-after: 1. Resetting the per-second window does not free a concurrency slot: an in-flight request must finish first. Which limit tripped is spelled out in the error message ("Concurrent request limit …" / "Requests per second limit …").

Error body

The error body follows the first path segment (see Errors). Native endpoints such as /model/v1/chat/completions return Problem Details:

{
  "type": "https://deepraxis.ai/problems/rate_limit",
  "title": "rate_limited",
  "status": 429,
  "detail": "Requests per second limit of 5 exceeded for this credential. Retry after the listed wait.",
  "instance": "problem_eqfnxu",
  "kind": "rate_limit",
  "scope": "platform",
  "retryable": true
}

scope: "platform" means the platform layer triggered it. Vendor-prefixed endpoints speak that vendor's own error format — for the OpenAI-compatible endpoints:

{ "error": { "code": 429, "type": "rate_limit_error", "message": "Requests per second limit of 5 exceeded for this credential. Retry after the listed wait." } }

Requests without a credential

Public endpoints (such as GET /model/v1/models) can be called without a Key; those are counted per source IP against a smaller anonymous allowance, with the same response headers. Sending a Key switches accounting to the Key, regardless of IP.

GET /meta/v1/health is never rate limited — point uptime monitors at it freely.

Model layer: requests and tokens per minute

The model layer counts per model × tenant × minute, not per API Key:

• Each model carries its own RPM and TPM in its model metadata; unset or 0 means unlimited.
• Hitting either limit returns 429 with type rate_limit_error, Retry-After: 60, and problem.details.rateLimit carrying the current value and the cap.
• BYOK calls bring their own provider credentials and do not count against platform model limits.
• Upstream provider rate limits are handled by automatic failover to other providers; when none remain, the upstream 429 is returned.

Models also carry a monthly budget based on cumulative revenue for the month; exceeding it returns 429 with type budget_error (QUOTA_EXHAUSTED) and no Retry-After — waiting does not help; the model's budget must be raised.

For status codes and error bodies see Errors; for account quotas see Quotas & rate limits.