Skip to content

feat(retry): add retry_policies.rate_limit() convenience policy#2995

Open
RhythrosaLabs wants to merge 1 commit intoopenai:mainfrom
RhythrosaLabs:feat/retry-policies-rate-limit
Open

feat(retry): add retry_policies.rate_limit() convenience policy#2995
RhythrosaLabs wants to merge 1 commit intoopenai:mainfrom
RhythrosaLabs:feat/retry-policies-rate-limit

Conversation

@RhythrosaLabs
Copy link
Copy Markdown

Summary

Fixes #782.

Adds retry_policies.rate_limit() — a ready-made helper that retries on HTTP 429 responses only, honouring Retry-After / Retry-After-Ms headers when present and falling back to the configured backoff schedule when absent.


Motivation

Issue #782 reports that there is no concise way to express "retry rate-limit errors, wait the prescribed cool-down". The existing primitives require users to manually compose:

retry_policies.any(
    retry_policies.http_status([429]),
    retry_policies.retry_after(),
)

This is both verbose and subtly wrong:

  • http_status([429]) fires on 429 but ignores the Retry-After header entirely.
  • retry_after() honours the header but triggers on any error that carries one, not just 429s.

Neither alone captures the intended contract. Users need domain knowledge of the internals to compose them correctly.


What this PR adds

# After
from agents import ModelRetrySettings, ModelSettings, retry_policies

ModelSettings(
    retry=ModelRetrySettings(
        max_retries=4,
        policy=retry_policies.rate_limit(),
    )
)

The policy:

  1. Returns RetryDecision(retry=False) for anything other than HTTP 429.
  2. Returns RetryDecision(retry=True, delay=<seconds>) when a Retry-After header was parsed (via context.normalized.retry_after) or surfaced by provider advice — so the runner waits the prescribed cool-down.
  3. Returns RetryDecision(retry=True, delay=None) when no header is present — the runner falls back to the configured backoff schedule.

Can be combined with provider_suggested() for deeper provider integration:

retry_policies.any(retry_policies.rate_limit(), retry_policies.provider_suggested())

Changes

File Change
src/agents/retry.py New rate_limit() method on _RetryPolicies
tests/test_model_retry.py 8 new tests (6 unit-level policy tests + 2 integration tests via get_response_with_retry)
examples/basic/rate_limit_retry.py New minimal example
docs/models/index.md New row in the ready-made helpers table

Tests

pytest tests/test_model_retry.py -q
60 passed in 0.36s

New tests cover:

  • Retries on 429 with no header (delay=None)
  • Uses normalized.retry_after delay when header is present
  • Falls back to provider_advice.retry_after when normalized value is absent
  • normalized.retry_after takes precedence over provider_advice.retry_after
  • Does not retry on non-429 status codes (e.g. 500)
  • Does not retry on network errors
  • Integration: full get_response_with_retry run with 429+Retry-After: 2 header → sleep(2.0) called
  • Integration: full run with 429, no header → falls back to configured backoff delay

Checklist

  • Implementation follows existing code style and uses _mark_retry_capabilities
  • All existing tests still pass
  • New tests added for all behaviour branches
  • Docs table updated
  • Example added

Closes openai#782.

Add a ready-made `rate_limit()` helper to `_RetryPolicies` that retries
on HTTP 429 responses only, honouring `Retry-After` / `Retry-After-Ms`
headers when present and falling back to the configured backoff schedule
when they are absent.

Before this change users had to compose policies manually:

    retry_policies.any(
        retry_policies.http_status([429]),
        retry_policies.retry_after(),
    )

This combination has subtle semantics: `http_status([429])` ignores the
header while `retry_after()` fires on any error that carries a header, not
just 429s.  Neither alone captures the intended "retry 429s, wait the
prescribed cool-down" contract.

After this change:

    retry_policies.rate_limit()

The implementation checks `context.normalized.status_code == 429`, then
reads the delay from `context.normalized.retry_after` (header parsed by
the runner) and falls back to `context.provider_advice.retry_after` when
the normalised value is absent.  This ensures the Retry-After header is
consumed only for the specific error it accompanies.

Changes
-------
* src/agents/retry.py   – new `rate_limit()` method on `_RetryPolicies`
* tests/test_model_retry.py – 8 new tests (6 unit + 2 integration)
* examples/basic/rate_limit_retry.py – minimal example (new file)
* docs/models/index.md  – new row in the ready-made helpers table
@github-actions github-actions Bot added documentation Improvements or additions to documentation enhancement New feature or request feature:core labels Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request feature:core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rate Limit Support

1 participant