Advanced Features
Take full control of your HTTP requests with powerful advanced features — cancellation, real concurrency without async, true streaming, caching, and more
Interceptors & Hooks
Interceptors are functions that can modify requests before they are sent or modify responses before they are returned to the caller. This is useful for adding authentication headers, logging, retry logic, error handling, or transforming data globally.
Request interceptors run in registration order; so do response interceptors. Remove one with the index returned by use(). Interceptors can be regular functions or async def -- the async client awaits async interceptors, and the sync client can run them too via a small internal event loop. For lighter-weight instrumentation, pass onRequestStart/onRetry/onRedirect directly on a request instead of registering a full interceptor.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
| fn | Callable | Required | Function receiving config (request) or response (response), returning the (possibly modified) value |
Returns
int — Index of the added interceptor (usable with eject())Example
# Request interceptor -- adds auth token to every request
def auth_interceptor(config):
config.headers['Authorization'] = 'Bearer my-secret-token'
return config
# Response interceptor -- logs status of every response
def log_interceptor(response):
print(f"{response.config.method} {response.config.url} -> {response.status}")
return response
client.interceptors.request.use(auth_interceptor)
index = client.interceptors.response.use(log_interceptor)
# Remove an interceptor by index
client.interceptors.response.eject(index)
# Lightweight observability hooks, no interceptor needed:
client.get(
'/report',
onRequestStart=lambda config: print(f"starting {config.url}"),
onRetry=lambda config, attempt, error: print(f"retry #{attempt}: {error}"),
onRedirect=lambda config, response: print(f"redirected: {response.status}"),
)Cancellation
Axios/fetch-style request cancellation via AbortController. Create a controller, pass its .signal into a request, and call controller.abort() from anywhere -- another thread, a UI "Cancel" button, a signal handler -- to stop it. Works for any request in the library: sync or async, uploads or downloads, with or without retries.
Interrupting a request that's genuinely blocked on a socket read requires force-closing the underlying connection from another thread -- there's no way to "just return early" from a blocked system call. AtomHTTP registers the live urllib3 connection with the signal for the duration of the request; calling abort() force-closes that socket, making the blocked read raise immediately. Between chunks (uploading or downloading), the adapter also proactively checks whether the signal has been aborted. One honest limitation: there's a brief window during initial DNS resolution/TCP connect (before a socket exists to close) where abort() takes effect at the next check point rather than instantaneously -- in practice this window is small.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
| signal | AbortSignal | None | Pass to any request via the signal= kwarg to make it cancellable |
Returns
None — raises AtomHTTPCancelError (code ERR_CANCELED) when abortedExample
from atomhttp import AtomHTTP, AbortController
from atomhttp.errors import AtomHTTPCancelError
client = AtomHTTP(base_url="https://api.example.com", timeout=30)
controller = AbortController()
# From another thread, UI callback, etc:
# controller.abort("user clicked cancel")
try:
response = client.get("/slow-report", signal=controller.signal)
except AtomHTTPCancelError as e:
print("request was cancelled:", e.message)
# One controller can cancel multiple in-flight requests at once.
# Works identically with AsyncAtomHTTP:
async def run():
try:
await async_client.get("/slow", signal=controller.signal)
except AtomHTTPCancelError:
print("cancelled")Multithreading & Concurrency
Every AtomHTTP client owns a persistent thread pool (max_workers=10 by default, configurable). Because urllib3's connection pools are thread-safe, this gives you real concurrency for I/O-bound batches of requests without needing async/await anywhere.
.all() reuses the client's persistent pool instead of spinning up a new ThreadPoolExecutor per call, avoiding thread-creation overhead. submit() gives you a plain concurrent.futures.Future for fire-and-forget workflows. map() runs the same request against many URLs and returns results in input order regardless of completion order. The async client uses asyncio.gather() for the equivalent behaviour.
Methods
| Method | Description |
|---|---|
| client.all(calls, max_workers=None) | Run request thunks concurrently, returns List[Response] in order |
| client.submit(method, url, **kwargs) | Fire off one request on the thread pool, returns a Future[Response] |
| client.map(method, urls, **kwargs) | Run the same request against many URLs concurrently, returns List[Response] in order |
| async_client.all(coros) | asyncio.gather shortcut for the async client |
Returns
List[Response] (all/map) — concurrent.futures.Future[Response] (submit)Example
client = AtomHTTP(base_url="https://api.example.com", max_workers=20)
# Run a batch concurrently:
responses = client.all([
lambda: client.get("/a"),
lambda: client.get("/b"),
lambda: client.get("/c"),
])
# Fire-and-forget:
future = client.submit("GET", "/report")
# ... do other work ...
response = future.result()
# Same request, many URLs, results in input order:
responses = client.map("GET", ["/users/1", "/users/2", "/users/3"])
# Async equivalent:
responses = await async_client.all([
async_client.get("/a"),
async_client.get("/b"),
])Streaming, Download & Pagination
client.stream() returns a response whose body hasn't been read yet, for processing large downloads incrementally instead of loading them fully into memory. client.download() streams straight to a file on disk. client.paginate() walks a paginated REST endpoint as a generator.
Always use stream() as a context manager (sync with / async async with) so the connection is released even if you stop reading partway through. download()'s async version runs the file I/O in the thread pool so disk writes don't block the event loop. paginate() defaults to a ?page=N query parameter and stops as soon as a page comes back with no items; pass extract_items/has_next for custom pagination schemes (cursor-based APIs, etc).
Methods
| Method | Description |
|---|---|
| client.stream(method, url, **kwargs) | Returns a StreamResponse with .iter_bytes()/.iter_lines() |
| client.download(url, path, onDownloadProgress=None, **kwargs) | Streams response body straight to disk |
| client.paginate(url, page_param='page', extract_items=None, has_next=None, **kwargs) | Generator yielding one page's items at a time |
Example
# Streaming a large response body
with client.stream("GET", "/export.csv") as response:
for line in response.iter_lines():
process(line)
# Downloading straight to disk, with progress
client.download(
"/video.mp4", "video.mp4",
onDownloadProgress=lambda loaded, total: print(f"{loaded}/{total}"),
)
# Pagination -- default ?page=N scheme
for items in client.paginate("/users"):
for user in items:
process(user)
# Pagination -- custom cursor-style API
for items in client.paginate(
"/items",
extract_items=lambda r: r.data["results"],
has_next=lambda r: r.data.get("next") is not None,
):
...
# Async versions are identical, with async with / async for:
async with await async_client.stream("GET", "/export.csv") as response:
async for line in response.iter_lines():
process(line)
async for items in async_client.paginate("/users"):
...FormData & File Uploads
Send multipart/form-data requests including both text fields and file uploads. This follows the browser FormData API, making it easy to construct complex form submissions. Large files stream automatically -- a multi-gigabyte upload never needs to fit in memory.
Files can be provided as bytes, file objects, or pathlib.Path objects. The content-type is automatically detected from file extensions via the standard mimetypes module. Multiple values can be appended to the same field name. When any field is a file, AtomHTTP switches to a generator-based streaming multipart encoder automatically -- verified with tracemalloc to use a small, fixed amount of memory regardless of file size.
Methods
| Method | Description |
|---|---|
| append(name, value, filename, content_type) | Adds a new value to the form data |
| set(name, value, filename, content_type) | Replaces all existing values for a field |
| delete(name) | Removes all values for a field |
| get(name) | Returns the first value for a field |
| get_all(name) | Returns all values for a field as a list |
| has(name) | Checks if a field exists |
| keys() | Returns all field names |
| items() | Returns all (name, value) pairs |
| to_multipart() | Builds the full body in memory -- fine for small forms |
| to_multipart_stream(chunk_size=65536) | Returns (generator, boundary, total_size) for streaming uploads |
Example
from atomhttp import FormData
from pathlib import Path
form = FormData()
form.append('username', 'johndoe')
form.append('email', 'john@example.com')
form.append('avatar', open('profile.jpg', 'rb'), filename='profile.jpg')
form.append('document', Path('resume.pdf'), filename='resume.pdf')
# Supports multiple values for the same field
form.append('tags', 'python')
form.append('tags', 'http')
form.append('tags', 'async')
response = client.post('https://api.example.com/upload', data=form)
# Large file, with real upload progress (streams automatically):
form2 = FormData()
form2.append('video', Path('large-video.mp4'), filename='large-video.mp4')
client.post(
'/upload', data=form2,
onUploadProgress=lambda loaded, total: print(f"{loaded}/{total} bytes"),
)Caching
atomhttp.cache.CacheInterceptor implements conditional GET requests using ETag/If-None-Match and Last-Modified/If-Modified-Since -- the same mechanism browsers use.
After a 200 response with an ETag or Last-Modified header comes back, its body is cached in memory (or in a store you provide). The next matching request sends back the cached validator; if the server replies 304 Not Modified, the cached body is served back instead of re-parsing a full response. This still makes a network round trip on every call -- it saves bandwidth and re-parsing cost, not the round trip itself. True zero-network-call caching within a max-age window isn't supported by the interceptor pipeline by design; wrap the client call yourself with your own TTL check if you need that.
Classes
| Class | Description | Methods |
|---|---|---|
| CacheInterceptor(store=None) | Holds cached entries and the interceptor functions | on_request(config), on_response(response), clear() |
Example
from atomhttp import AtomHTTP
from atomhttp.cache import CacheInterceptor
client = AtomHTTP(base_url="https://api.example.com")
cache = CacheInterceptor()
client.interceptors.request.use(cache.on_request)
client.interceptors.response.use(cache.on_response)
client.get("/users/1") # normal request, response cached (has ETag)
client.get("/users/1") # sends If-None-Match; 304 -> served from cache
cache.clear() # drop everythingAuthentication
Built-in support for standard authentication mechanisms including HTTP Basic Auth (applied automatically via auth=) and Bearer Token authentication. Helper classes are also available for building headers by hand.
Prefer RequestConfig(auth={...}) for plain HTTP Basic Auth -- the client applies it automatically. BasicAuth/BearerAuth are small convenience helpers for building Authorization headers for custom interceptors or non-standard auth flows.
Classes
| Class | Description | Methods |
|---|---|---|
| BasicAuth(username, password) | Creates Basic Authentication handler | get_header() returns {"Authorization": "Basic base64..."} |
| BearerAuth(token) | Creates Bearer Token handler | get_header() returns {"Authorization": "Bearer token"} |
Example
# Preferred: built-in Basic Auth, applied automatically
response = client.get(
'https://httpbin.org/basic-auth/username/password',
auth={'username': 'username', 'password': 'password'},
)
# Helper classes for custom flows / interceptors
from atomhttp.auth import BasicAuth, BearerAuth
basic = BasicAuth('username', 'password')
response = client.get('https://httpbin.org/basic-auth/username/password',
headers=basic.get_header())
bearer = BearerAuth('your-jwt-token-here')
response = client.get('https://api.example.com/protected',
headers=bearer.get_header())
# Complex auth flows via interceptors
def oauth_interceptor(config):
config.headers['Authorization'] = f'Bearer {get_fresh_token()}'
return config
client.interceptors.request.use(oauth_interceptor)Proxies, TLS & Unix Sockets
Configure proxies (including SOCKS), TLS verification and mTLS client certificates, and Unix domain socket connections. Brotli response decoding is available as an optional extra.
If proxy= isn't given, HTTP_PROXY/HTTPS_PROXY/NO_PROXY environment variables are honored automatically. verify accepts True/False or a path to a custom CA bundle. cert accepts a single file path or a (cert, key) tuple for mTLS. socketPath connects over AF_UNIX instead of TCP -- useful for talking to services like the Docker daemon.
Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
| proxy | dict | None | {"host": "http://proxy:8080"} or socks5://... (needs atomhttp[socks]) |
| verify | bool | str | True | TLS verification on/off, or a custom CA bundle path |
| cert | str | tuple | None | mTLS client certificate: path, or (cert_path, key_path) |
| socketPath | str | None | Unix domain socket path (alternative to TCP/IP) |
| decompress | bool | True | Auto-decode gzip/deflate/brotli (brotli needs atomhttp[brotli]) |
Example
# Proxy (explicit, or read from HTTP_PROXY/HTTPS_PROXY automatically)
client.get("/data", proxy={"host": "http://proxy.example.com:8080"})
client.get("/data", proxy={"host": "http://proxy:8080", "auth": {"username": "u", "password": "p"}})
# SOCKS proxy (pip install atomhttp[socks])
client.get("/data", proxy={"host": "socks5://127.0.0.1:1080"})
# TLS / mTLS
client.get("/secure", verify="/path/to/custom-ca.pem")
client.get("/secure", cert=("/path/to/cert.pem", "/path/to/key.pem"))
# Unix domain socket -- e.g. the Docker daemon
client.get("http://localhost/containers/json", socketPath="/var/run/docker.sock")
# Brotli (pip install atomhttp[brotli]) -- decoded transparently, no code changesRetries & Backoff
Configure automatic retries for transient network/server failures. Retry-After response headers are honored automatically. Redirects are always followed independently of retryConfig, up to maxRedirects.
If retryConfig isn't set, no error-based retries happen (only redirects, up to maxRedirects). Retries apply exponential backoff (backoff_factor * 2^attempt) and only retry on the configured status_forcelist codes plus connection/read errors. Use onRetry to observe each attempt.
Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
| retryConfig.max_retries | int | 3 | Maximum retry attempts for connect/read/status failures |
| retryConfig.backoff_factor | float | 0.3 | Exponential backoff multiplier between attempts |
| retryConfig.status_forcelist | list[int] | [408,429,500,502,503,504] | Status codes that trigger a retry |
| maxRedirects | int | 5 | Independent of retryConfig -- always applies |
Example
response = client.get(
"/flaky-endpoint",
retryConfig={
"max_retries": 5,
"backoff_factor": 0.3,
"status_forcelist": [408, 429, 500, 502, 503, 504],
},
onRetry=lambda config, attempt, error: print(f"retry #{attempt}: {error}"),
)
# A server-sent "Retry-After" header is respected automatically,
# taking priority over the exponential backoff calculation.