inline::builtin
Description
Implements the Anthropic Messages API by delegating to the inference API's anthropic_messages() method. OpenAIMixin provides default translation via openai_chat_completion. Providers with native /v1/messages support (e.g., Anthropic, Meta AI, Ollama, vLLM, Fireworks, DeepSeek) use AnthropicMixin to forward directly to the provider's own /v1/messages endpoint. Message batch operations are implemented locally.
Configuration
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
kvstore | KVStoreReference | No | Configuration for the key-value store backend used by message batches. | |
kvstore.namespace | str | No | Key prefix for KVStore backends | |
kvstore.backend | str | No | Name of backend from storage.backends | |
max_concurrent_batches | int | No | 1 | Maximum number of concurrent message batches to process simultaneously. |
max_concurrent_requests_per_batch | int | No | 10 | Maximum number of concurrent requests to process per batch. |
Sample Configuration
kvstore:
namespace: message_batches
backend: kv_default