External APIs

Llama Stack supports external APIs that live outside of the main codebase. This allows you to:

• Create and maintain your own APIs independently
• Share APIs with others without contributing to the main codebase
• Keep API-specific code separate from the core Llama Stack code

Configuration

To enable external APIs, you need to configure the external_apis_dir in your Llama Stack configuration. This directory should contain your external API specifications:

external_apis_dir: ~/.llama/apis.d/

Directory Structure

The external APIs directory should follow this structure:

apis.d/
  custom_api1.yaml
  custom_api2.yaml

Each YAML file in these directories defines an API specification.

API Specification

Here's an example of an external API specification for a weather API:

module: weather
api_dependencies:
  - inference
protocol: WeatherAPI
name: weather
pip_packages:
  - llama-stack-api-weather

API Specification Fields

• module: Python module containing the API implementation
• protocol: Name of the protocol class for the API
• name: Name of the API
• pip_packages: List of pip packages to install the API, typically a single package

Required Implementation

External API modules must expose three things:

1. available_providers() - returns a list of provider specs
2. A Protocol class - defines the API contract
3. create_router() - returns a FastAPI router that maps HTTP endpoints to the protocol methods

# llama_stack_api_weather/weather.py
from typing import Protocol

from fastapi import APIRouter

from llama_stack_api import Api, ProviderSpec, RemoteProviderSpec


def available_providers() -> list[ProviderSpec]:
    return [
        RemoteProviderSpec(
            api=Api.weather,
            provider_type="remote::kaze",
            adapter_type="kaze",
            module="llama_stack_provider_kaze",
            pip_packages=["llama_stack_provider_kaze"],
            config_class="llama_stack_provider_kaze.KazeProviderConfig",
        ),
    ]


class WeatherProvider(Protocol):
    """A protocol for the Weather API."""

    async def get_available_locations() -> dict[str, list[str]]:
        """Get the available locations."""
        ...


def create_router(impl: WeatherProvider) -> APIRouter:
    router = APIRouter()

    @router.get("/v1/weather/locations")
    async def get_available_locations() -> dict[str, list[str]]:
        return await impl.get_available_locations()

    return router

The __init__.py must re-export these:

# llama_stack_api_weather/__init__.py
"""Weather API for Llama Stack."""

from .weather import WeatherProvider, available_providers, create_router

__all__ = ["WeatherProvider", "available_providers", "create_router"]

Example: Custom API

Using the weather API shown above, here's the full setup:

1. Create the API package:

mkdir -p llama-stack-api-weather/src/llama_stack_api_weather
cd llama-stack-api-weather
uv init

2. Edit pyproject.toml:

[project]
name = "llama-stack-api-weather"
version = "0.1.0"
description = "Weather API for Llama Stack"
requires-python = ">=3.12"
dependencies = ["llama-stack-api", "fastapi"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]
include = ["llama_stack_api_weather", "llama_stack_api_weather.*"]

3. Add the weather.py and __init__.py files shown in the Required Implementation section above.

4. Create the API specification:

# ~/.llama/apis.d/weather.yaml
module: llama_stack_api_weather
name: weather
pip_packages: ["llama-stack-api-weather"]
protocol: WeatherProvider

5. Install and configure:

uv pip install -e .

# run config
version: "2"
distro_name: "weather-demo"
apis:
  - weather
providers: {}
external_apis_dir: ~/.llama/apis.d

The API is now available at /v1/weather/locations.

Example: custom provider for the weather API

1. Create the provider package:

mkdir -p llama-stack-provider-kaze
cd llama-stack-provider-kaze
uv init

2. Edit pyproject.toml:

[project]
name = "llama-stack-provider-kaze"
version = "0.1.0"
description = "Kaze weather provider for Llama Stack"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["llama-stack", "pydantic", "aiohttp"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]
include = ["llama_stack_provider_kaze", "llama_stack_provider_kaze.*"]

3. Create the initial files:

touch src/llama_stack_provider_kaze/__init__.py
touch src/llama_stack_provider_kaze/kaze.py

4. Create the provider implementation:

Initialization function:

# llama-stack-provider-kaze/src/llama_stack_provider_kaze/__init__.py
"""Kaze weather provider for Llama Stack."""

from .config import KazeProviderConfig
from .kaze import WeatherKazeAdapter

__all__ = ["KazeProviderConfig", "WeatherKazeAdapter"]


async def get_adapter_impl(config: KazeProviderConfig, _deps):
    from .kaze import WeatherKazeAdapter

    impl = WeatherKazeAdapter(config)
    await impl.initialize()
    return impl

Configuration:

# llama-stack-provider-kaze/src/llama_stack_provider_kaze/config.py
from pydantic import BaseModel, Field


class KazeProviderConfig(BaseModel):
    """Configuration for the Kaze weather provider."""

    base_url: str = Field(
        "https://api.kaze.io/v1",
        description="Base URL for the Kaze weather API",
    )

Main implementation:

# llama-stack-provider-kaze/src/llama_stack_provider_kaze/kaze.py
from llama_stack_api_weather.api import WeatherProvider

from .config import KazeProviderConfig


class WeatherKazeAdapter(WeatherProvider):
    """Kaze weather provider implementation."""

    def __init__(
        self,
        config: KazeProviderConfig,
    ) -> None:
        self.config = config

    async def initialize(self) -> None:
        pass

    async def get_available_locations(self) -> dict[str, list[str]]:
        """Get available weather locations."""
        return {"locations": ["Paris", "Tokyo"]}

5. Create the provider specification:

# ~/.llama/providers.d/remote/weather/kaze.yaml
adapter_type: kaze
pip_packages: ["llama_stack_provider_kaze"]
config_class: llama_stack_provider_kaze.config.KazeProviderConfig
module: llama_stack_provider_kaze
optional_api_dependencies: []

6. Install the provider package:

uv pip install -e .

7. Configure Llama Stack to use the provider:

# ~/.llama/config.yaml
version: "2"
distro_name: "llama-stack-api-weather"
apis:
  - weather
providers:
  weather:
  - provider_id: kaze
    provider_type: remote::kaze
    config: {}
external_apis_dir: ~/.llama/apis.d
external_providers_dir: ~/.llama/providers.d
server:
  port: 8321

8. Run the server:

llama stack run ~/.llama/config.yaml

9. Test the API:

curl -sSf http://127.0.0.1:8321/v1/weather/locations
{"locations":["Paris","Tokyo"]}%

Best Practices

1. Package Naming: Use a clear and descriptive name for your API package.

2. Version Management: Keep your API package versioned and compatible with the Llama Stack version you're using.

3. Dependencies: Only include the minimum required dependencies in your API package.

4. Documentation: Include clear documentation in your API package about:

   • Installation requirements
   • Configuration options
   • API endpoints and usage
   • Any limitations or known issues

5. Testing: Include tests in your API package to ensure it works correctly with Llama Stack.

Troubleshooting

If your external API isn't being loaded:

1. Check that the external_apis_dir path is correct and accessible.
2. Verify that the YAML files are properly formatted.
3. Ensure all required Python packages are installed.
4. Check the Llama Stack server logs for any error messages - turn on debug logging to get more information using LLAMA_STACK_LOGGING=all=debug.
5. Verify that the API package is installed in your Python environment.