LiteLLM + Aethon

LiteLLM provides a unified interface for 100+ LLM providers. Since Aethon is OpenAI-compatible, you can route requests through LiteLLM using the openai/ prefix with a custom API base.

Install

bash

pip install litellm

Basic Usage

python

import litellm

response = litellm.completion(
    model="openai/llama3.1:8b",
    api_base="https://api.aethon.ai/v1",
    api_key="your-aethon-key",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(response.choices[0].message.content)

Streaming

python

import litellm

response = litellm.completion(
    model="openai/llama3.1:8b",
    api_base="https://api.aethon.ai/v1",
    api_key="your-aethon-key",
    messages=[{"role": "user", "content": "Write a poem about GPUs"}],
    stream=True,
)

for chunk in response:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

LiteLLM Proxy Config

If you're running the LiteLLM proxy server, add Aethon to your config.yaml:

yaml

model_list:
  - model_name: llama-8b
    litellm_params:
      model: openai/llama3.1:8b
      api_base: https://api.aethon.ai/v1
      api_key: your-aethon-key

  - model_name: mistral-7b
    litellm_params:
      model: openai/mistral:7b
      api_base: https://api.aethon.ai/v1
      api_key: your-aethon-key

  - model_name: qwen-7b
    litellm_params:
      model: openai/qwen2.5:7b
      api_base: https://api.aethon.ai/v1
      api_key: your-aethon-key

Then start the proxy: litellm --config config.yaml

Environment Variables

You can also set the API base and key as environment variables:

bash

export OPENAI_API_BASE=https://api.aethon.ai/v1
export OPENAI_API_KEY=your-aethon-key

python

import litellm

# No need to pass api_base/api_key — reads from env
response = litellm.completion(
    model="openai/llama3.1:8b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

Prefix with openai/ — this tells LiteLLM to use its OpenAI-compatible provider, which works with Aethon's API. Model name goes after the prefix: openai/llama3.1:8b.

LangChain Guide →LlamaIndex Guide →Open WebUI Guide →