Models

Available models via AWS Bedrock, OpenAI-compatible API, latency benchmarks, and versioning policy.

All models run on AWS Bedrock via a private PrivateLink endpoint inside your VPC. Capped AI exposes an OpenAI-compatible API — you can switch to Capped AI with two lines of code in most stacks.

Available Models (Phase 1)

Provider	Models	Best for
Anthropic	Claude Haiku 3.5, Claude Sonnet 3.5, Claude Sonnet 4	Reasoning, analysis, code, long context
Meta	Llama 3.3 70B, Llama 3.1 405B	Open-weights, lower token cost, fine-tuning candidate
Amazon	Titan Text Premier, Titan Embeddings	Embeddings, RAG, cost-sensitive completions
Mistral	Mixtral 8×7B, Mistral Large	European data residency preference, multilingual

Self-hosted open-weights models on EKS + vLLM are available for clients needing lower per-token cost at high volume. We do not host OpenAI GPT models directly — Azure OpenAI is available as a follow-on engagement.

OpenAI Compatibility

The endpoint spec matches OpenAI's /v1/chat/completions and /v1/embeddings paths. If your application uses the OpenAI SDK, you change two lines:

# Python
from openai import OpenAI

client = OpenAI(
    api_key="your-lomao-key",
    base_url="https://api.{your-account}.capped.ai/v1",
)

# TypeScript / Node.js
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LOMAO_API_KEY,
  baseURL: "https://api.{your-account}.capped.ai/v1",
});

Model names in requests map to Bedrock model IDs on the server side. Common aliases are accepted:claude-sonnet-4, claude-haiku-3.5, llama-3-70b, mixtral-8x7b.

Streaming

Server-Sent Events (SSE) streaming is supported on all /v1/chat/completions endpoints. Set stream: true in your request — the response will be a standard OpenAI-compatible SSE stream.

Latency

Measured on us-east-1 → Bedrock for a 1,000-token input / 500-token output request:

Model	P50	P95	P99
Claude Sonnet 4	~1.2s	~2.8s	~5s
Claude Haiku 3.5	~0.5s	~1.2s	~2.5s
Llama 3.3 70B	~0.8s	~2.0s	~4s
Mixtral 8×7B	~0.7s	~1.8s	~3.5s

Provisioned throughput (Pro and Enterprise tiers) cuts P95 to ~1.5s for steady-state traffic on Claude Sonnet 4. EU regions add ~100ms if your client applications are US-based.

Fine-tuning

Fine-tuning is available as a separate engagement ($50,000 initial + $5,000/month retainer) for open-weights models (Llama family) hosted in your private environment. We do not fine-tune Claude or Mistral — those model families do not support customer fine-tuning under current terms.

For most clients, retrieval-augmented generation (RAG) on a base Claude or Llama is strictly better than fine-tuning, and we can help you architect that without a separate engagement.

Model Versioning & Deprecations

You pin a specific model version in your requests. When a new version becomes available in Bedrock, Capped AI notifies you 30 days ahead of the model's eventual deprecation date. You decide whether to upgrade immediately, after testing, or at deprecation. We never silently change the model behind your requests.

Bring Your Own Model

Yes — fine-tuned models hosted on SageMaker or on a dedicated EC2 inference endpoint within your account can be integrated into the Capped AI router. Available as an add-on engagement. Contact hello@capped.ai to scope it.

Back to Docs