AI for your apps

Berth can give any app an LLM: small models running on the Mac itself, cloud models with keys you add once, or both. Apps reach all of them through one gateway in Berth using the OpenAI or Anthropic SDK they'd use anyway, and they never see your provider keys.

Set it up once

berth ai:setup

This installs Ollama with Homebrew and starts it with the Mac (it asks first), then downloads a model that fits: qwen3:8b with 16 GB of memory or more, qwen3:4b below that. Pick another with --model gemma3:4b, or skip local models with --cloud-only. Local models run on the Mac's GPU, outside the app containers, which can't use it.

Add whichever cloud providers you want (the key prompt is hidden):

berth ai:key anthropic          # also openai, gemini, openrouter, groq, mistral
berth ai:key together --url https://api.together.xyz/v1    # anything OpenAI-compatible

Give an app access

berth ai:enable myapp

The app gets its own token and the usual variables, and redeploys:

The official SDKs read those by themselves, so there's nothing Berth-specific in your code:

import OpenAI from "openai";
const client = new OpenAI();   // reads OPENAI_BASE_URL and OPENAI_API_KEY
const reply = await client.chat.completions.create({
  model: "default",
  messages: [{ role: "user", content: "Hello!" }],
});

Python's openai and anthropic packages work the same way. Every provider speaks the OpenAI format (/v1/chat/completions, /v1/embeddings, /v1/models), and Claude models also speak Anthropic's own API, for tools, prompt caching and extended thinking.

Choosing models

Apps ask for a model by name, like local/qwen3:8b or anthropic/claude-sonnet-5-5, or by an alias you set up once for every app:

berth ai:alias smart anthropic/claude-sonnet-5-5
berth ai:alias fast  groq/llama-3.3-70b-versatile
berth ai:models                  # aliases, local models, providers
berth ai:pull nomic-embed-text   # another local model, e.g. for embeddings

ai:setup points default at the local model. I'd write apps against aliases ("default", "smart") rather than model names: change what an alias points at and every app picks it up immediately, no redeploy.

Limits and usage

berth ai:enable myapp --models default,smart --monthly-tokens 2000000 --rpm 60
berth ai:usage                   # requests and tokens per app and model this month
berth ai:status                  # what's running, what's loaded, each app's usage
berth ai:disable myapp           # revoke its token

--models takes aliases, provider/model or provider/*. An app over a limit gets a 429 that says which limit and how to raise it. Usage records tokens and timings, never prompts or replies.

Dev versions of an AI app get their own token that only reaches local models, so an agent testing changes can't run up a cloud bill (berth ai:enable myapp-dev --models 'local/*,smart' lifts that).

Memory

A loaded local model takes real memory: about 5 GB for an 8B model, 2.5 GB for a 4B one. Ollama lets go of it after 5 idle minutes. berth status shows what's loaded, counts it, and warns when apps, databases and the model together get close to the Mac's RAM. On an 8 GB Mac, use a cloud provider (or a really small model).

The gateway listens on port 7790 on the containers' network, and only containers on this Mac can reach it. To change the port, set ai_port in ~/.berth/config.json and run berth reload. To use Ollama on another machine, set ollama_url in ~/.berth/ai/config.json.