# Analysis and Embedding Models

MailSubsystem uses two independent model roles. Keep them separate when
configuring or changing providers.

## Model Roles

| Role | Environment variable | Used for | Change impact |
| --- | --- | --- | --- |
| Analysis | `ANALYSIS_MODEL=provider/model` | Classification, summaries, threat analysis, location reasoning, and other LLM work | Can be changed for future analysis work. Existing analysis results remain until re-run. |
| Embeddings | `EMBEDDING_MODEL=provider/model` | Vector search, retrieval context, and related-message lookup | Defines the vector space. Changing it requires reembedding stored mail. |

Use separate API keys when the providers or billing scopes differ:

```env
ANALYSIS_MODEL=gemini/gemini-2.0-flash
ANALYSIS_API_KEY=...

EMBEDDING_MODEL=gemini/gemini-embedding-001
EMBEDDING_API_KEY=...
```

Provider-specific keys such as `GEMINI_API_KEY`, `OPENAI_API_KEY`, and
`ANTHROPIC_API_KEY` are still accepted for compatibility, but the purpose-scoped
keys are clearer when analysis and embeddings use different providers.

## Model Reference Format

Model selectors use:

```text
provider/model-name
```

The provider side is normalized to lowercase. The model side is passed through
to the selected provider.

Examples:

```env
ANALYSIS_MODEL=openai/gpt-4o
ANALYSIS_MODEL=anthropic/claude-3-5-sonnet-20241022
ANALYSIS_MODEL=codex/your-codex-model
ANALYSIS_MODEL=lmstudio/local-analysis-model

EMBEDDING_MODEL=gemini/gemini-embedding-001
EMBEDDING_MODEL=openai/text-embedding-3-small
EMBEDDING_MODEL=ollama/nomic-embed-text
```

## Analysis Provider Options

| Provider ref | Required configuration | Notes |
| --- | --- | --- |
| `gemini/<model>` | `ANALYSIS_API_KEY` or `GEMINI_API_KEY` | Google Gemini frontier analysis. Review current Google pricing and quotas before large batches. |
| `openai/<model>` | `ANALYSIS_API_KEY` or `OPENAI_API_KEY` | OpenAI API analysis. Review current pricing and project limits before large batches. |
| `anthropic/<model>` | `ANALYSIS_API_KEY` or `ANTHROPIC_API_KEY` | Anthropic Claude analysis. Review current pricing and billing requirements before large batches. |
| `codex/<model>` | Local Codex CLI login | Uses `codex exec --model` through the `codex` binary on `PATH`. `CODEX_BIN` is optional only when the binary is not on `PATH`. |
| `lmstudio/<model>` | `LOCAL_LLM_URL` | Local OpenAI-compatible analysis endpoint, commonly LM Studio. `LOCAL_LLM_API_KEY` or `ANALYSIS_API_KEY` is optional when the server requires auth. |
| `ollama/<model>` | `LOCAL_LLM_URL` | Local OpenAI-compatible analysis endpoint for Ollama-compatible serving. |
| `omlx/<model>` | `LOCAL_LLM_URL` | Local OpenAI-compatible analysis endpoint for oMLX or similar local servers. |
| `local/<model>` | `LOCAL_LLM_URL` | Generic local OpenAI-compatible analysis endpoint. |

For local analysis, concurrency defaults are conservative. Tune
`LOCAL_LLM_CONCURRENCY`, `ANALYZE_CONCURRENCY`, and `LOCATE_CONCURRENCY` only
after the local model server is stable under load.

## Embedding Provider Options

| Provider ref | Required configuration | Notes |
| --- | --- | --- |
| `gemini/<model>` | `EMBEDDING_API_KEY` or `GEMINI_API_KEY` | Gemini embeddings. The common default is `gemini/gemini-embedding-001`. |
| `openai/<model>` | `EMBEDDING_API_KEY` or `OPENAI_API_KEY` | OpenAI embeddings. The default OpenAI-compatible base URL is `https://api.openai.com/v1`. |
| `lmstudio/<model>` | `EMBEDDING_BASE_URL` or `LOCAL_LLM_URL` | Local OpenAI-compatible embedding endpoint. The model must be installed and serving embeddings. |
| `ollama/<model>` | `EMBEDDING_BASE_URL` or `LOCAL_LLM_URL` | Local OpenAI-compatible embedding endpoint for Ollama-compatible serving. |
| `omlx/<model>` | `EMBEDDING_BASE_URL` or `LOCAL_LLM_URL` | Local OpenAI-compatible embedding endpoint for oMLX or similar local servers. |
| `local/<model>` | `EMBEDDING_BASE_URL` or `LOCAL_LLM_URL` | Generic local OpenAI-compatible embedding endpoint. |
| `openai-compatible/<model>` | `EMBEDDING_BASE_URL` | Explicit generic OpenAI-compatible embedding endpoint. |

Embedding startup probes the configured provider to discover vector dimensions.
That dimension is stored with the embedding model metadata so the database can
detect incompatible model changes.

## Safe Configuration Patterns

### Frontier analysis and frontier embeddings

```env
ANALYSIS_MODEL=gemini/gemini-2.0-flash
ANALYSIS_API_KEY=...
EMBEDDING_MODEL=gemini/gemini-embedding-001
EMBEDDING_API_KEY=...
```

This is the simplest first-run path, but email content is sent to the configured
frontier provider.

### Codex analysis and Gemini embeddings

```env
ANALYSIS_MODEL=codex/your-codex-model
CODEX_SANDBOX=read-only
CODEX_TIMEOUT_SECS=300

EMBEDDING_MODEL=gemini/gemini-embedding-001
EMBEDDING_API_KEY=...
```

This uses the local Codex CLI login for analysis and a direct embedding API for
retrieval.

### Local analysis and frontier embeddings

```env
ANALYSIS_MODEL=lmstudio/local-analysis-model
LOCAL_LLM_URL=http://localhost:1234/v1

EMBEDDING_MODEL=openai/text-embedding-3-small
EMBEDDING_API_KEY=...
```

This keeps analysis local while using a managed embedding model for retrieval.

### Local analysis and local embeddings

```env
ANALYSIS_MODEL=lmstudio/local-analysis-model
LOCAL_LLM_URL=http://localhost:1234/v1

EMBEDDING_MODEL=lmstudio/local-embedding-model
EMBEDDING_BASE_URL=http://localhost:1234/v1
```

Use this when the local server can serve both chat completions and embeddings.

## Changing Models Safely

Analysis models are operational choices. Change `ANALYSIS_MODEL`, restart the
runtime, and new analysis work will use the new model. Existing stored analysis
does not automatically disappear; force or re-run analysis when you want old
rows refreshed.

Embedding models are data-shape choices. Changing `EMBEDDING_MODEL` changes the
meaning and often the dimensions of stored vectors. By default,
`EMBEDDING_MODEL_MISMATCH=rebuild` clears incompatible embedding values and
lets backfill regenerate them.

Use this when you want startup to stop instead of resetting embeddings:

```env
EMBEDDING_MODEL_MISMATCH=fail
```

Then run an explicit embedding rebuild when ready:

```bash
mailsubsystem embed-rebuild
```

If the database only contains legacy raw model metadata and the current
provider/model plus dimensions match, MailSubsystem updates the stored metadata
to canonical `provider/model` form without rebuilding.

## Legacy Compatibility

The preferred selectors are `ANALYSIS_MODEL` and `EMBEDDING_MODEL`, but older
configuration still works:

| Legacy setting | Preferred setting |
| --- | --- |
| `AI_PROVIDER`, `FRONTIER_PROVIDER`, `FRONTIER_MODEL`, `LOCAL_LLM_MODEL` | `ANALYSIS_MODEL=provider/model` |
| `GEMINI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY` for analysis | `ANALYSIS_API_KEY` |
| `EMBEDDING_PROVIDER`, `EMBEDDING_GEMINI_MODEL` | `EMBEDDING_MODEL=provider/model` |
| Provider-specific embedding API keys | `EMBEDDING_API_KEY` |

Prefer the new settings for new deployments so analysis and embeddings can move
independently.

## Rate and Spend Notes

`API_RATE_LIMIT_RPM` is an initial provider-pressure guardrail for frontier
traffic. The runtime also adapts from provider pressure signals such as rate
limits and retry-after responses. Local providers are not given a fixed app-side
RPM by default.

Paid frontier providers can generate real charges. Before processing a large
mailbox, confirm the selected analysis and embedding models, current provider
pricing, and project limits.