Skip to content

Server Deployment

Server mode is the PostgreSQL composition selected with cortex --mode server.

Docker Deployment (GHCR Official Images)

Cortex publishes one official multi-platform image on GitHub Container Registry: - Cortex Server (API + embedded web UI): ghcr.io/lleontor705/cortex:latest

The single binary serves the operator web UI at / and the API at /api/* on the same listener; there is no separate web image and no second UI port.

# 1. Configure environment variables (optional overrides)
cp .env.example .env

# 2. Pull and start PostgreSQL and Cortex Server
docker compose up -d

The Compose file creates PostgreSQL, applies the embedded server schema, bootstraps one development deployment identity (server.tenant_id, server.workspace_id, server.principal_subject), and starts Cortex Server on port 7438, which also serves the embedded web UI at / on that same origin. On first boot the process prints the web UI endpoint, the key file path, and the web access key shown exactly once; retrieve them with docker compose logs cortex-server and paste the key at http://localhost:7438/. To rebuild locally from source code, use docker compose up --build -d.

For a full reference of all supported environment variables, see .env.example and CONFIGURATION.md.

Production Requirements

  • PostgreSQL 16 or newer supported by the deployment policy.
  • Separate privileged migration DSN and non-privileged runtime DSN.
  • Configured server.tenant_id, server.workspace_id, and server.principal_subject constants for the synthetic service principal.
  • A secret-managed static bearer token (http.token), or an upstream authenticated gateway that injects it.
  • Explicit http.allowed_origins for browser clients.
  • TLS at the deployment boundary.

Server persistence remains behind AuthorizedStore; transports never receive raw repositories, transactions, or client-selected tenant authority. Every authenticated request is the single synthetic service principal built from the configured server.tenant_id, server.workspace_id, and server.principal_subject constants, so tenant and workspace never come from client input. /health is public, while /api/* and /mcp require the static bearer.

PostgreSQL role boundary

Production-like server startup requires both connection strings and validates their parsed PostgreSQL role names before opening a database handle:

  • server.storage.dsn / CORTEX_SERVER_STORAGE_DSN uses the long-lived, non-superuser cortex_app role with no BYPASSRLS.
  • server.storage.migration_dsn / CORTEX_SERVER_STORAGE_MIGRATION_DSN uses the privileged cortex_migration role for migrations and startup bootstrap reconciliation.
  • The two roles must differ. They may target the same PostgreSQL host and database.

DSNs and passwords are never included in boundary-validation errors. The runtime closes the migration handle before creating its long-lived application pool. Moving migrations and bootstrap reconciliation into a standalone one-shot job remains a required follow-up.

Only local development may set server.bootstrap_development: true and omit migration_dsn; that explicit switch permits Cortex to reuse the runtime DSN for migration/bootstrap convenience. Never enable this fallback in production.

Conversational Agent Configuration

The conversational agent uses administrator-owned server LLM configuration; clients cannot send a provider, model, URL, API key, tool list, or output limit. At minimum configure a supported provider and model, plus a secret-managed API key when the provider requires one:

CORTEX_LLM_PROVIDER=openai
CORTEX_LLM_MODEL=gpt-4o-mini
CORTEX_LLM_API_KEY=<from-secret-manager>

CORTEX_LLM_BASE_URL is optional for provider presets. Custom destinations must be absolute HTTPS URLs without embedded credentials. The hardened outbound policy also supports CORTEX_LLM_ALLOWED_HOSTS, CORTEX_LLM_ALLOWED_PORTS, CORTEX_LLM_MAX_CONCURRENT, CORTEX_LLM_MAX_REDIRECTS, CORTEX_LLM_MAX_RESPONSE_BODY_BYTES, CORTEX_LLM_MAX_ERROR_BODY_BYTES, CORTEX_LLM_TIMEOUT, and CORTEX_LLM_CA_FILE. The loopback switches CORTEX_LLM_ALLOW_LOOPBACK and CORTEX_LLM_ALLOW_LOOPBACK_HTTP are for explicit local development only. Invalid destination policy fails server startup; an unconfigured provider leaves the agent endpoints fail-closed with a sanitized unavailable response.

Prometheus Metrics

GET /metrics serves a hand-rolled Prometheus text exposition (v0.0.4, # HELP/# TYPE counters and gauges — no scrape dependency needed). It is admin-gated on the same authenticated origin (7438): the bearer authentication wraps the handler and the handler enforces the same administrative capability as GET /api/admin/ai/status; there is no separate unauthenticated metrics port. Scrape config needs only the bearer token:

scrape_configs:
  - job_name: cortex
    metrics_path: /metrics
    authorization:
      type: Bearer
      credentials: <server bearer>

Exposed series: cortex_query_embedding_cache_{hits,total}-style counters for the query-embedding cache (cortex_query_embedding_cache_hits_total, cortex_query_embedding_cache_misses_total), rerank gate outcomes (cortex_rerank_applied_total, cortex_rerank_gate_fail_total), provider traffic (cortex_embedding_provider_requests_total, cortex_embedding_provider_errors_total), background embedding worker activity (cortex_worker_embedding_batches_total, cortex_worker_embeddings_total, cortex_worker_errors_total), and the cortex_process_uptime_seconds gauge.

Authenticated API

  • GET /api/agent/projects returns only projects for which the verified principal has searchable memory or a ready, readable code index.
  • POST /api/agent/answer returns one canonical JSON answer.
  • POST /api/agent/stream returns SSE events and disables proxy buffering.

All three endpoints require server bearer authentication and set Cache-Control: no-store. A request body has this closed shape; unknown fields are rejected:

{
  "project_id": "server-issued-project-id",
  "question": "¿Qué se desarrolló para aislar los vectores?",
  "history": [
    {"role": "user", "content": "Resume el modo server"},
    {"role": "assistant", "content": "..."}
  ]
}

The JSON response contains answer, validated sources, confidence, and a content-free retrieval trace: tier, unique ordered stages with status/count, optional refinement_count, allowlisted degraded codes, and invalid-citation count. A trusted corpus generation may be added when the retrieval port carries one; clients must not synthesize it from request data or internal checksums. Streaming emits meta, delta, zero or more citation, and a terminal done carrying the same canonical object; a failure after the stream starts terminates it with a sanitized error event. Reverse proxies must preserve client disconnects, use a timeout greater than 60 seconds, and disable response buffering for /api/agent/stream.

Requests are bounded to an 8 KiB question, at most 12 history messages (six turn pairs), 4 KiB per message, and 24 KiB aggregate history. The standard tier currently allows 30 requests and 60,000 reserved/reconciled output tokens per minute, two concurrent requests per tenant, a 1,200-token default, a 4,096-token hard cap, a 30-second JSON deadline, and a 60-second SSE deadline. Exhaustion returns 429 quota_exceeded with bounded Retry-After; timeouts return 504 agent_timeout. These counters are process-local, so multiple replicas must sit behind a shared limiter before general production enablement.

Agent audit records must contain only correlation and actor identifiers, tenant/workspace/project, transport, result class, duration, token/source counts, confidence, and degradation flags. They must never contain questions, history, answers, retrieved memory/code, embeddings, secrets, or provider URLs. The pre-provider authorization record is mandatory and fail-closed (503 audit_unavailable); outcome delivery is best effort with content-free failure telemetry. Treat a deployment without a durable wired agent audit sink as not ready for production.

MVP limits

  • Read-only answers: no tools, writes, administrative actions, or repository commands.
  • Code context is AST metadata (symbols, signatures, summaries, paths, positions, and relations), not complete source-file bodies.
  • Browser conversation history is memory-only and is cleared on logout, project change, unmount, or New conversation.
  • Missing vector or code evidence may produce a degraded answer from other authorized evidence. No authorized evidence returns a sanitized unavailable result without broadening scope.
  • Adaptive retrieval uses four bounded tiers: direct factual, semantic hybrid, multi-hop graph, and architectural global. Dense retrieval is fused with lexical results through RRF and reranked with MaxSim; graph tiers use bounded HippoRAG-style PPR and LightRAG community summaries. CRAG can reformulate locally at most once. A second low-confidence result abstains with crag_insufficient_confidence.
  • Community-summary caches are scoped by tenant, workspace and project identity, fingerprint both memory and AST inputs, and have a bounded process-local cardinality. Cancellation and unavailable/truncated graphs fail closed instead of serving stale summaries.

Rollout Order And Stage Gates

The server schema is embedded in the binary (migrations/v2/100_server.sql … 112_static_bind_contract.sql). Rollout always advances in this exact order; each stage has explicit advance, abort, and retry criteria, and no stage starts before the previous one advances.

Stage 1 — Migrations (database)

  1. Bootstrap PostgreSQL 16 and create the non-superuser application and authorization roles with scripts/postgres/bootstrap-authz.sql through a privileged DSN.

  2. Configure the runtime and migration DSNs simultaneously, using distinct roles, then start the new runtime. Each migration applies in strict version order inside one transaction guarded by the pg_advisory_xact_lock(hashtext('cortex:v2:server-migrations')) lock and is recorded in the cortex_server_migrations ledger with its SHA-256 checksum.

  3. Advance when the ledger records every expected version (100…112) with matching checksums, migration 109 post-apply verification confirms scoped tables/RLS/grants, and startup completes without ErrFutureMigration.

  4. Abort when any preflight fails (orphan rows, unledgered artifacts, checksum mismatch, future ledger version): the transaction rolls back, the ledger keeps no partial entry, and the runtime refuses to serve.
  5. Retry by restarting the same binary: applied migrations are checksum-verified and skipped, so a retry is always idempotent. Never hand-edit schema or ledger rows to "get past" an abort.

Stage 2 — Server (traffic)

  1. Confirm the long-lived pool uses the non-privileged runtime DSN; the startup migration handle uses the separately configured migration DSN and is closed before traffic is served.

  2. Put the deployment behind TLS at the boundary and wire the secret-managed bearer token or authenticated gateway.

  3. Advance when /health answers 200 and an authenticated /api/me round-trip returns the configured synthetic service principal.

  4. Abort when startup validation or authentication smoke checks fail: the process exits fail-closed without serving; the database is never mutated at this stage.
  5. Retry by restarting after fixing configuration; re-entering Stage 1 is unnecessary because migrations are already ledgered.

Stage 3 — Proxy/local clients

  1. Point local SQLite clients (cortex serve, MCP profiles agent/admin/temporal) and any HTTP proxies at the deployment.

  2. Local HTTP still refuses non-loopback binding without http.token, and app.Open refuses v1/Engram/foreign databases without mutation.

  3. Advance when the local unit/build gates (go build ./..., go test -v -count=1 ./...) and the focused portability gates pass on the client release.

  4. Abort when the read-only compatibility probe refuses a local database: keep the old local database untouched and investigate; the probe never mutates.
  5. Retry freely; all client checks are read-only.

Stage 4 — Plugins

  1. Distribute the OpenCode plugin (cortex setup opencode) and the Claude Code hooks package.

  2. Both plugins talk to the server through the same authenticated HTTP surface as any other client.

  3. Advance when plugin/opencode passes npm ci && npm test (Vitest) and the Claude harness bash plugin/claude-code/scripts/hooks_test.sh exits 0.

  4. Abort when the harness exits 127 (missing bash/jq/python3/timeout): that is a BLOCKED toolchain, not a plugin defect — fix the toolchain and rerun. Plugin HTTP failures are classified, never silently dropped.
  5. Retry by rerunning the harness, which is fully offline/deterministic. Live hook delivery is NOT idempotent by itself: a retried subagent-stop capture that POSTs to /api/observations can create a duplicate observation, because retries do not deduplicate. Treat a retry as safe only after the prior attempt's failure classification confirms the request never reached the server, or after checking the evidence (for example, searching for the observation before re-sending). When the idempotent handoff receipt path is available and configured, prefer it for retries — handoff writes are deduplicated server-side by receipt identity.

Commit evidence per stage

Every stage rollout records durable evidence before the next stage starts: the CI workflow run URL (the ubuntu-latest executor is authoritative for PostgreSQL/race/plugin gates), the coverage threshold result (≥ 70%), the migration ledger head reached, and a Cortex observation or ForgeSpec task note linking the artifacts. Release additionally requires the manual approval gate to have observed all twelve stage-1 gates green on the exact commit being shipped.

Workspace-scoped vector rollout

Vector replicas are optional and PostgreSQL lexical search remains authoritative. Roll out vector isolation in this order:

  1. Deploy the runtime and adapter schema/filter support first. Confirm every server vector operation is wrapped with the configured tenant and workspace boundary and that project-selected searches require the server-resolved public project_id UUID.
  2. Reindex one project at a time, stamping tenant_id, workspace_id, and project_id on every point; the project label is display metadata only. Do not rewrite or delete legacy vectors during rollout: points missing project_id cannot satisfy a project-selected query and remain fail-closed until reindexed.
  3. Verify coverage for that exact project UUID: indexed observations must match the authoritative PostgreSQL count, and same-label sibling projects plus sibling workspaces must contribute zero candidates.
  4. Only then enable or advertise semantic recall for that workspace. Missing metadata, incomplete coverage, unhealthy adapters, or embedding failures keep /api/search/hybrid on authorized lexical results.

Rollback disables the vector provider or redeploys the previous runtime while retaining the authorization, CORS and DSN boundaries. It never removes PostgreSQL data or weakens tenant/workspace filtering.

Run the scoped vector replica reindex synchronously from the server composition:

cortex --mode server reindex --project-id <public-project-uuid> --config /path/to/server.yaml

The command verifies the configured http.token through the durable token repository, requires audited ResourceAdmin/ActionManage, and resolves the project UUID plus canonical label inside the principal-bound PostgreSQL tenant/workspace. There are deliberately no tenant, workspace, internal ID or label override flags. It refuses absent or unhealthy embedding/vector providers, an empty corpus, skipped observations, or coverage where upserted points do not equal the authoritative corpus. Audit rows contain only project identity, counts, status and duration; cancellation stops provider/vector work but still attempts one bounded terminal audit. The command is idempotent because the external replica upserts by observation ID. It does not expose a long-running HTTP endpoint and does not modify or delete legacy points.

Conversational agent rollout

Migration 109 is additive and forward-only. It creates the scoped AST tables and revokes cortex_app access to legacy unscoped code_symbols and code_relations; it intentionally copies no legacy rows. Roll out the agent in these waves:

  1. Apply 109 with cortex_migration. Verify the ledger checksum, composite foreign keys, forced RLS, and that cortex_app cannot read legacy AST.
  2. Deploy the 109-aware runtime while keeping agent traffic disabled at the ingress or tenant-release layer. Confirm /health, /api/me, and the normal search surfaces remain healthy.
  3. Run the authenticated administrative AST reindex for one canary project. Tenant and workspace must come from trusted server composition and the project from durable grants; never accept them as an operator-provided SQL mapping. The job must be idempotent and record only counts/checksum/status.
  4. Confirm scoped_code_index_state reaches ready, counts match the approved checkout, and sibling project/workspace/tenant isolation probes return zero rows. States missing, indexing, or failed are not ready.
  5. Enable JSON for the canary, then SSE. Verify citation handles, cancellation, timeouts, quota rejection, content-free audit delivery, and proxy streaming.
  6. Reindex and enable remaining projects/tenants only after each project has equivalent readiness evidence.

Until a project is ready, legacy AST is excluded. The agent may answer from authorized memory and report code degradation; when no authorized corpus is available it must fail without calling the provider. The current repository does not provide a production administrative caller for the scoped AST reindex, so this remains a deployment blocker rather than permission to use direct SQL.

Rollback disables agent routes at ingress/tenant release and scoped AST ingestion, then redeploys the previous compatible runtime if needed. Preserve migration 109 and indexed data. Do not restore runtime access to legacy AST, drop scoped tables, rewrite the ledger, weaken RLS/CORS/SSRF controls, or delete PostgreSQL/vector data.

Migration 105_workspace_binding.sql is additive: it adds workspace_id to observations and handoff receipts, backfills strictly from the durable session -> observation -> receipt chain, and aborts the whole transaction (no partial ledger entry) on any orphan or unresolved row. Like 104, it omits IF NOT EXISTS on purpose so stale, unledgered artifacts fail closed instead of being silently adopted.

Rollback Policy

The migration line is forward-only; there is no Down.

  • Rollback means redeploying the previous runtime binary. Migration 105 is additive, so a 104-era runtime keeps working: it keeps writing observations through the compatibility BEFORE trigger, while durable pending receipts intentionally fail closed rather than accept an insecure tenant-wide workspace default.
  • Never "undo" a migration with DROP/DELETE against schema objects or ledger rows; that forks the migration line and makes the database unverifiable.
  • Never rewrite a checksum in cortex_server_migrations. A checksum mismatch fails closed, and a ledger recording a version beyond the runtime's head (ErrFutureMigration, REM-ROLLOUT-001) means a newer runtime owns the line: this runtime refuses to read or write any migration row before operating.

Local vs CI Gates

PostgreSQL-dependent gates (integration, e2e, scoped AST isolation, and the global coverage threshold of at least 70%) fail — they do not skip — without CORTEX_TEST_POSTGRES_DSN, CORTEX_TEST_POSTGRES_MIGRATION_DSN, and CORTEX_TEST_POSTGRES_AUTHZ_ADMIN_DSN. Run the agent isolation gate with:

go test -v -count=1 -tags postgres_integration ./internal/migration ./internal/store/postgres ./internal/platform/server

The race gate requires CGO with a working gcc, and the Claude plugin harness requires jq; on typical Windows workstations these report BLOCKED locally. .github/workflows/ci.yml (ubuntu-latest with a PostgreSQL 16 service) is the authoritative executor for them, and the release workflow re-runs the same suite before its manual approval gate. The Web gate is npm test && npm run build inside web/.