Self-hosted deployment problems

For the people who run a deployment. Find the problem by what you see. The commands assume the supplied compose file.

The API stops right after starting#

What you see: The API container exits. The last log lines start with Config error: or report the database.

Why it happens: A required setting is missing or wrong.

Message	Fix
`ENCRYPTION_KEY is required`	Set it. Production has no fallback
`PUBLIC_API_URL is required in production`	Set the public HTTPS address
`AUTH_BYPASS=true needs NODE_ENV=development or test`	Remove `AUTH_BYPASS`
`Evidence keys are invalid`	Check `EVIDENCE_CHAIN_KEYS`, the active key and the signing key
`Database connection failed after 10 attempts`	Check `DB_HOST`, the credentials and the network
`ALLOW_PRIVATE_WEBHOOK_URLS is no longer read`	Remove it. List private networks in `PRIVATE_NETWORK_ALLOWLIST` and turn on Private network on those endpoints
`PRIVATE_NETWORK_ALLOWLIST:` and an entry	Fix or remove that entry. A range wider than `/8` is refused

Fix: Read the log, correct the variable and start again. See Configuration.

Shell

docker compose logs --tail 50 consent-api

What you see: Every console request answers 403 with "Unauthorized Access".

Why it happens: The API cannot read the session that the sign-in service made.

LOGIN_COOKIE or LOGIN_SECRET does not match the IAM deployment.
The API is not connected to the Redis that IAM writes sessions to.
The session cookie's domain does not cover the console's hostname.

Fix: Check redis in the /health answer and REDIS_HOST. Then compare the login variables with the IAM deployment. See Deploy with Docker Compose.

A variable you set has no effect#

What you see: You add a variable such as TRUST_PROXY or LOG_LEVEL to .env, and the API behaves as before.

Why it happens: The compose file passes only the variables it lists to the API container. It also sets DB_HOST, DB_PORT, REDIS_HOST and REDIS_PORT to its own containers, whatever .env says.

Fix: Set the variable in an override file, which Compose merges on its own. See the compose file.

docker-compose.override.ymlYAML

services:
  consent-api:
    environment:
      SARVAM_API_KEY: ${SARVAM_API_KEY:-}
      LOG_LEVEL: warn

The console reports a CORS error#

What you see: The browser reports a CORS error on console requests.

Why it happens: The console is served from an origin the API does not know.

Fix: Set UI_URL to the exact origin, or ADMIN_UI_URL for a second origin, and restart. See Configuration.

What you see: A customer's site shows no banner. The browser's network panel shows a failed request.

Why it happens: The status names the cause:

Request	Cause
`401` on `/sdk/v1/form`	The origin is not on the key's list, or the key is revoked
`503` with `// runtime not built` on `/sdk/v1/loader.js`	The image was built without the runtime files
`404` on `/sdk/v1/runtime/<version>/…`	That version's folder is not in the image

Fix: For 401, see Banner and tag problems. For the others, rebuild the API image with the runtime files:

Shell

./scripts/sync-sdk-assets.sh
docker compose build consent-api

Keep the older version folders in the image. Sites with a pinned version load them.

Many callers get 429#

What you see: Many visitors or one customer are rate limited.

Why it happens: It depends on who is limited:

Almost everyone at once: the API sees your proxy's address as the client, so all visitors share one budget.
One customer: their key's limit is reached. A test key allows 600 requests a minute.
One server on calls about people: it reached SDK_CRITICAL_CALLS_PER_MINUTE.

Fix: Set TRUST_PROXY so the real client address is used. Do not set it to true on a public port. A customer whose key is at its limit creates a key with a higher Rate limit, or goes live. For calls about people, raise SDK_CRITICAL_CALLS_PER_MINUTE. See Configuration.

Install snippets point at localhost#

What you see: Install snippets, confirmation links and hosted page links start with http://localhost:8080.

Why it happens: PUBLIC_API_URL is not set, and the compose file falls back to localhost.

Fix: Set PUBLIC_API_URL to the public HTTPS address, without a trailing slash, and restart. See Configuration.

A key cannot go live#

What you see: The platform endpoints answer 503 with PLATFORM_API_DISABLED.

Why it happens: PLATFORM_API_KEY is not set. Until it is, no key can become live.

Fix: Set the variable and restart. See Approve live keys.

A property import is refused with 413#

What you see: Importing a large property file fails with 413.

Why it happens: A proxy limits the request body. The console's nginx sets no size, so its default of 1 MB applies.

Fix: Add client_max_body_size 200m; to web/nginx.conf and allow the same in your own proxy. See Deploy with Docker Compose.

Scans fail#

What you see: Every site scan ends as failed.

Why it happens: The API has no browser, the target is not a public address, or the API restarted during the scan.

Fix: Build the image without INSTALL_SCANNER=false, and check SCANNER_CHROMIUM_PATH. After a restart, start the scan again. See Configuration.

Translate asks for a Sarvam key#

What you see: Translate in the console says to add a Sarvam key.

Why it happens: SARVAM_API_KEY is not set, and the project has no key of its own.

Fix: Set the variable. Or have the customer add a key or choose another service in Project settings. See Set up the translation service.

Every secret key is refused#

What you see: Every call with an sk_ key is refused.

Why it happens: ENCRYPTION_KEY changed. A new value makes every secret key invalid.

Fix: Set it back. See Rotate secrets.

Records cannot be found by identifier#

What you see: Lookups by email address or user id find nothing for records that exist.

Why it happens: SUBJECT_HASH_PEPPER changed, or was first set to something other than ENCRYPTION_KEY.

Fix: Set it back. Records made under the wrong value are then the ones that cannot be found. See Rotate secrets.

A record's chain key is not held#

What you see: A record reads Cannot be checked here. Under Technical proof, its Proof chain reads Key not held here.

Why it happens: A key was removed from EVIDENCE_CHAIN_KEYS. Or dedicated keys were set on a deployment that used derived ones.

Fix: Put the missing key back in the list. Never remove a key while evidence sealed with it is kept. See Rotate secrets.

An anchor does not match the database#

What you see: Verify on an evidence anchor shows a failed check.

Why it happens: The failed check points to the cause:

Failed check	Likely cause
`events`, with fewer events	Rows were removed outside the product, or the database was restored to an earlier point
`events`, with the same count	An event of that day was altered
`signature`	The signing key changed after the anchor was made
`sequence`	An earlier anchor is missing or was made again

Fix: For a failed events check, compare with the customer's stored copy of the anchor. Restore from a backup taken after that day if you have one. For a failed signature, see Rotate secrets. See also Back up and restore.

Evidence is not being deleted#

What you see: Records past their retention period stay in the database.

Why it happens: ENFORCE_EVIDENCE_RETENTION is not true, or the record is not due. A record still in force, or one with no expiry date, is never deleted.

Fix: Turn enforcement on only when the data owner has decided to. See Record retention.

Inactivity erasure is not happening#

What you see: A project has Erase after inactivity on, but nobody is erased. The log repeats Inactivity erasure is on for project every hour.

Why it happens: The project has no enabled endpoint subscribed to subject.erasure_notice. Nothing is erased until the customer can be told first.

Fix: Have the customer add the endpoint. See Erase people after inactivity.

A webhook endpoint is disabled#

What you see: A customer's endpoint shows Disabled, and the log has a line that starts Disabled webhook.

Why it happens: Ten events in a row failed every attempt.

Fix: Have the customer fix the receiver and switch the endpoint on. Then send the failed events again from Failed events on Install. See Replay failed webhooks.

Deleting from audit_log is refused#

What you see: A statement on audit_log fails with audit_log is append-only; DELETE is refused, or the same for UPDATE or TRUNCATE.

Why it happens: A trigger keeps the audit log append-only, so neither the application nor an accidental statement can change it.

Fix: To archive old entries, download them first, then delete them as the table's owner with the trigger turned off for that one transaction. See Remove old entries.

Still stuck#

Collect these before you ask for help:

the answer from /health
the log lines with "level":"error" around the time of the problem
the X-Request-Id of a failed request
the commit your images were built from