Consent Docs
Open console

Send the audit log to your SIEM

Pull the audit log into your SIEM with a key, or collect it from the API's standard output on your own deployment.

How-to2 min readFor project administrators and deployment operators

Send the audit log to your SIEM so it keeps its own copy. A collector pulls it with a key, or, on your own servers, reads the API's standard output.

Choose a way#

Where the product runs Way
Our cloud The feed, GET /sdk/v1/audit-log
Your own servers Standard output, or the feed

Pull the feed#

  1. On Install, open Scopes on a live key that has a secret.
  2. Tick Read the project’s audit log, for your SIEM and choose Save scopes.
  3. Store the secret on the collector host only.
  4. Run the relay below, or point your collector at GET /sdk/v1/audit-log.
  5. Keep the last nextCursor and send it as after on the next call.
  6. Remove duplicates by id. An entry can arrive more than once.

The feed needs a live key: a test key gets 403 LIVE_KEY_REQUIRED. A key may read it 60 times a minute. Entries younger than 60 seconds are held back, so an entry still being written is not skipped. See Audit log.

Collect standard output#

  1. Each audit entry is also one JSON line on the API's standard output, with "context":"AUDIT".
  2. Route the container's logs to your collector with a Compose override.
  3. Keep only lines whose context is AUDIT.
  4. Reload the collector.
  5. Change a setting in the console and check that its entry arrives.

A line with "stored": false is an entry the database did not store. Whoever runs the deployment can turn the copy off with AUDIT_LOG_STDOUT=false.

Examples for your SIEM#

Compose override, to the host's syslog:

docker-compose.override.ymlYAML
services:
  consent-api:
    logging:
      driver: syslog
      options:
        syslog-address: "unixgram:///dev/log"
        tag: piedap-consent

rsyslog, forwarding audit lines over TLS:

/etc/rsyslog.d/40-piedap-audit.conf
global(DefaultNetstreamDriverCAFile="/etc/ssl/certs/siem-ca.pem")
if $programname == "piedap-consent"
   and $msg contains "\"context\":\"AUDIT\"" then {
  action(type="omfwd" target="siem.example.in" port="6514"
         protocol="tcp" StreamDriver="gtls" StreamDriverMode="1"
         StreamDriverAuthMode="x509/name"
         StreamDriverPermittedPeers="siem.example.in")
  stop
}

Fluent Bit to the Splunk HTTP Event Collector, with the Compose driver: fluentd, fluentd-address: "127.0.0.1:24224" and tag: piedap.api:

fluent-bit.conf
[INPUT]
    Name   forward
    Listen 127.0.0.1
    Port   24224

[FILTER]
    Name         parser
    Match        piedap.*
    Key_Name     log
    Parser       json
    Reserve_Data On

[FILTER]
    Name  grep
    Match piedap.*
    Regex context ^AUDIT$

[OUTPUT]
    Name             splunk
    Match            piedap.*
    Host             splunk.example.in
    Port             8088
    TLS              On
    Splunk_Token     ${SPLUNK_HEC_TOKEN}
    Event_Sourcetype piedap:audit

A relay for the feed, for Node.js 18 or later with no packages:

audit-relay.mjsJavaScript
import { readFileSync, writeFileSync } from 'node:fs';

const API = 'https://consent.example.in/sdk/v1/audit-log';
const KEY = process.env.PIEDAP_AUDIT_KEY;
const STATE = '/var/lib/piedap/audit-cursor';
const sleep = (ms) => new Promise((done) => setTimeout(done, ms));

let cursor = '';
try {
  cursor = readFileSync(STATE, 'utf8').trim();
} catch {}

for (;;) {
  const query = new URLSearchParams({ limit: '1000' });
  if (cursor) query.set('after', cursor);
  const response = await fetch(`${API}?${query}`, {
    headers: { Authorization: `Bearer ${KEY}` },
  });
  if (response.status === 429) {
    await sleep(Number(response.headers.get('retry-after') || 60) * 1000);
    continue;
  }
  if (!response.ok) throw new Error(`Feed answered ${response.status}`);
  const page = await response.json();
  for (const item of page.items) {
    process.stdout.write(`${JSON.stringify(item)}\n`);
  }
  if (page.nextCursor) {
    cursor = page.nextCursor;
    writeFileSync(STATE, cursor);
  }
  if (!page.hasMore) await sleep(60000);
}
Shell
node audit-relay.mjs >> /var/log/piedap/audit.jsonl

A Splunk Universal Forwarder can watch that file:

inputs.conf
[monitor:///var/log/piedap/audit.jsonl]
sourcetype = _json
index = security
index=security sourcetype=_json action=lookup
| table createdOn userEmail details.records

For IBM QRadar, forward the relay file with rsyslog imfile, or the syslog lines above, on TCP 514 or TLS 6514. Use the log source type Universal DSM. Read fields with the JSON expressions /"action", /"resource", /"userEmail", /"ipAddress" and /"details"/"statusCode".

Limits#

  • These are examples, not a connector. Keep the collector running yourself.
  • Failed sign-ins and sessions are logged by PIEDAP IAM, not here.
  • An entry whose write commits later than the 60-second hold reaches the feed late or not at all. The download and standard output still have it.
  • Platform actions that belong to no project are on standard output only.
  • The log is append-only in the database. The table's owner can still turn the guard off. Each entry is sealed to the one before it, but the log is not anchored, so your SIEM copy is the independent record.

↑↓ to move, Enter to open