Data model

The objects the product stores, how they point at each other, and which links the database enforces. Key ideas explains the same objects in plain language.

The hierarchy#

Organisation, account and project come from the PIEDAP IAM service. The first level this product owns is the property: one site or app.

Project (from IAM)
├── Shared by every property
│   ├── Purposes            why data is used; reused by forms
│   ├── Policies            privacy notices a form links to, in versions
│   ├── Project settings    one row per project
│   ├── Webhook endpoints   with an outbox and a delivery log
│   ├── Rights requests     by person
│   ├── Breach register
│   ├── Evidence anchors    one per day with sealed events
│   ├── Partner register    what each partner is to you
│   ├── Erased hashes       identifiers of erased people
│   └── Audit log           append-only
├── Personal dashboards     each person's own Overview charts
└── Property                a site or app, recognised by its origins
    ├── Consent forms       what is asked
    │   ├── Versions        frozen when published
    │   └── Form purposes   which purposes the form offers, in order
    ├── Region rules        which form each place sees
    ├── Experiments         two forms of the property compared
    ├── Keys                tied to the property, or to the whole project
    ├── Scans and cookies   what was found on the property
    ├── Consent imports     one per file imported
    ├── Notice campaigns    a notice version sent to a list
    │   ├── Uploads, people and their identifiers
    │   ├── Batches and links
    │   └── Campaign anchors, one per day with answers
    └── Consent records     one per person per form
        ├── Events          every state, sealed in a chain
        ├── Linked identifiers
        ├── Email confirmations
        └── Processor tasks, with their reports

Rows of configuration, records and requests carry the organisation, account and project ids. Three tables differ:

• the audit log may leave them empty
• a leaf tombstone carries only the project
• an idempotency record carries the project and the key

The objects#

How they connect#

One object on the left has many on the right, unless the note says otherwise.

A region rule does not own a form, and a form does not own a region rule. Both belong to the property, and the rule points at one of that property's forms.

From a page view to a sealed record#

1. The request carries a key. A key tied to a property reaches only that property.
2. Without such a key, a property named in the request is used. Failing that, the request's origin decides, and the most specific match wins.
3. The property's region rules are matched against the person's location. The most specific place wins: subdivision, then country, then continent, then the fallback.
4. The rule names a form, and its published version is served. A running experiment can serve its other form instead.
5. The answer is written as a consent record against the version the person was shown.
6. The new state is sealed as an event and chained to the one before.
7. After the day closes, the event becomes a leaf of that day's anchor.

How consent is proven explains the seal and the anchor.

What the database enforces#

Foreign keys#

Every key inside the hierarchy includes the project id, so a row cannot point at another project's data. Keys that must stay on one site also include the property id. Changing a referenced id is always refused.

Two links use a single column: a form's policy and a record's region rule. Deleting the parent clears the link, and a key that included the project would clear the project id with it. The application checks the project when it writes them.

Uniqueness#

What follows from these rules#

• A form, version or property that has consent records cannot be deleted. The evidence would lose the wording it points at.
• A consent record always names the property of its form and a version of that same form.
• A region rule can only show a form of its own property.
• An experiment can only compare two forms of the same property.
• A form offers only purposes of its own project, and a purpose that a form offers cannot be deleted.
• Deleting a record removes its events, linked identifiers and email confirmations with it. The retention job keeps a tombstone of each sealed event first.

What the service enforces#

The application checks these rules. No database constraint does.

Project settings and form settings#

How a banner behaves is set on each form and published with it. The project keeps what applies to the whole organisation. A published version keeps the settings it was published with.

People across records#

There is no "person" table. A person is whatever shares an identifier:

• A consent record holds the keyed hash of the identifier it was created under.
• Linked identifiers add more hashes that lead to the same record.
• A rights request holds the hash of the identifier it was filed under. It can be pointed at a different identifier later.

A lookup hashes the identifier you give and returns every record that matches directly or through a link.

• How identifiers are stored: Identifiers
• The lookup call: Find a person