Documentation · Bulk data

Delta files

Instead of downloading a whole collection after every build, you can apply only what changed. Each build publishes the records added, changed and removed since the build before it.

The bulk files are free to download, with no account. Non-commercial use is covered by our data licence; commercial use needs a licence. To check names one at a time instead, use the screening API.

Deltas cover the FollowTheMoney entities in entities.ftm.json (the listed records, not their Sanction entities). There are no deltas for the CSV, names or Senzing files; rebuild those from your updated entities, or download them again.

What is a delta?

At every build we take a fingerprint of each record: the SHA-1 of its complete JSON line. Comparing it with the previous build's fingerprint gives three kinds of change:

A change is not always a change at the source. A record is also marked MOD when our own processing changes it, for example a corrected entity type, a new merge (canonical_id) or a new confirmation date (last_seen). A record can leave a collection because its publisher removed it, or because its register is no longer cleared for redistribution.

How to use deltas

1. Download the full collection once and note its version from index.json.
2. Later, read the collection's deltas.json, which lists recent builds oldest first.
3. Take every build after your version, in order, and apply its delta file. Builds that changed nothing have "url": null; skip them.
4. Store the last version you applied, and repeat.

If your version is older than the first entry in deltas.json (we keep the last 30 builds), the chain is broken: download the full collection again.

/downloads/collections/<collection>/index.json     version of the current build
/downloads/collections/<collection>/deltas.json    recent builds, oldest first
/downloads/collections/<collection>/delta.json     changes in the current build only

An entry in deltas.json:

{
  "scope": "sanctions",
  "keep": 30,
  "versions": [
    {
      "version": "20261004012811-k2p",
      "previous_version": "20261003174737-un0",
      "generated_at": "2026-10-04T01:28:11.000Z",
      "added": 12,
      "modified": 40,
      "removed": 3,
      "url": "/downloads/collections/sanctions/deltas/20261004012811-k2p.json",
      "bytes": 48213,
      "sha256": "…"
    }
  ],
  "oldest_version": "20261004012811-k2p",
  "latest_version": "20261004012811-k2p"
}

The values above show the layout; they are not real figures. The live list is below.

Recent builds

While the register is not being refreshed, builds find no changes and publish no delta file. See freshness for when the data was last processed.

Understanding delta files

A delta file is JSON lines: one operation per line, each with op and entity. ADD and MOD carry the whole entity, exactly as it appears in entities.ftm.json; DEL carries only the id.

{"op":"ADD","entity":{"id":"crown-380054154","caption":"PELIKAN","schema":"Vessel","properties":{…},"datasets":["paris_mou_banned"],…}}
{"op":"MOD","entity":{"id":"crown-379700342","caption":"CHIRURGUL","schema":"Person","properties":{…},…}}
{"op":"DEL","entity":{"id":"crown-378468435"}}

import json

def apply_delta(store: dict, path: str):
    """store maps record id -> entity"""
    with open(path, encoding="utf-8") as f:
        for line in f:
            change = json.loads(line)
            entity = change["entity"]
            if change["op"] == "DEL":
                store.pop(entity["id"], None)
            else:  # ADD or MOD: replace in full
                store[entity["id"]] = entity

Merges and id changes

A record id (crown-<number>) belongs to one row on one register; merging records does not change it. A merge shows up as MOD operations that add or change canonical_id on each record involved, and a split removes or changes it. If you group records by canonical_id, regroup the records named in each MOD. If a publisher re-issues a listing under a new reference, expect a DEL for the old record and an ADD for the new one. See identifiers and deduplication.

Last updated 2026-10-03