Documentation · Data model

Dataset metadata

Every collection comes with machine-readable files that describe it: when it was built, what it holds, which registers contribute and which files to download. Use them to automate downloads and to check that a file is complete.

The catalogue

One file lists every collection with its current version, figures and files:

/downloads/collections/index.json

It lists — collections today. Fetch it on a schedule, compare each collection’s version with the one you hold, and download again only when it has changed. Collections can be added; your code should ignore ones it does not know.

To be provided: how often customers may poll the catalogue and how much notice is given before a collection is removed or renamed (see Change policy).

Collection metadata: index.json

Each collection has its own index.json, for example /downloads/collections/maritime/index.json. Its fields:

Section	Field	Description
—	`scope`	Short name of the collection, used in file paths.
—	`title`	Display name of the collection.
—	`version`	Build version: the UTC date and time of the build plus a short random suffix. Changes with every build.
—	`generated_at`	When the build was written.
—	`last_change`	When the content last changed compared with the previous build.
—	`records`	Number of records (listed parties).
—	`searchable`	Records with a usable name.
—	`distinct_entities`	Records counted once per cleaned name and birth date. A rough measure; see Identifiers.
—	`sources`	Number of contributing registers.
—	`first_seen`	Earliest first-seen date of any record.
—	`last_processed`	Most recent date any record was confirmed at its publisher.
`types`	`type`, `records`	Records per entity type (our internal type codes: INDIVIDUAL, COMPANY, ORGANIZATION, VESSEL, AIRCRAFT, UNKNOWN).
`countries`	`country`, `records`	Records per country code (nationality, or the country of the register).
`fill`	`name`, `alias`, `birthDate`, …	Records with at least one value for each property. See field fill.
`delta`	`added`, `modified`, `removed`	Change counts against the previous build, or null on a first build.
—	`licence`	A pointer to our data licence.
`files`	`name`, `type`, `path`, `bytes`, `rows`, `sha256`	Every file in the collection with its size, line count and SHA-256 checksum.

Each build also adds a line to statistics.json with the same figures, newest first, which is what the statistics page reads.

Source metadata: sources.json and the API

sources.json lists the registers that contribute to a collection, one object per register:

Field	Description
`name`	Register name, as on our source pages.
`list`	Short list name used inside records.
`country`	Country or jurisdiction of the publisher.
`records`	Records from this register in the collection.
`first_seen`	When the register was first loaded.
`last_loaded`	Most recent load.
`publisher_type`	Kind of source, as an internal code (for example `SANCTIONS_LIST`, `PEP_REGISTER`, `DEBARMENT_LIST`, `COURT_PORTAL`).
`cadence_hours`	How often the source is meant to be refreshed, in hours.
`source_url`	The publisher’s page or feed.

The same details, with more figures, are available per register from /api/source?name= and for all registers from /api/sources; see the endpoint reference.

Fields we do not publish yet: a publisher’s organization name and whether it is an official body (data backlog item 20), a deprecated flag for stopped sources, and a controlled list of refresh frequencies.

List types

Instead of free-form tags, every register has one list type. It decides which collections a record belongs to and which topic it gets.

List type	Meaning	Topic	Collections
`SANCTION`	Sanctions, asset-freeze, terrorist and similar designation lists.	`sanction`	Crown Default, Consolidated sanctions
`PEP`	Registers of politically exposed persons and public office holders.	`role.pep`	Crown Default, Politically exposed persons
`DEBARMENT`	Exclusions from public contracts, health programmes and regulated professions.	`debarment`	Crown Default, Debarment and enforcement
`TAX_WARRANT`	Tax warrants and similar public enforcement notices.	`reg.action`	Crown Default, Debarment and enforcement
`INSOLVENCY`	Published insolvency proceedings.	`reg.action`	Crown Default, Debarment and enforcement

The maritime collection is not a list type: it holds records typed as vessels or carrying an IMO number, from any list type. Some registers carry the wrong list type today (data backlog item 33).

Terms used here are defined in the glossary.

Last updated 2026-10-04