Articles · Bulk data, Formats

Every collection, six ways: our new download formats

Published By Crown Intelligence team

Each Crown Intelligence collection is now published in six formats, from a plain list of names to a statement-level CSV that records where every value came from. Here is what each file is for, and how to check you have the right one.

Why more than one file#

People use screening data in very different ways. A payments team wants a fast first pass over names. An investigator wants a graph of people, companies and the sanctions attached to them. An auditor wants to know where each value came from and when it was first seen. One file cannot serve all three well, so every collection is now built into six.

The six formats#

  • names.txt: every distinct name and alias, one per line. Use it for a quick first-pass match.
  • targets.simple.csv: one row per record in a flat table, using the widely used simple-CSV column layout so existing loaders work unchanged.
  • targets.nested.json: each record with its sanction details nested inside, one per line.
  • entities.ftm.json: every record as a FollowTheMoney entity with typed properties, for graph and entity stores.
  • senzing.json: one Senzing record per line, ready for Senzing entity resolution.
  • statements.csv: one row for every property value, with its source and the dates it was first and last seen. This is the file for audits and change analysis.

The build index#

Alongside the files, each collection publishes index.json: the build version, the collection's figures, its entity types and countries, how often each field is filled, and every file with its size, row count and SHA-256 checksum. Compare the checksum after downloading, for example:

shasum -a 256 targets.simple.csv

A catalogue of all collections is at /downloads/collections/index.json.

Staying current#

From the second build on, each collection also publishes delta.json: one ADD, MOD or DEL operation per line, so a local copy can be brought up to date without downloading everything again. If you miss a build, download the full files instead.

Like what we are building?Get the newsletter, follow the RSS feed, read the documentation or write to us.