Hash List API

The Hash List API provides users with a daily file containing all content hashes within the TCAP Archive database. The hash list file is updated daily, and users can submit three requests per day to retrieve the list as a single file to use in their content moderation processes.

The TCAP Archive is a repository of known terrorist or violent extremist content (TVEC) media files, including images, videos and documents. The TCAP Archive Hash List API allows platforms to ingest the hashes produced from these media files in bulk, so they can use them in their content moderation processes.

The TCAP Archive’s hashes are distinct from any existing TVEC hash lists, as they complement Tech Against Terrorism’s proactive monitoring of terrorist internet usage by its team of open-source intelligence specialists. By leveraging this expertise, alongside a suite of automated monitoring capabilities, the TCAP Archive hash list reflects content created and uploaded over a number of years by a range of violent Islamist and violent far-right terrorist entities.

Authentication

In order to use any of the Hash List endpoints you will need to be an on-boarded Hash List TCAP user. These endpoints sit inside the main TCAP backend so you will be able to access all with a standard TCAP JWT token.

To obtain a token make a request to the TCAP authentication endpoint with your username and password.

POST https://beta.terrorismanalytics.org/token-auth/tcap/
{
  username: YOUR_TCAP_USERNAME,
  password: YOUR_TCAP_PASSWORD,
}

Response

The Authentication endpoint returns the following data on each request:

  • token : String Token to be used to on each request you make the the API as a Bearer token

  • user : User Your system user information

Hash List API endpoints

The Hash List API includes 3 endpoint including a development endpoint:

Endpoint for testing your hash list integration

GET https://beta.terrorismanalytics.org/api/hash-list/dev

Endpoint(s) for your production system

GET https://beta.terrorismanalytics.org/api/hash-list/far-right
GET https://beta.terrorismanalytics.org/api/hash-list/islamist
GET https://beta.terrorismanalytics.org/api/hash-list/all

Response

The Hash List endpoint returns the following data on each request:

  • file_url (String): The pre-signed URL to the Hash List file (JSON) expires after 5 minutes

  • file_name (String): Name of the file

  • created_on (Date): The date the file was created

  • total_hashes (Int): The number of hashes in the given file

  • ideology ('islamist' | 'far-right' | 'all'): The ideology made in the request, will return 'all' on the /dev endpoint

Hash List JSON File

The hash list file will contain an array of hash objects with the following properties.

  • id (Int): Hash ID

  • hash_digest (String): The hash of the content

  • algorithm ("MD5" | "SHA256" | "SHA512" | "PDQ"): The algorithm used to generate the hash

  • ideology ('islamist' | 'far-right'): The ideology associated with the original content

Usage Limits

The development endpoint allows for up to 50 requests per day.

The production endpoint will allow up to 3 requests per day.