Hash Verification API v2

The Hash Verification API provides platforms with an interface to check if any content uploaded by their users has already been identified as terrorist content by the TCAP. Platforms query the API with the hash strings of their media files from their own systems. In turn, they receive a response indicating if any of those files are, or are likely to be, terrorist content.

What is the Hash Verification API

The API provides an endpoint that users can query with their own hash strings. These hash strings are then checked for exact or partial matches against the TCAP repository of hashed terrorist content and a response is returned indicating the results of these checks.

What hash algorithms are provided

We currently provide exact cryptographic hash-matching for the MD5, SHA256 and SHA512 algorithms as well as partial, or perceptual, hash matching for the PDQ (image) and TMK (video) algorithms.

How to use the Hash verification API v2

In order to use the API, you need to hash any content you wish to check against our archive. In the case of MD5, SHA256 and SHA512 this is fairly straightforward, however for TMK and PDQ you will likely have to fork Threat Exchange's code.

Your requests

You can send a json list of items, where an item is of the following form for cryptographic hashes:

POST https://beta.terrorismanalytics.org/hash-verification/api/v2
{
  "body":[ 
    {
        "hash_value": "YOUR_HASH_STRING", 
        "hash_type": <"MD5" | "SHA256" | "SHA512">
    },
    ...
  ]
}

and the following for perceptual_hashes:

POST https://beta.terrorismanalytics.org/hash-verification/api/v2
{
  "body":[ 
    {
      "hash_value":  < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
      "hash_type": <"TMK" | "PDQ">
      "confidence": 0-1
    },
    ...
  ]
}

Filtering by ideology

You can include a query parameter, ideologies, if you want to restrict your search to a specific ideology. The options are far-right and islamist.

POST https://beta.terrorismanalytics.org/hash-verification/api/v2?ideologies=islamist

If you do not include the query parameter, all content will be searched regardless of ideology.

How many items can I send in a request?

For non-TMK hash verification requests, you can send up to 20 items in a list.

For TMK hash-verification requests, one item in a list. We may increase the number of TMKs per request in due course and, as such, have built the service to accept a list in the request.

Our responses

After receiving your list of items, we will match each one against our system and return a list with the same number of items, where an item is the following for cryptographic hashes (where True represents a match against content in our archive):

200 OK
{
    "hash_value": "YOUR_HASH_STRING",
    "hash_type": <"MD5" | "SHA256" | "SHA512">,
    "result": <True/False>, 
    "error": <null | error message>
}

and the following for perceptual_hashes:

200 OK
{
    "hash_value":  < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
    "hash_type": <"TMK" | "PDQ">,
    "result": <True/False>, 
    "confidence": <0-1 | [0-1, 0-1], 
     "error": <null | error message>
}

What are confidence scores?

While cryptographic hash verification simply involves checking your hash string for an exact match against our system, perceptual hashes check for similarity between two hashes and their respective pieces of content.

With this in mind, when you request a check on a cryptographic hash, you must include a confidence parameter between 0 and 1 where 1 is complete confidence, ie. an exact match.

We will return True in the result field if we have a piece of content whose hash matches or exceeds the confidence threshold you provided when compared with your hash_value. If we do not have any confidence that matches or exceeds this threshold, we will return False.

In addition, if the result is True, we will return a confidence score that represents just how similar the piece of content on our system was to your piece of content. For PDQ hashes, this score will a decimal between 0 and 1, where 1 is an exact match. For TMK hashes, this score will be a pair of decimals between 0 and 1, where [1,1] is an exact match.

How do I authenticate to use the service?

Please see our authentication documentation