Hash Verification API

The Hash Verification API provides platforms with an interface to check if any content uploaded by their users has already been identified as terrorist content by the TCAP. Platforms query the API with a list of up to 50 hash strings corresponding to files from their own systems. In turn, they receive a response indicating if any of those files are, or are likely to be, terrorist content.

What is the Hash Verification API

The API provides an endpoint that users can query with their own hash strings. These hash strings are then checked for exact or partial matches against the TCAP repository of hashed terrorist content and a response is returned indicating the results of these checks.

What hash algorithms are provided

We currently provide exact cryptographic hash-matching for the MD5, SHA256 and SHA512 algorithms as well as partial, or perceptual, hash matching for the PDQ (image) and TMK (video) algorithms.

How to use the Hash verification API

In order to use the API, you need to hash any content you wish to check against our archive. In the case of MD5, SHA256 and SHA512 this is fairly straightforward, however for TMK and PDQ you will likely have to fork Threat Exchange's code.

Your requests

You can send a json list of items, where an item is of the following form for cryptographic hashes:

POST https://beta.terrorismanalytics.org/hash-verification/api/
{
  "body":[ 
    {
        "hash_value": "YOUR_HASH_STRING", 
        "hash_type": <"MD5" | "SHA256" | "SHA512">
    },
    ...
  ]
}
and the following for perceptual_hashes:
POST https://beta.terrorismanalytics.org/hash-verification/api/
{
  "body":[ 
    {
      "hash_value":  < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
      "hash_type": <"TMK" | "PDQ">
      "confidence": 0-1
    },
    ...
  ]
}

How many items can I send in a request?

For non-TMK hash verification requests, you can send up to 50 items in a list.

For TMK hash-verification requests, we recommend sending a list of two at most. This is because, for security reasons, the API does not currently allow for ingesting larger request bodies (and binary encoded TMK files are quite sizeable). You can try to send more, but the request will likely error -- at no risk to your systems or our own.

Our responses

After receiving your list of items, we will match each one against our system and return a list with the same number of items, where an item is the following for cryptographic hashes (where True represents a match against content in our archive):

200 OK
{
    "hash_value": "YOUR_HASH_STRING",
    "hash_type": <"MD5" | "SHA256" | "SHA512">,
    "result": <True/False>, 
    "error": <null | error message>
}
and the following for perceptual_hashes:
200 OK
{
    "hash_value":  < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
    "hash_type": <"TMK" | "PDQ">,
    "result": <True/False>, 
    "confidence": <0-1 | [0-1, 0-1], 
     "error": <null | error message>
}

What are confidence scores?

While cryptographic hash verification simply involves checking your hash string for an exact match against our system, perceptual hashes check for similarity between two hashes and their respective pieces of content.

With this in mind, when you request a check on a cryptographic hash, you must include a confidence parameter between 0 and 1 where 1 is complete confidence, ie. an exact match.

We will return True in the result field if we have a piece of content whose hash matches or exceeds the confidence threshold you provided when compared with your hash_value. If we do not have any confidence that matches or exceeds this threshold, we will return False.

In addition, if the result is True, we will return a confidence score that represents just how similar the piece of content on our system was to your piece of content. For PDQ hashes, this score will a decimal between 0 and 1, where 1 is an exact match. For TMK hashes, this score will be a pair of decimals between 0 and 1, where [1,1] is an exact match.