Hash Verification API v2
The Hash Verification API provides platforms with an interface to check if any content uploaded by their users has already been identified as terrorist content by the TCAP. Platforms query the API with the hash strings of their media files from their own systems. In turn, they receive a response indicating if any of those files are, or are likely to be, terrorist content.
Overview
What is the Hash Verification API
The API provides an endpoint that users can query with their own hash strings. These hash strings are then checked for exact or partial matches against the TCAP repository of hashed terrorist content and a response is returned indicating the results of these checks.
Key features and use cases
What hash algorithms are provided
We currently provide exact cryptographic hash-matching for the MD5, SHA256 and SHA512 algorithms as well as partial, or perceptual, hash matching for the PDQ (image) and TMK (video) algorithms.
How to use the Hash verification API v2
In order to use the API, you need to hash any content you wish to check against our archive. In the case of MD5, SHA256 and SHA512 this is fairly straightforward, however for TMK and PDQ you will likely have to fork Threat Exchange's code.
Endpoints
Verify cryptographic hashes
You can send a json list of items, where an item is of the following form for cryptographic hashes:
Endpoint
POST /hash-verification/api/v2
Request
Headers
- Content-Type (Required: Yes): Must be
application/json
Query Parameters
- ideologies (Type: string, Required: No): <"far-right" | "islamist">
Request Body
This endpoints expects a list of items with the following fields
hash_value (Type:
string
, Required: Yes): A unique identifier representing either a PDQ hash string or a base-64 encoded TMK file.hash_type (Type: string, Required: Yes): Specifies the format of the hash_value. It indicates whether the value is a PDQ hash or a TMK file.
confidence (Type: string, Required: No): A numerical value between 0 and 1 representing the level of certainty or reliability associated with the data represented by the hash
Example Request:
POST https://beta.terrorismanalytics.org/hash-verification/api/v2
{
"body":[
{
"hash_value": "YOUR_HASH_STRING",
"hash_type": <"MD5" | "SHA256" | "SHA512">
},
...
]
}
and the following for perceptual_hashes:POST https://beta.terrorismanalytics.org/hash-verification/api/v2
{
"body":[
{
"hash_value": < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
"hash_type": <"TMK" | "PDQ">
"confidence": 0-1
},
...
]
}
Filtering by ideology
You can include a query parameter, ideologies
, if you want to restrict your search to a specific ideology. The options are far-right
and islamist
.
POST https://beta.terrorismanalytics.org/hash-verification/api/v2?ideologies=islamist
If you do not include the query parameter, all content will be searched regardless of ideology.
How many items can I send in a request?
For non-TMK hash verification requests, you can send up to 20 items in a list.
For TMK hash-verification requests, one item in a list. We may increase the number of TMKs per request in due course and, as such, have built the service to accept a list in the request.
Response
After receiving your list of items, we will match each one against our system and return a list with the same number of items, where an item is the following for cryptographic hashes (where True
represents a match against content in our archive):
Status Code: 200
- hash_value (Type: string): The calculated hash value of the data.
- hash_type (Type: string): The algorithm used to generate the hash (e.g., MD5, SHA256, SHA512).
- result (Type: boolean): Indicates whether a match was found for the provided hash.
- error (Type: string or null): An optional error message if the request failed.
Example responses
200 OK
{
"hash_value": "YOUR_HASH_STRING",
"hash_type": <"MD5" | "SHA256" | "SHA512">,
"result": <True/False>,
"error": <null | error message>
}
and the following for perceptual_hashes:200 OK
{
"hash_value": < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
"hash_type": <"TMK" | "PDQ">,
"result": <True/False>,
"confidence": <0-1 | [0-1, 0-1],
"error": <null | error message>
}
Usage examples
Typescript example:
const serviceUrl = "https://beta.terrorismanalytics.org/hash-verification/api/v2"
const headers = {'Authorization': `Bearer ${token}`};
const response = await fetch(service_url, {headers: headers});
Python example:
service_url = "https://beta.terrorismanalytics.org/hash-verification/api/v2"
headers = {'Authorization': f'Bearer {token}'}
response = requests.get(serviceUrl, headers=headers)
Frequently Asked Questions
What are confidence scores?
While cryptographic hash verification simply involves checking your hash string for an exact match against our system, perceptual hashes check for similarity between two hashes and their respective pieces of content.
With this in mind, when you request a check on a cryptographic hash, you must include a confidence
parameter between 0 and 1 where 1 is complete confidence, ie. an exact match.
We will return True
in the result
field if we have a piece of content whose hash matches or exceeds the confidence threshold you provided when compared with your hash_value
. If we do not have any confidence that matches or exceeds this threshold, we will return False
.
Please see our authentication documentation