URL Hash List API (v1.1.2)
The URL Hash List API provides a robust solution for retrieving all MD5 URL hashes stored within the TCAP database. This powerful tool is designed to address the challenge of ensuring that no link to a specific piece of content is overlooked.
Transforming URLs into Working Variations
At the core of this API's functionality is the innovative process of transforming each incoming URL into up to 13 potential working variations. This is achieved through the application of a sophisticated algorithm that generates these alternatives, which are then hashed using the industry-standard MD5 algorithm.
Comprehensive URL Hash Retrieval
By leveraging this API, users can effortlessly retrieve the complete collection of MD5 URL hashes from the TCAP database. This comprehensive data set ensures that even the most obscure or hidden links are surfaced, enabling users to maintain a complete and up-to-date record of all terrorist-content-related URLs.
By harnessing the capabilities of the URL Hash List API, you can unlock the full potential of your content, ensuring that no link is left behind and that every piece of information is readily accessible.
In order to use the URL Hash List you will need to be an on-boarded URL Hash List TCAP user. If you're interested in working with us please get in touch.
Authentication
To obtain a token make a request to the TCAP authentication endpoint with your username and password.
POST https://beta.terrorismanalytics.org/token-auth/tcap/
{
username: YOUR_TCAP_USERNAME,
password: YOUR_TCAP_PASSWORD,
}Response
The Authentication endpoint returns the following data on each request:
token: String - Token to be used to on each request you make the the API as a Bearer token
user: User Your system user information
URL Hash List
GET https://beta.terrorismanalytics.org/core/url-hash-list
headers = {
"Authorization": "Bearer {AUTH_TOKEN}"
...
}from="<YYYY-MM-DD>"
to="<YYYY-MM-DD>"
limit=<INT> (maximum 1000)
offset=<INT>
ideology=< "islamist | far-right" > (defaults to all if not provided)
Response
The Hash List endpoint returns a paginated response of URL Hash objects, where each object contains a collection of URL hash digests.
Fields"count" the total number of hashes in the archive when the request was made
"next" endpoint to retrieve the next page
"previous" endpoint to the previous page
"results" list of hashes with all variations:
Example response
200 OK
{
"count": 1169,
"next": "https://beta.terrorismanalytics.org/core/url-hash-list?<QUERY PARAMS>",
"previous": "https://beta.terrorismanalytics.org/core/url-hash-list?<QUERY PARAMS>",
"results": [
{
"hash_digest": "<MD5 HASH DIGEST>",
},
......
]
}