# Authentication

To use some TCAP API services, you will need to obtain a resource-specific, short-lived access token.

In order to get one of these, you will already need to be an on-boarded TCAP or TCAP Archive user with a username and password.

Services which currently utilise this authentication system:
- [Hash Verification API](https://terrorismanalytics.org/docs/hash-verifier-v2)

Authentication for other services is more straightforward, see [other docs](https://terrorismanalytics.org/docs)

## Obtaining an Access Token
To obtain a token make a request to the TCAP authentication endpoint with your username, password and the name of the requested service, for example `Hash Verification`:

```
POST https://beta.terrorismanalytics.org/token-auth/api/login
{
  username: YOUR_TCAP_USERNAME,
  password: YOUR_TCAP_PASSWORD,
  requested_service: NAME_OF_REQUESTED_API_SERVICE
}
```

## Using the token
Once you have received a token, you should attach it as a header to any API service request.

Typescript example:

```
    const serviceUrl = "https://beta.terrorismanalytics.org/hash-verification/api/v2"
    const headers = {'Authorization': `Bearer ${token}`};
    const response = await fetch(service_url, {headers: headers});

```

Python example:

```
    service_url = "https://beta.terrorismanalytics.org/hash-verification/api/v2"
    headers = {'Authorization': f'Bearer {token}'}
    response = requests.get(serviceUrl, headers=headers)

```

## How long does the token last?
At the moment, tokens are configured to be valid for 5 minutes. When a token expires, you will receive the following response with a status of `403`:

```
Authentication error: Token has expired. Please refresh.
```

You can request a new token anytime and continue using it to make requests to your chosen service.

If engaging with the service programmatically, we recommend writing a script to request a new token whenever you receive a response with body "Authentication error: Token has expired. Please refresh."

## Further help
Our dev team would be happy to walk you through the authentication process. 

If you wish to reach out in relation to this or anything else surrounding our API services, please contact us through [this form](https://terrorismanalytics.org/about/contact) 

Get started using the Tech Against Terrorism TCAP API. This document outlines how you can obtain an authentication token and use our API services.

Authenticating for TCAP API Services

## Two endpoints
The Submissions API consists of two endpoints for tech platform TCAP users. 

###Content endpoint
The first returns a list of urls marked as terrorist content by Tech Against Terrorism:

```
GET https://beta.terrorismanalytics.org/integrations/tech_platform_api/content
```

###Alerts endpoint
The second returns a list of alerts we have sent as well as information about the url associated with each alert:

```
GET https://beta.terrorismanalytics.org/integrations/tech_platform_api/content_alert
```

##Authentification
In order to hit either endpoint you will need to be an on-boarded TCAP user associated with a tech platform. These endpoints sit inside the main TCAP backend so you will be able to access both with a standard TCAP JWT token.

To obtain a token make a request to the TCAP authentication endpoint with your username and password.

```
POST https://beta.terrorismanalytics.org/token-auth/tcap/
{
  username: YOUR_TCAP_USERNAME,
  password: YOUR_TCAP_PASSWORD,
}
```

## Your platform's content
When  you send an authenticated request to either of these endpoints, the service will filter content or content alerts based on the tech platform associated with your email address. If your email address happens to be associated with more than one tech platform, it will return data associated with each platform. However, if you wish to refine the  search, you can include a query parameter specifying the tech platform, for example:

```
GET https://beta.terrorismanalytics.org/integrations/tech_platform_api/content_alert
?tech_platform=Platform C
```

or

```
GET https://beta.terrorismanalytics.org/integrations/tech_platform_api/content
?tech_platform=Platform C
```

##Refining your request
You can further refine your requests to either endpoint by including the following query paramaters:

- `created_on` : the date the URL was submitted to the TCAP.
- `terrorist_group`: the terrorist group that is associated to the content.

Here are some example urls:

```
GET https://beta.terrorismanalytics.org/integrations/tech_platform_api/content
?created_on=2023-06-29T10:03:54.828554

GET https://beta.terrorismanalytics.org/integrations/tech_platform_api/content_alert
?terrorist_group=Terrorist Group A


```

##Our Responses
###Content API response

The content endpoint returns the following data on each url that satisfies your query:

- `created_on` `(Date)` : Timestamp at which the URL was approved for submission to the TCAP

- `terrorist_group` `(String)`: Terrorist group associated with the content submitted to the TCAP

- `tech_platform` `(String)`: Platform where the content was collected from

- `tier` `(String)`: Tier of alert submitted for this URL (Threat To Life, Crisis, Designation, Promotional, Recommended)

- `md5_hash` `(String)`: MD5 hash encryption of the URL where the content was found

- `url_status` `("Online" | "Offline")`: Online status of the URL (online or offline)

- `extreme_content` `(Boolean)`: Whether the URL contains graphic and/or extreme content

- `pii` `(Boolean)`: Whether the URL contains personally identifiable information (PII)

- `content_alert` `(Date)`: Time/date when the tech platform was alerted for this URL


####What is the md5 hash?
The MD5 hash is a cryptographic hash of the url associated with the content. In essence it is the unique identifier of that piece of content. We are sending it in this format for security purposes.

###Content Alert API response

The content alert endpoint returns the following data on each successfully sent alert that satisfies your query:

- `content_item` `(ContentItem)`: all of the information from the Content API response above, nested

- `to` `(String)`: the email address the alert was sent to

- `alerted_date` `(Date)`: the date the alert was sent





The Submissions API provides two endpoints for tech platforms to query our database and find out what contents our OSINT team have found on their platforms as well what alerts we have issued to them. Platforms can refine their queries to specify dates and specific terrorist groups.

TCAP Submissions API

# Key features and use cases

The TCAP Archive is a repository of known terrorist or violent extremist content (TVEC) media files, including images, videos and documents. The TCAP Archive Hash List API allows platforms to ingest the hashes produced from these media files in bulk, so they can use them in their content moderation processes.

The TCAP Archive’s hashes are distinct from any existing TVEC hash lists, as they complement Tech Against Terrorism’s proactive monitoring of terrorist internet usage by its team of open-source intelligence specialists. By leveraging this expertise, alongside a suite of automated monitoring capabilities, the TCAP Archive hash list reflects content created and uploaded over a number of years by a range of violent Islamist and violent far-right terrorist entities.

# Authentication

In order to use any of the Hash List endpoints you will need to be an on-boarded Hash List TCAP user. These endpoints sit inside the main TCAP backend so you will be able to access all with a standard TCAP JWT token.

To obtain a token make a request to the TCAP authentication endpoint with your username and password.

```
POST https://beta.terrorismanalytics.org/token-auth/tcap/
{
  username: YOUR_TCAP_USERNAME,
  password: YOUR_TCAP_PASSWORD,
}
```

## Response
The Authentication endpoint returns the following data on each request:

- `token` : `String` Token to be used to on each request you make the the API as a `Bearer` token

- `user` :  `User` Your system user information



## Endpoints 

## Hash List by Ideology
`GET /api/hash-list/:ideology`

This Hash List API endpoint retrieves a hash list file filtered by a specified ideology. The API has 3 endpoints including a development endpoint.

### Parameters
#### Path Parameters
- **ideology** (Type: String, Required: Yes): Specifies the ideology to be used in generating the hash list file.

```
GET /api/hash-list/:ideology  <'islamist' | ' far-right ' | 'all (default)' >
```

Endpoint for testing your hash list integration

```
GET /api/hash-list/dev
```

#### Query Parameters
- **include_tmk** (Type: String, Required: No): `'true' | ' 1 ' | 'false (default)'` Indicates whether to include TMK hash types in the output file. 


### Response
The Hash List endpoint returns the following data on each request:

- `file_url` :  `String` The pre-signed URL to the Hash List file (JSON) expires after 5 minutes

- `file_name` : `String` Name of the file

- `created_on` :  `Date` The date the file was created

- `total_hashes` :  `Int` The number of hashes in the given file

- `ideology` : `'islamist' | 'far-right' | 'all'` The ideology made in the request, will return 'all' on the `/dev` endpoint 

### Hash List JSON File 
The hash list file will contain an array of hash objects with the following properties. 

- `id` :  `Int` Hash ID

- `hash_digest` : `String` The hash of the content for all hash types excluding TMK's in which case it will be the base64 encoded string of the hash file content 

- `algorithm` :  `"MD5" | "SHA256" | "SHA512" | "PDQ" | "TMK"` The algorithm used to generate the hash

- `ideology` : `'islamist' | 'far-right' | 'all' (default) ` The ideology associated with the original content
 
- `file_type` : `'String' ` The file type of the original content

## TMK Hash List by Ideology
`GET /api/hash-list/:ideology/tmk`

This Hash List API endpoint retrieves a hash list file but for only TMK hashes filtered by a specified ideology.

### Parameters
#### Path Parameters
- **ideology** (Type: String, Required: Yes): Specifies the ideology to be used in generating the hash list file.

```
GET /api/hash-list/:ideology/tmk  <'islamist' | ' far-right ' | 'all (default)' >
```

### Response
The Hash List endpoint returns the following data on each request:

- `file_url` :  `String` The pre-signed URL to the Hash List file (JSON) expires after 5 minutes

- `file_name` : `String` Name of the file

- `created_on` :  `Date` The date the file was created

- `total_hashes` :  `Int` The number of hashes in the given file

- `ideology` : `'islamist' | 'far-right' | 'all'` The ideology made in the request, will return 'all' on the `/dev` endpoint 

### Hash List JSON File 
The hash list file will contain an array of hash objects with the following properties. 

- `id` :  `Int` Hash ID

- `hash_digest` : `String` The base64 encoded string of the hash file content. 

- `algorithm` :  `"TMK"` The algorithm used to generate the hash

- `ideology` : `'islamist' | 'far-right' | 'all' (default) ` The ideology associated with the original content
 
- `file_type` : `'String' ` The file type of the original content

### **Usage With Metas Threat Exchange**
If you want to use the Hash List from within [Meta's Threat Exchange](https://developers.facebook.com/docs/threat-exchange/) you can create a collaboration configuration to out api, fetch and compare PDQ Image and MD5 video hashes. 

Step 1 - Install threat exchange

```
$ pip install threatexchange
```

Step 2 - Configure the default credentials

```
$ threatexchange config api tat --credentials '<TCAP_USERNAME>' '<TCAP_PASSWORD>'
```

Step 3 - Set up config

```
$ threatexchange config collab edit tat --create 'TAT'
```

Step 4 - Fetch hashes with verbose logging

```
$ threatexchange -v fetch
```

Step 5 - View dataset

```
$ threatexchange dataset
```

Step 6 - Match a piece of content

```
$ threatexchange match ~/path/to/image.jpg
```

[ThreatExchange docs](https://developers.facebook.com/docs/threat-exchange/)

### Usage Limits

The development endpoint allows for up to **50 requests per day**.

The production endpoint will allow up to **50 requests per day**.

*Usage limits apply to both the API and Threat Exchange usage*

The Hash List API provides users with a daily file containing all content hashes within the TCAP Archive database. The hash list file is updated daily, and users can submit three requests per day to retrieve the list as a single file to use in their content moderation processes.

Hash List API

## What is the Hash Verification API
The API provides an endpoint that users can query with their own hash strings. These hash strings are then checked for exact or partial matches against the TCAP repository of hashed terrorist content and a response is returned indicating the results of these checks.


## What hash algorithms are provided
We currently provide exact cryptographic hash-matching for the MD5, SHA256 and SHA512 algorithms as well as partial, or perceptual, hash matching for the PDQ (image) and TMK (video) algorithms.


## How to use the Hash verification API v2
In order to use the API, you need to hash any content you wish to check against our archive. In the case of MD5, SHA256 and SHA512 this is fairly straightforward, however for TMK and PDQ you will likely have to fork [Threat Exchange's code](https://github.com/facebook/ThreatExchange/tree/main).


### Your requests
You can send a json list of items, where an item is of the following form for cryptographic hashes:

```
POST https://beta.terrorismanalytics.org/hash-verification/api/v2
{
  "body":[ 
    {
        "hash_value": "YOUR_HASH_STRING", 
        "hash_type": <"MD5" | "SHA256" | "SHA512">
    },
    ...
  ]
}

```

 and the following for perceptual_hashes:

```  
POST https://beta.terrorismanalytics.org/hash-verification/api/v2
{
  "body":[ 
    {
      "hash_value":  < "YOUR_PDQ_HASH_STRING" | base-64 encoded TMK file>,
      "hash_type": <"TMK" | "PDQ">
      "confidence": 0-1
    },
    ...
  ]
}

```

### Filtering by ideology
You can include a query parameter, `ideologies`, if you want to restrict your search to a specific ideology. The options are `far-right` and `islamist`.

```  
POST https://beta.terrorismanalytics.org/hash-verification/api/v2?ideologies=islamist
```

If you do not include the query parameter, all content will be searched regardless of ideology.


### How many items can I send in a request?
For non-TMK hash verification requests, you can send up to 20 items in a list.

For TMK hash-verification requests, one item in a list. We may increase the number of TMKs per request in due course and, as such, have built the service to accept a list in the request.

### Our responses
After receiving your list of items, we will match each one against our system and return a list with the same number of items, where an item is the following for cryptographic hashes (where `True` represents a match against content in our archive):

- `hash_value` `(String)`: HASH_STRING

- `hash_type` `(String)`:  "MD5" | "SHA256" | "SHA512"

- `result` `(Boolean)`: True | False

- `error` `(String)` | `Null`: An error message or null if successful

and the following for perceptual_hashes:

- `hash_value` `(String)`: Your hash string

- `hash_type` `(String)`: "MD5" | "SHA256" | "SHA512"

- `result` `(Boolean)`: True | False
 
-  `confidence` `(Float)`: 0-1 | [0-1, 0-1]

- `error` `(String | Null)`: An error message or null if successful

### What are confidence scores?
While cryptographic hash verification simply involves checking your hash string for an exact match against our system, perceptual hashes check for similarity between two hashes and their respective pieces of content.

With this in mind, when you request a check on a cryptographic hash, you must include a `confidence` parameter between 0 and 1 where 1 is complete confidence, ie. an exact match. 

We will return `True` in the `result` field if we have a piece of content whose hash matches or exceeds the confidence threshold you provided when compared with your `hash_value`. If we do not have any confidence that matches or exceeds this threshold, we will return `False`.

In addition, if the result is `True`, we will return a `confidence` score that represents just how similar the piece of content on our system was to your piece of content. For PDQ hashes, this score will a decimal between 0 and 1, where 1 is an exact match. For TMK hashes, this score will be a pair of decimals between 0 and 1, where [1,1] is an exact match.

### How do I authenticate to use the service?
Please see our [authentication documentation](https://terrorismanalytics.org/docs/getting-started)

The Hash Verification API provides platforms with an interface to check if any content uploaded by their users has already been identified as terrorist content by the TCAP.

Platforms query the API with the hash strings of their media files from their own systems. In turn, they receive a response indicating if any of those files are, or are likely to be, terrorist content.

Hash Verification V2

This page contains technical documentation on how to use the Tech Against Terrorism API services. If you are interested in using our services please get in touch through [this form](https://terrorismanalytics.org/about/contact)

DOCUMENTATION

Authenticating for TCAP API Services

TCAP Submissions API

Hash List API

Hash Verification V2