Database Service Documentation¶

The database service is composed of two main parts. A local database (using SQLite) and an external repository of data (Pastebin). The local database is very simple, it only saves entries of the shape (app_id, date, category, URL), where app_id tells the database who is accessing it, the date is the date for which we want to get/create a category, and category is the key we’re interested in.

All requests to databse must be authenticated with an app_id parameter. This allows the database to return the values for said application. Basically, the database service keeps a (key : value) collection for a specific application.

When a request is made to the database, for a specifc combination of [app_id, date, category] the service will obtain the pastebin URL (if any) and it will make a request to this URL and obtain the actual data the user is interested in. We can think that pastebin is our actual data repository and the local database is only an index.

Required Authentication¶

The authentication steps in the database service are two, one made by the user (when specifying the app_id parameter), and the other is made by the service when it authenticates with pastebin so that it can create pastes.

Endpoint Reference Table¶

Resource	Operation	Description
Create Entry	POST /records/(date)/categories	Creates entry for date+category combination.
Get Categories for Date	GET /records/(date)/categories	Returns list of categories for which we have information for a date.
Get Category Information for Date	GET /records/(date)/categories/(category)	Returns information we have saved for a category on a specific date.
Get Records	GET /records	Returns list of dates in database.
Update Entry	PUT /records/(date)/categories/(category)	Updates an entry of date+category combination.

Endpoint Documentation¶

GET /records¶

Returns the records (dates) that we have information of in the database.

Example request:

GET /records HTTP/1.1
Host: http://tupini07.pythonanywhere.com
Accept: application/json

Example response:

HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json

{
    "data":
    [
        "2019-01-14",
        "2019-01-15",
        "2019-01-16",
        "2019-01-17"
    ]
}

Query Parameters:
	app_id – this is the id of the app
Status Codes:	422 Unprocessable Entity – the `app_id` parameter was not provided 206 Partial Content – no records found in database 200 OK – records found

GET /records/(date)/categories/(category)¶

Get the entry we have saved for a specific category on a specific date. This endpoint will first check if we have a corresponding entry for the combined (date, category) and if yes then it will attempt to pull the information from the corresponding pastebin page.

On some occasions Pastebin will complain saing that a paste is SPAM and a captcha has to be filled to see the paste’s content. In this case the endpoing will just return the pastebin URL and it will be up to the user to go to the URL, solve the captcha, and see the paste.

Note that the content returned for the category is completely arbitrary, and is up to the creator of said content to specify it.

Example request:

GET /records/2019-01-16/categories/ebooks HTTP/1.1
Host: http://tupini07.pythonanywhere.com
Accept: application/json

Example response: (Note: this result has been shortened to save space. Normally the result will contain 7 entries insted of 2)

HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript

{
    "data": `
    Title: Accidental Texting: Finding Love despite the Spotl .EPUB
    Seeders: 4
    Leechers: 0

    Title: Tear You Apart by Megan Hart .PDF
    Seeders: 3
    Leechers: 0

    This data can also be found in pastebin, at the following URL: https://pastebin.com/9tggfzGT
    `
}

Query Parameters:
	app_id – this is the id of the app category date – the date which we’re interested in category – the category we’re interested in
Status Codes:	422 Unprocessable Entity – the `app_id` parameter was not provided 422 Unprocessable Entity – the date parameter is not properly formatted 500 Internal Server Error – pastebin is asking for captcha verification (only pastebin URL is returned in this case) 206 Partial Content – no records found in database 200 OK – records found

PUT /records/(date)/categories/(category)¶

Allow us to update an entry for the specified date and category, with the specified content (which is passed as parameter in the request body).

Basically what is done is that a new paste in pastebin is created and the URL associated with the record gets updated to the URL of the new paste.

Example request:

PUT /records/2019-01-17/categories/movies HTTP/1.1
Host: http://tupini07.pythonanywhere.com
Accept: application/json

{
    "content": "some new content to replace the old content",
}

Example response:

HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json

{
    "data": "https://pastebin.com/raw/sXCVakjQ"
}

Query Parameters:
	app_id – this is the id of the app date – the date for which we want to update the entry category – the category for which we want to update the entry
JSON Parameters:
	content – this is the new content that we want to associate with the category
Status Codes:	422 Unprocessable Entity – the `app_id` parameter was not provided 422 Unprocessable Entity – the date parameter is not properly formatted 422 Unprocessable Entity – no `content` parameter is present in the request body 206 Partial Content – there is no record with the combination date+category so no update is possible 500 Internal Server Error – cannot create entry in pastebin. This happens when the service exceeds the 24h paste limit 200 OK – record updated successfully

GET /records/(date)/categories¶

Returns the list of categories for which we have information in the database for a specific date.

Example request:

GET /records/2019-01-16/categories HTTP/1.1
Host: http://tupini07.pythonanywhere.com
Accept: application/json

Example response:

HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json

{
    "data":
    [
        "ebooks",
        "movies",
        "TV-series"
    ]
}

Query Parameters:
	app_id – this is the id of the app date – the date for which we want to get the list of available categories
Status Codes:	422 Unprocessable Entity – the `app_id` parameter was not provided 422 Unprocessable Entity – the date parameter is not properly formatted 206 Partial Content – no records found in database 200 OK – records found

POST /records/(date)/categories¶

Allow us to create a new entry for the specified date and category, which will have the specified content (both content and category are passed as parameters in the request body).

Example request:

POST /records/2019-01-17/categories HTTP/1.1
Host: http://tupini07.pythonanywhere.com
Accept: application/json

{
    "content": "some test content",
    "category": "test-category"
}

Example response:

HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json

{
    "data": "some test content\n\nThis data can also be found in pastebin, at the following URL: https://pastebin.com/raw/2VTRywd3"
}

Query Parameters:
	app_id – this is the id of the app date – the date for which we want to create a new category entry
JSON Parameters:
	category – this is the name of the category which we want to create content – this is the content that we want to associate with the category
Status Codes:	422 Unprocessable Entity – the `app_id` parameter was not provided 422 Unprocessable Entity – the date parameter is not properly formatted 422 Unprocessable Entity – no `category` parameter is present in the request body 422 Unprocessable Entity – no `content` parameter is present in the request body 409 Conflict – records already exists for combination date+category 500 Internal Server Error – cannot create entry in pastebin. This happens when the service exceeds the 24h paste limit 201 Created – new record created successfully