Gavagai Explorer API Tutorial

This is a basic tutorial for using the Gavagai Explorer API.

We recommend that you get familiar with using Gavagai Explorer before you start developing with the API. Make sure that you understand the basics, such as creating projects and uploading texts, exploring and refining your project, or creating reports and applying models. All functionality in Gavagai Explorer is built on this API, so you will have a much easier time understanding the different steps if you have already seen them in the Explorer web interface.

Please note

The instructions in this tutorial contain API calls that will upload data that counts towards your verbatim quota. When signing up for Explorer, you get 200 free texts for testing purposes, which should be sufficient to complete the first four basic steps in this tutorial.

I. Getting Started

To help you with making calls to the API for testing purposes, we use Swagger UI as a Rest API Documentation tool, which allows you to easily interact with our API, through your browser. You can access the documentation for the Explorer API here.

Get an account with Gavagai Explorer at explorer.gavagai.io if you haven’t already done so. For using the Gavagai Explorer API, you will need your login credentials (email address and password).
Test that the account is setup and working properly by performing a call to the following method.
https://api.gavagai.se/explorer/v1/projects
Curl example:
curl -u username:password "https://api.gavagai.se/explorer/v1/projects"
This should give you the following response, as you haven’t created any projects yet (if you already did create some projects, you should get an overview):
[]

For support, you should consult the API documentation. If you need more support than that, you can request it from support@gavagai.io Just send an email with any issues you might have and our support group will get back to you.

II. Creating a project

To create a project, you will need a text file in one of the following formats:

JSON. For more information, click here and select "application/json" as the Request Body format.
CSV or XLSX. You can learn more about the structure of these formats here.

We will use the following request to create a new project:

https://api.gavagai.se/explorer/v1/projects

Download the csv file here: tripadvisor-hotel-monaco-san-francisco-100.csv (This is a shortened version of the Hotel Monaco TripAdvisor example on our website.)
Send a POST request to the /projects endpoint using the csv file in the body and "Content-Type: multipart/form-data" in the header. You can use the "title" query parameter to set a title for your project.
Curl example:
curl -X POST -u username:password -H "Content-Type: multipart/form-data" -F "file=@/mypath/tripadvisor-hotel-monaco-san-francisco-100.csv" "https://api.gavagai.se/explorer/v1/projects?title=Hotel%20Monaco"
This should give you a response in form of the id of the project that you just created, for example: {"id":5374}
Now send a GET request to the /projects endpoint to see some information about your recently created project (as well as all the other projects your account owns). To see information about a specific project, append its id to your URL: https://api.gavagai.se/explorer/v1/projects/{id}

For more information, you can check out the API documentation for /projects.

III. Exploring a project

To explore a project, you will send a request to the following URL:

https://api.gavagai.se/explorer/v1/projects/{id}/explore

You will also need to supply a ProjectExplorationContext containing several different properties that are needed for exploration. The context definition has to be supplied in JSON format. You can learn more about these formats here.

{
  "columnHeaderId" : 12345,
  "language" : "...",
  "ignoreTerms" : [ "...", "..." ],
  "filters" : [ {
    "value" : "...",
    "operator" : "NOT_EQUAL",
    "columnHeaderId" : 12345
  }, {
    "value" : "...",
    "operator" : "EQUALS",
    "columnHeaderId" : 12345
  } ],
  "groups" : [ {
    "title" : "...",
    "topics" : [ {
      "title" : "...",
      "terms" : [ "...", "..." ]
    }, {
      "title" : "...",
      "terms" : [ "...", "..." ]
    } ],
    "pinned" : true
  }, {
    "title" : "...",
    "topics" : [ {
      "title" : "...",
      "terms" : [ "...", "..." ]
    }, {
      "title" : "...",
      "terms" : [ "...", "..." ]
    } ],
    "pinned" : true
  } ],
  "acceptOverageToken" : "...",
  "conceptFilter" : {
    "inclusive" : true,
    "filtered" : true,
    "conceptId" : 12345
  }
}

Some of the attributes in the ProjectExplorationContext are optional, others are required. For this tutorial, we will set the attributes columnHeaderId, language, ignoreTerms, and groups (of which the first two are mandatory).

Perform a GET request to https://api.gavagai.se/explorer/v1/projects. Look at the recently created project and note down the relevant attribute values from its headers array. We want to explore the header "review", so its id is the columnHeaderId needed. Here we can also note down the language (specified according to ISO 639-1) from the suggestedLanguage attribute in case we don't already know what language our texts are written in.
Let's assume we already know that "hotel" is not a very informative term for exploring hotel reviews (all the reviews should be about this particular hotel, after all). So we add it together with its morphological variant "hotels" to the list of ignoreTerms.
Finally, we want to specify a couple of topics and terms. These are always organised in groups, and each group has to contain at least one topic. Let's add one group called "staff" with a topic titled "staff" that contains the terms "staff", "the staff", "general staff", as well as the topic "manager", containing "manager" and "the manager". Add another group titled "stay" with a topic "stay" and the terms "stay", "stayed", and "staying". We also want to pin the groups ("pinned": true) to ensure they will be noticeable.

Our final ProjectExplorationContext will look like as follows:

{
  "columnHeaderId" : your_id,
  "language" : "EN",
  "ignoreTerms" : [ "hotel", "hotels" ],
  "groups" : [ {
    "title" : "staff",
    "topics" : [ {
      "title" : "staff",
      "terms" : [ "staff", "the staff", "general staff" ]
    }, {
      "title" : "manager",
      "terms" : [ "manager", "the manager" ]
    } ],
    "pinned" : true
  }, {
    "title" : "stay",
    "topics" : [ {
      "title" : "stay",
      "terms" : [ "stay", "stayed", "staying" ]
    } ],
    "pinned" : true
  } ]
}

Send a POST request to the /explore endpoint that contains your exploration context as body and "Content-Type: application/json" as header.
Curl example:
curl -X POST -u username:password -H "Content-Type: application/json" -d '{
"columnHeaderId" : your_id,
"language" : "EN",
"ignoreTerms" : [ "hotel", "hotels" ],
"groups" : [ {
"title" : "staff",
"topics" : [ {
"title" : "staff",
"terms" : [ "staff", "the staff", "general staff" ]
}, {
"title" : "manager",
"terms" : [ "manager", "the manager" ]
} ],
"pinned" : true
}, {
"title" : "stay",
"topics" : [ {
"title" : "stay",
"terms" : [ "stay", "stayed", "staying" ]
} ],
"pinned" : true
} ]
}' "https://api.gavagai.se/explorer/v1/projects/{id}/explore"
Now, check your project and see for yourself if your project was explored by sending a GET request to https://api.gavagai.se/explorer/v1/projects/{id}/explore.

For more information, you can check out the API documentation for projects/{id}/explore.

IV. Appending more texts to your project

Appending additonal texts to your project is very similar to sending a POST request to the /projects endpoint and requires texts in one of the following formats:

JSON. For more information, click here and select "application/json" as the Request Body format.
CSV or XLSX. You can learn more about the structure of these formats here. Be aware that the file should be in the same format as the original file you used to create the project (i.e., it should have the same columns), but it should not contain a header row.

In this part of the tutorial we will make a request to the /append endpoint.

https://api.gavagai.se/explorer/v1/projects/{id}/append

Download the following csv file: tripadvisor-hotel-monaco-san-francisco-30-append.csv (This is another shortened version of the Hotel Monaco TripAdvisor example on our website, containing the last 30 texts.)
Send a POST request to the /append endpoint using the csv file in the body and "Content-Type: multipart/form-data" in the header. Use your project id as a url parameter.
Curl example:
curl -X POST -u username:password -H "Content-Type: multipart/form-data" -F "file=@/mypath/tripadvisor-hotel-monaco-san-francisco-30-append.csv" "https://api.gavagai.se/explorer/projects/{id}/append
This should give you a response in form of the id of the project that you just appended to, for example: {"id":5374}
Now send a GET request to the /projects endpoint (with the project id as a url parameter) endpoint to confirm the number of rows (which should be 130).
To make the changes show in your exploration, send another POST request to the /explore endpoint (see Gavagai Explorer API Tutorial#III. Exploring a project).

For more information, you can check out the API documentation for /append (form data). You can switch the Request Body format to "application/json" if you would like know more about the JSON Request.

V. Insufficient Credtis

If your account does not have sufficient credits to explore a project, you will need to purchase additional credits before you an explore it (please be aware that this will result in a charge to your credit card). To purchase credits, you will have to perform an extra step prior to performing an exploration.

After creating your project (see Gavagai Explorer API Tutorial#II. Creating a project), send a GET request to https://api.gavagai.se/explorer/v1/projects/{id}/explore and note the status message and state which should say "Not started" / "NOT_STARTED".
Send a POST request to https://api.gavagai.se/explorer/v1/projects/{id}/explore, as you did in Gavagai Explorer API Tutorial#III. Exploring a project.
Send a GET request to the same endpoint to see if the project was explored or not. The response will look something like this:
{
"status": {
"startTime": 0,
"progress": 100,
"message": "No credits left. You need to buy more credits",
"state": "NO_CREDITS",
"overageAmount": 213,
},
"groups": [],
"ignoreTerms": [
"hotels",
"hotel"
],
"language": "EN",
"columnHeaderId": 70075,
"filters": [],
"tones": [
"NEGATIVITY",
"POSITIVITY",
"SKEPTICISM"
],
"totalDocumentCount": 0
}
As you can see, the status message and code should now say "No credits left. You need to buy more credits" and "NO_CREDITS" respectively. You will also see that the overageAmount now has a value, which corresponds to the additional credits you need to purchase.
You can purchase the credits using the Purchase Credits endpoint. You can either purchase the exact number of credits you need or a some extra credits - the credits will never expire.
Curl example:
curl -X POST -u username:password -H "Content-Type: application/json" -d "{\"quantity\": 213}" "https://api.gavagai.se/explorer/projects/account/credits
Once you have purchase additional credits, send a POST request to the /explore endpoint with the same ProjectExplorationContext as before.
Now, check your project again to see if your project was explored by sending a GET request to https://api.gavagai.se/explorer/v1/projects/{id}/explore.

Public