Management API

or send an email to support@doofinder.com

In this article

Doofinder's management API allows you to perform the same administrative tasks you can do on your search engines using the doofinder control panel, directly from your code.

The tasks you can currently perform are:

  • List your SearchEngines
  • Create a new SearchEngine
  • Obtain info about a particular SearchEngine
  • Update or delete any of your SearchEngines
  • Tell doofinder to start processing your data feed.
  • Obtain info about last processed feeds.
  • Obtain info about a particular processing task.
  • Obtain info about the last task of any kind you are/have perfoming/performed
  • List the types of data your search engine can store items for
  • Create a new type of data for your search engine
  • Delete a type of data along with all the items belonging to that data type CAUTION
  • Scroll through the items you have stored of a specific type
  • Add a new item of a specific type to your search engine
  • Update, create or delete an item of a specific type
  • Obtain statistics about searches performed, search results clicked and non-productive searches
  • Obtain aggregated info about requests usage

Warning: The management API has a limit of two requests per second. If you exceed this limit the server will return a 429 HTTP status code (Too many requests).

Basics

Authentication

API Key

You need to generate an API Key in order to use the management API. You can do this in the "My Account" section of your doofinder control panel.

This is one possible value for an API Key:

API Key: {search_zone}-{token}
  • {search_zone} Depending on your location, is the geographic zone your search engine is located at. i.e.: eu1. Also, indicates which host to use in your API calls.
  • {token} The authentication token you have to include in all your requests to your management API

Example of the generated API Key:

API Key: eu1-ab46030xza33960aac71a10248489b6c26172f07

API Endpoint

All requests are made using https protocol, to this host:

{search_zone}-api.doofinder.com

where {search_zone} depends on your location. {search_zone} is part of your API Key, too. i.e.: eu1

API Root

We provide an "api root" url to which you can point your browser and, after proper authentication, navigate through the api urls you have available for each of your search engines. We hope this will ease your learning process.

You can point your browser to the following location:

https://{search_zone}-api.doofinder.com/v1

Of course, you'll click on "login" and authenticate yourself.

Authentication

Doofinder's API uses Token Authentication, which means you'll have to include the following HTTP header in every request:

Authorization: Token d8fdeab7fce96a19d3fc7b0ca7a1e98b

Notice: The value of the token is the part of your API key that is not the zone, and can be obtained in your control panel and you can re-generate your API key anytime.

Warning: Don't share your authentication token or API key with anyone.

Conventions

Along most of the code samples you will find placeholders for some common variable values. They are:

  • {hashid}: The search engine's unique id. i.e.: d8fdeab7fce96a19d3fc7b0ca7a1e98b
  • {type}: When storing items, they're always stored under a certain "type of data". i.e.: product.
  • {token}: Your personal authentication token obtained in the control panel.
  • {uid}: The Id of a Doofinder User

API Objects

  • SearchEngines: Create, list, modify your SearchEngines
  • Types: The types of data your search engine can store items under
  • Items: The items that are stored under any of your search engine's types of data. These are the ones you're actually searching for when using our service.
  • Processing Tasks: Instruct our server to start processing a feed.

SearchEngines

Item attributes

A SearchEngine has a group of user-defined attributes

  • site_url
  • name
  • language
  • currency

There are some special attributes:

  • id: Unique identifier for the SearchEngine. Ready-Only attribute
  • hashid: System identifier for each SearchEngine. This will be useful for requesting a special SearchEngine. Read-Only attribute
  • user: The SearchEngine owner Id. Read-only attribute generated by the server

List SearchEngines

Returns a list of the SearchEngines of the user

GET https://{search_zone}-api.doofinder.com/v1/searchengines HTTP/1.1
AUTHORIZATION: Token {token}

curl -H 'Authorization: Token {token}' https://{search_zone}-api.doofinder.com/v1/searchengines

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "count": 2,
    "next": null,
    "previous": null,
    "results": [
        {
            "site_url": "http://www.example1.com/",
            "name": "example-1.com",
            "language": "es",
            "currency": "USD",
            "hashid": "6a96504dc173514cab1e0198af92e6e9",
            "id": 3,
            "url": "https://eu1-api.doofinder.com/v1/searchengines/6a96504dc173514cab1e0198af92e6e9",
            "account": {uid}
        },
        {
            "site_url": "http://example2.com/sales",
            "name": "example2",
            "language": "en",
            "currency": "EUR",
            "hashid": "4358f4b61d5da0be3b48abe610fff5b0",
            "id": 12,
            "url": "https://eu1-api.doofinder.com/v1/searchengines/4358f4b61d5da0be3b48abe610fff5b0",
            "account": {uid}
        }
    ]
}

Notice: The next and previous attributes, to provide you with the urls of the next paged results. Also count gives you the total number of SearchEngines.

Create a new SearchEngine

Creates a new SearchEngine for the current user. Returns resulting SearchEngine info.

POST https://{search_zone}-api.doofinder.com/v1/searchengines HTTP/1.1
Authorization: Token {token}

{
    "site_url": "http://www.example3.com/",
    "name": "Example3",
    "language": "en",
    "currency": "EUR"
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPOST \
https://{search_zone}-api.doofinder.com/v1/searchengines -d '
{
    "site_url": "http://www.example3.com/",
    "name": "Example3",
    "language": "en",
    "currency": "EUR"
}'

Response

HTTP/1.1 201 CREATED
Content-Type: application/json; charset=UTF-8

{
    "site_url": "http://www.example3.com/",
    "name": "Example3",
    "language": "en",
    "currency": "EUR",
    "hashid": "65133554e9b177ad5b8033e8aca8e1de",
    "id": 43,
    "user": {uid}
}

Notice: The id, hashid and user fields are filled automatically. You don't need to specify them.

Retrieve specific SearchEngine

Returns details of a single SearchEngine

GET https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}'  \
https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid}

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
    "site_url": "http://www.example3.com/",
    "name": "Example3",
    "language": "en",
    "currency": "EUR",
    "hashid": "65133554e9b177ad5b8033e8aca8e1de",
    "id": 43,
    "user": {uid}
}

Update a SearchEngine

Modify some data from a SearchEngine.

There are two ways to update a search engine:

  • Update all fields (full update).
  • Update some fields (partial update).

Full Update

Are made with the PUT method.

For the full update it is necessary to send all fields of search engine.

  • site_url
  • name
  • language
  • currency
PUT https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid}  HTTP/1.1
Authorization: Token {token}

{
    "site_url": "http://www.new-example3.com/",
    "name": "new_Example3",
    "language": "es",
    "currency": "USD"
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPUT \
https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid} -d '
{
    "site_url": "http://www.new-example3.com/",
    "name": "new_Example3",
    "language": "es",
    "currency": "USD"
}'

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
    "site_url": "http://www.new-example3.com/",
    "name": "new_Example3",
    "language": "es",
    "currency": "USD",
    "hashid": "65133554e9b177ad5b8033e8aca8e1de",
    "id": 43,
    "user": {uid}
}

Partial Update

Are made with the PATCH method.

Partial updates modify only some fields of a search engine.

PATCH https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid}  HTTP/1.1
Authorization: Token {token}

{
    "currency": "EUR"
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPATCH \
https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid} -d '
{
    "currency": "EUR"
}'

Delete SearchEngine

Delete an existing SearchEngine

DELETE https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' -XDELETE \
https://{search_zone}-api.doofinder.com/v1/searchengines/{hashid}

Response

HTTP/1.1 204 NO CONTENT
Content-Type: application/json; charset=utf-8

Types

A "type" is essentially a label under which a group of items of a certain kind are stored in your search engine. The most common type is "product", and if you are using doofinder for an ecommerce site you should use this type because it has special tweaks for product search. Anyway you can define as many types as you want. (Usually "product" is more than enough, though).

Type attributes

Since a type is essentially a label, it only has two attributes

  • name: The name of the type. (product) . This attribute is mandatory
  • sources: List of data feeds for this type . This attribute is optional. You may only index items through the API, and not through a data feed

List types

Returns a list of the types defined for the search engine

GET https://{search_zone}-api.doofinder.com/v1/{hashid}/types HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/types

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

[
    "product",
    "article"
]

Get Type

Returns information about a single type.

GET https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type}  HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}'  https://{search_zone}-api.doofinder.com/v1/{hashid}/{type}

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
    "name": "product",
    "sources": ["http://www.exammple.com/feed.xml"],
    "df_url": "https://{search-zone]-api.doofinder.com/v1/{hashid}/types/{type}"
}

Create type

Creates a new type for you search engine. The response is a new types listing with your new type included.

POST https://{search_zone}-api.doofinder.com/v1/{hashid}/types HTTP/1.1
Autorization: Token {token}

{
    "name": "paper",
    "sources": ["http://www.example.com/feed.xml"]
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPOST \
https://{search_zone}-api.doofinder.com/v1/{hashid}/types -d '
 {
    "name": "paper",
    "sources": ["http://www.example.com/feed.xml"]
  }'

Notice: The name of the type must not contain special characters or whitespaces

Notice: The sources parameter is optional, and refers to one or several data feeds for that type

Response

HTTP/1.1 201 Created
Content-Type: application/json

[
    "product",
    "article",
    "paper"
]

Update or Create Type

Updates an existing type. If the provided type name doesn't exist, a new type will be created.

Notice: You can only update the type's sources attribute, because name can't be changed.

PUT https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type} HTTP/1.1
Authorization: Token {token}

{
    "name": "product",
    "sources": ["http://www.modified.example.com/feed.xml", "http://newserver.com/test.txt"]
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPUT \
https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type} -d '
{
    "name": "product",
    "sources": ["http://www.modified.example.com/feed.xml", "http://newserver.com/test.txt"]
}'

Warning: The crawling and shopify types can't be updated this way.

Delete type

Delete one type and all the items stored within that type from the search engine.

DELETE https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' -XDELETE \
https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type}

Response

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=utf-8

{
    "state": "STARTED",
    "message": "Deletion of the {type} type has started.",
    "detail": "Indexing operations will be blocked until the type deletion is finished"
}

More than one type can be deleted, by either using a comma separated list, or on all types using the special parameter '_all'.

DELETE https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type1},{type2} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' -XDELETE \
https://{search_zone}-api.doofinder.com/v1/{hashid}/types/{type1},{type2}

Response

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=utf-8

{
    "state": "STARTED",
    "message": "Deletion of the {type1},{type2} type(s) has started.",
    "detail": "Indexing operations will be blocked until the type deletion is finished"
}

Delete ALL types:

DELETE https://{search_zone}-api.doofinder.com/v1/{hashid}/types/_all HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' -XDELETE \
https://{search_zone}-api.doofinder.com/v1/{hashid}/types/_all

Response

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=utf-8

{
    "state": "STARTED",
    "message": "Deletion of the {type1},{type2} type(s) has started.",
    "detail": "Indexing operations will be blocked until the type deletion is finished"
}

Warning: Handle with extreme care. There is no 'undo' for this.

Info: Type deletion involves larger internal operations, therefore is an asynchronous operation. Once you've started a type deletion, you won't be able to index or delete items during a short period of time. When the delete type operation is finished, the indexing ops will be enabled again.

Items

These are the actually searchable items in your search engine. They are always of a certain type (i.e. product)

Item attributes

An item can have any number of attributes with arbitrary names. It depends on what you defined in your data feed. Some typical attributes are:

  • title
  • description
  • price

Warning: Field name is Case Sensitive.

There are some special attributes:

  • id: Unique identifier for the item.
  • df_manual_boost: Manual search boost for the item.
  • df_indexed_text: Searchable, not shown text associated with the item.
  • df_url: This is a read-only attribute generated by the server. It's the location you can get the item details at. It's only for information purposes.

Warning: Please note that these are the only reserved words.

Get Item

Returns information about a single item.

GET https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id}  HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}'  https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id}

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
    "df_manual_boost": 1,
    "df_indexed_text": null,
    "df_url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id}",
    "link": "http://www.example.com/img/1123.png",
    "id": "{item_id}",
    "title": "my custom title",
    "categories": [
        "El ba\u00f1o",
        "Cambiador"
    ]
}

Create Item

Creates a new item. The response is a representation of the recently created item. Notice the dynamically, read-only url field that points to the item's resource location.

POST https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} HTTP/1.1
Autorization: Token {token}

{
    "id": "AAX023",
    "title": "sample product",
    "price": 23.2,
    "link": "http://www.example.com/products/sample-product.html",
    "image_link": "http://www.example.com/products/img/AAX023.png",
    "df_manual_boost": 1.23,
    "df_indexed_text": "find me, but don't see me"
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPOST \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} -d '
 {
    "id": "AAX023",
    "title": "sample product",
    "price": 23.2,
    "link": "http://www.example.com/products/sample-product.html",
    "image_link": "http://www.example.com/products/img/AAX023.png",
    "df_manual_boost": 1.23,
    "df_indexed_text": "find me, and see me"
}'

Notice: The id field is not mandatory. If you don't provide an id, the server will automatically create one.

Response

HTTP/1.1 201 Created
Content-Type: application/json

{
    "df_manual_boost": 1.23,
    "df_indexed_text": "find me, but don't see me",
    "id": "AAX023",
    "title": "sample product",
    "price": 23.2,
    "link": "http://www.example.com/products/sample-product.html",
    "image_link": "http://www.example.com/products/img/AAX023.png",
    "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AAX023"
}

Create Items in bulk

You can create up to 100 Items in a single API call. Just upload a list of objects. The response is, as expected, a representation of the recently created list of items.

POST https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} HTTP/1.1
Autorization: Token {token}

[
  {
      "id": "AAX023",
      "title": "sample product",
      "price": 23.2,
      "link": "http://www.example.com/products/sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX023.png",
  },
  {
      "id": "AAX024",
      "title": "other sample product",
      "price": 23.25,
      "link": "http://www.example.com/products/other-sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX024.png"
  }
]

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPOST \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} -d '
[
   {
      "id": "AAX023",
      "title": "sample product",
      "price": 23.2,
      "link": "http://www.example.com/products/sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX023.png",
      "df_manual_boost": 1.23,
      "df_indexed_text": "find me, and see me"
  },
  {
      "id": "AAX024",
      "title": "other sample product",
      "price": 23.25,
      "link": "http://www.example.com/products/other-sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX024.png"
  }
]'

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
  {
      "df_manual_boost": 1.23,
      "df_indexed_text": "find me, but don't see me",
      "id": "AAX023",
      "title": "sample product",
      "price": 23.2,
      "link": "http://www.example.com/products/sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX023.png",
      "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AAX023"
  },
  {
      "df_manual_boost": 1.0,
      "df_indexed_text": null,
      "id": "AAX024",
      "title": "other sample product",
      "price": 23.25,
      "link": "http://www.example.com/products/other-sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX024.png",
      "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AAX024"
  }
]

Response if some items could not be indexed

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "detail": "Some items were not indexed. ",
    "unindexed_items": {
        "XXX2": {{
            "status": "400",
            "reason": "failed to parse [date-fin-promo]",
            "type": "mapper_parsing_exception"
        }
    }
}

Notice: There is a strict 100 items limit policy. An error response with 413 status code will be returned in case more than 100 items are tried to be created in bulk.

Update or Create Item

Updates an existing item. If the provided {item_id} doesn't exist, a new item will be created.

PUT https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id} HTTP/1.1
Authorization: Token {token}

{
    "df_manual_boost": 1.23,
    "df_indexed_text": "Modified indexed text",
    "id": "{item_id}",
    "title": "sample modified product",
    "price": 23.2,
    "link": "http://www.example.com/products/sample-product.html",
    "image_link": "http://www.example.com/products/img/AAX023.png"
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPUT \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id} -d '
{
    "df_manual_boost": 1.23,
    "df_indexed_text": "Modified indexed text",
    "id": "{item_id}",
    "title": "sample modified product",
    "price": 23.2,
    "link": "http://www.example.com/products/sample-product.html",
    "image_link": "http://www.example.com/products/img/AAX023.png"
}'

Notice: If the {item_id} you provide in the REST url is not found in the indexed items , a new item will be created with that {item_id}.

Warning: For Partial updates use the PATCH method.

Warning: In case the {item_id} provided in the url doesn't match with the id field in the submitted data, or there is no id field in the submitted data, the url's {item_id} value is always used.

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "df_manual_boost": 1.23,
    "df_indexed_text": "Modified indexed text",
    "id": "{item_id}",
    "title": "sample modified product",
    "price": 23.2,
    "link": "http://www.example.com/products/sample-product.html",
    "image_link": "http://www.example.com/products/img/AAX023.png",
    "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id}"
}

Notice: If the item has been created, the response status is not 200 OK but 201 Created.

As you can see, POST and PUT requests are almost exactly the same, except that PUT requests don't allow update operations if you don't provide the {item_id}.

Update or Create Items in bulk

Updates or creates several items in one REST operation. You have to provide a list of items, and every item has to have the id attribute. If, for any of the items in the list, there is an already indexed item with the same id attribute, it will be replaced with the uploaded item. If not, a new item will be created.

PUT https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/ HTTP/1.1
Authorization: Token {token}

[
  {
      "df_manual_boost": 1.23,
      "df_indexed_text": "Modified indexed text",
      "id": "AX1",
      "title": "sample modified product",
      "price": 23.2,
      "link": "http://www.example.com/products/sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX023.png",
      "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AX1"
  },
  {
      "id": "AX2",
      "title": "second  modified product",
      "price": 22,
      "link": "http://www.example.com/products/sample-second-product.html",
      "image_link": "http://www.example.com/products/img/AAX999.png",
      "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AX2"
  }
]

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPUT \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} -d '
[
  {
      "df_manual_boost": 1.23,
      "df_indexed_text": "Modified indexed text",
      "id": "AX1",
      "title": "sample modified product",
      "price": 23.2,
      "link": "http://www.example.com/products/sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX023.png"
  },
  {
      "id": "AX2",
      "title": "second  modified product",
      "price": 22,
      "link": "http://www.example.com/products/sample-second-product.html",
      "image_link": "http://www.example.com/products/img/AAX999.png"
  }
]'

Response if some items could not be indexed

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "detail": "Some items were not indexed. ",
    "unindexed_items": {
        "badid2": {
            "status": "400",
            "reason": "failed to parse [date-fin-promo]",
            "type": "mapper_parsing_exception"
        }
    }
}

Notice: If the {item_id} you provide in some of the items in the uploaded list is not found in the indexed items , a new item will be created with that {item_id}.

Warning: For Partial updates use the PATCH method.

Warning: You'll get an error response if some of the uploaded items doesn't have the id attribute. It's mandatory.

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
  {
      "df_manual_boost": 1.23,
      "df_indexed_text": "Modified indexed text",
      "id": "AX1",
      "title": "sample modified product",
      "price": 23.2,
      "link": "http://www.example.com/products/sample-product.html",
      "image_link": "http://www.example.com/products/img/AAX023.png",
      "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AX1"
  },
  {
      "id": "AX2",
      "title": "second  modified product",
      "price": 22,
      "link": "http://www.example.com/products/sample-second-product.html",
      "image_link": "http://www.example.com/products/img/AAX999.png",
      "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/AX2"
  }
]

Partially Update Item

Update or create only some attributes of an existing item. If the provided {item_id} doesn't exist, no action is performed.

PATCH https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id} HTTP/1.1
Authorization: Token {token}

{
    "title": "sample modified product",
    "price": 23.2
}

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPUT \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id} -d '
{
    "title": "sample modified product",
    "price": 23.2
}'

Notice: If the {item_id} you provide in the REST url is not found in the indexed items , NO new item will be created with that {item_id}.

Warning: For Partial updates use the PATH method.

Warning: In case the {item_id} provided in the url doesn't match with the id field in the submitted data, or there is no id field in the submitted data, the url's {item_id} value is always used.

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "id": "{item_id}",
    "df_url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id}"
    "title": "sample modified product",
    "price": 23.2
}

Partially Update Items in bulk

Partially updates item's attributes in one REST operation. YOu have to provide a list of objects containing the attributes to be modified or added. The id attribute is mandatory. Only the uploaded attributes are modifed for every item. Every other attribute an item may have beside the ones in the PATCH operation is kept

PATCH https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/ HTTP/1.1
Authorization: Token {token}

[
  {
      "id": "AX1",
      "title": "sample modified product"
  },
  {
      "id": "AX2",
      "price": 23,
      "image_link": "http://www.example.com/products/img/AAX999.png"
  }
]

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XPUT \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} -d '
[
  {
      "id": "AX1",
      "title": "sample modified product"
  },
  {
      "id": "AX2",
      "price": 23,
      "image_link": "http://www.example.com/products/img/AAX999.png"
  }
]'

Notice: If the {item_id} you provide in some of the items in the uploaded list is not found in the indexed items , no new item will be created with that {item_id}.

Warning: You'll get an error response if some of the uploaded items doesn't have the id attribute. It's mandatory.

Warning: You'll get an error response if some of the uploaded items id doesn't exist in the index.

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
  {
      "id": "AX1",
      "title": "sample modified product"
  },
  {
      "id": "AX2",
      "price": 23,
      "image_link": "http://www.example.com/products/img/AAX999.png"
  }
]

Response if some items could not be indexed

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "detail": "Some items were not indexed. ",
    "unindexed_items": {
        "XXX2": {
            "status": "404",
            "reason": "[<hashid>@product@<md5id>]: document missing",
            "type": "document_missing_exception"
        },
        "badid2": {
            "status": "400",
            "reason": "failed to parse [date-fin-promo]",
            "type": "mapper_parsing_exception"
        }
    }
}

Notice: Only the updated attributes for every item are returned in the response.

Delete Item

Delete an existing item.

DELETE https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' -XDELETE https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/{item_id}

Response

HTTP/1.1 204 NO CONTENT
Content-Type: application/json; charset=utf-8

Delete Items in bulk

You can also delete up to 100 Items in a single API call. You must provide a list of objects each containing the id attribute. The response contains two lists of ids: those that could be successfully deleted and those that failed.

DELETE https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} HTTP/1.1
Autorization: Token {token}

[
  {
      "id": "AAX023"
  },
  {
      "id": "AAX024"
  }
]

curl -H 'Authorization: Token {token}' -H 'content-type: application/json' -XDELETE \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} -d '
[
  {
      "id": "AAX023"
  },
  {
      "id": "AAX024"
  }
]'

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "errors": [],
    "success": [
        "AAX023",
        "AAX024"
    ]
}

Notice: There is a strict 100 items limit policy. An error response with 413 status code will be returned in case more than 100 items are tried to be deleted in bulk.

List/Scroll Items

Returns a list of the items of a certain type you have indexed.

  • The response always include a scroll_id parameter.
  • If the petition is made without the scroll_id parameter, the response is the first page of results. Also the response includes the scroll_id parameter for that pagination.
  • If the petition is made with the scroll_id parameter, the response is always the next page of results corresponding to the pagination specified by that scroll_id parameter. So each time you make a request with the same scroll_id, you get consecutive results pages
  • The scroll_id is valid for 5 minutes after each request

A typical case would be.

  • Make a first request, without the scroll_id parameter, to obtain the scroll_id value that will identify that listing and the first page of results
  • Any subsequent request, is made with the scroll_id parameter, showing the following pages of the listing.

First request

GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "scroll_id": "{scroll_id}",
    "count": 24,
    "next": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}?scroll_id={scroll_id}",
    "results": [
        {
            "df_manual_boost": 1,
            "df_indexed_text": null,
            "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/020809",
            "id": "020809",
            "title": "Bolso paci bag negro red castle",
            "link": "http://www.example.com/bolso-paci-bag-negro",
        },
        {
            "df_manual_boost": 1.3,
            "df_indexed_text": null,
            "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/040231",
            "id": "040231",
            "title": "Cambiador sobremesa celeste red castle",
        }
    ]
}

Subsequent requests

GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}?scroll_id={scroll_id} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}?scroll_id={scroll_id}

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "count": 24,
    "next": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}?scroll_id={scroll_id}",
    "results": [
        {
            "df_manual_boost": 1,
            "df_indexed_text": null,
            "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/020809",
            "id": "020809",
            "title": "Bolso paci bag negro red castle",
            "link": "http://www.example.com/bolso-paci-bag-negro",
        },
        {
            "df_manual_boost": 1.3,
            "df_indexed_text": null,
            "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}/040231",
            "id": "040231",
            "title": "Cambiador sobremesa celeste red castle",
        }
    ]
}

Notice: The next attribute, to provide you with the urls of the next paged results, which, as it turns out, is the very same url. Also count gives you the total number ot items.

Pagination

The results are paginated. You can control pagination with the following request parameters:

  • rpp Results per page how many items to display per page. The default is 10

If you wanted to display results in 20 results pages, use:

https://{search_zone}-api.doofinder.com/v1/{hashid}/items/{type}?rpp=20

on the first request.

Notice: The maximum allowed value for the rpp parameter is 100.

Notice: You can only specify pagination size on the first scroll request. i.e. when you haven't specified scroll_id yet.

Statistics

Stats aggregates

This will return daily aggregated info about the following concepts:

  • requests: The number of requests (of either 'query', 'api' or 'parser' type) made.
  • searches: The number of searches made.
  • api: The number of api requests
  • parser: The number of requests used for indexing items through feed parsing
  • clicks: The number of clicks made on any search rseult.
GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/stats HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/stats

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "start": "2016-01-18",
    "end": "2016-02-01",
    "count": 15,
    "next": "https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/?page=2",
    "previous": null,
    "aggregates": [
        {
            "date": "2016-01-18",
            "searches": 233,
            "clicks": 2,
            "api": 10,
            "parser": 0,
            "queries": 600,
            "requests": 243
        },
        {
            "date": "2016-01-19",
            "searches": 98,
            "clicks": 21,
            "api": 15,
            "parser": 0,
            "queries": 243,
            "requests": 264
        },
        {
            ...
        }
    ]
}

Notice: If none specified, the date interval is the last 15 days. You can specify an alternate date interval using from and to parameters, using the YYYYMMDD format.

Notice: The next and previous attributes, to provide you with the urls of the next paged results. Also count gives you the total number ot items.

Pagination

The results are paginated. Works the same as in the list items section

Top Searches

This will return all the searches made during certain interval of time, ordered by frequency.

Notice: When these stats are requested, the system can take some time to obtain the results. You'll get responses indicating to check later while the stats are being generated. Once the stats are generated for that particular period of time, you'll get them every time you request them.

GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_searches HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_searches

Response while the stats are being processed

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=UTF-8
link: <https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_searches?>

{
    "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_searches?",
    "message": "Your stats request is being processed. Please check this url again later."
}

Response when the stats are already generated

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "start": "2016-01-18",
    "end": "2016-02-01",
    "count": 171,
    "next": "http://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_searches?page=2",
    "previous": null,
    "searches": [
        {
            "count": 295,
            "term": "lightsaber"
        },
        {
            "count": 179,
            "term": "portable x-wing"
        },
        {
            "count": 87,
            "term": "r2d2"
        },
        {
            "count": 30,
            "term": "deadthstar"
        }
    ]
}

Notice: If none specified, the date interval is the last 15 days. You can specify an alternate date interval using from and to parameters with the YYYYMMDD format.

Notice: The next and previous attributes, to provide you with the urls of the next paged results. Also count gives you the total number ot items.

Pagination

The results are paginated. Works the same as in the list items section

Top Clicked

This will return all the clicked search results during certain interval of time, ordered by frequency.

Notice: When these stats are requested, the system can take some time to obtain the results. You'll get responses indicating to check later while the stats are being generated. Once the stats are generated for that particular period of time, you'll get them every time you request them.

GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_clicked HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_clicked

Response while the stats are being processed

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=UTF-8
link: <https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_clicked?>

{
    "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_clicked?",
    "message": "Your stats request is being processed. Please check this url again later."
}

Response when the stats are already generated

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "start": "2016-01-18",
    "end": "2016-02-01",
    "count": 171,
    "next": "http://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_clicked?page=2",
    "previous": null,
    "clicked": [
        {
            "count": 29,
            "term": "Red lightsaber",
            "url": "http://www.youryedistop.com/products/443"
        },
        {
            "count": 17,
            "term": "Protocol Droid spare batteries",
            "url": "http://www.welovetech.com/products/12"
        },
        {
            "count": 87,
            "term": "r2d2"
        },
        {
            "count": 30,
            "term": "deadthstar"
        }
    ]
}

Notice: If none specified, the date interval is the last 15 days. You can specify an alternate date interval the same format than specified in "top searches" section

Pagination

The results are paginated. Works the same as in the list items section

Top Opportunities

This will return all the query terms that have been used by your visitors but didn't return proper results and were handled by our fuzzy and phonetic algorithms, ordered by frequency.

Notice: When these stats are requested, the system can take some time to obtain the results. You'll get responses indicating to check later while the stats are being generated. Once the stats are generated for that particular period of time, you'll get them every time you request them.

GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_opportunities HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_opportunities

Response while the stats are being processed

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=UTF-8
link: <https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_opportunities?>

{
    "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_opportunities?",
    "message": "Your stats request is being processed. Please check this url again later."
}

Response when the stats are already generated

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "start": "2016-01-18",
    "end": "2016-02-01",
    "count": 171,
    "next": "http://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_opportunities?page=2",
    "previous": null,
    "opportunities": [
        {
            "count": 29,
            "term": "free beer"
        },
        {
            "count": 21,
            "term": "chocolate buiscuits"
        },
        {
            "count": 21,
            "term": "neverending batteries"
        },
        {
            "count": 5,
            "term": "excewaswrfl x"
        }
    ]
}

Notice: If none specified, the date interval is the last 15 days. You can specify an alternate date interval in the same format specified in "top searches" section

Pagination

The results are paginated. They work the same as in the list items section

No Results

This will return all the searches made that haven't thrown any search result, ordered by frequency.

Notice: When these stats are requested, the system can take some time to obtain the results. You'll get responses indicating to check later while the stats are being generated. Once the stats are generated for that particular period of time, you'll get them every time you request them.

GET  https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_noresults HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' \
https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_noresults

Response while the stats are being processed

HTTP/1.1 202 ACCEPTED
Content-Type: application/json; charset=UTF-8
link: <https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_noresults?>

{
    "url": "https://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_noresults?",
    "message": "Your stats request is being processed. Please check this url again later."
}

Response when the stats are already generated

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "start": "2016-01-18",
    "end": "2016-02-01",
    "count": 171,
    "next": "http://{search_zone}-api.doofinder.com/v1/{hashid}/stats/top_noresults?page=2",
    "previous": null,
    "noresults": [
        {
            "count": 29,
            "term": "free beer"
        },
        {
            "count": 21,
            "term": "chocolate buiscuits"
        },
        {
            "count": 21,
            "term": "neverending batteries"
        },
        {
            "count": 5,
            "term": "excewaswrfl x"
        }
    ]
}

Notice: If none specified, the date interval is the last 15 days. You can specify an alternate date interval in the format specified in "top searches" section

Pagination

The results are paginated. They work the same as in the list items section

Processing Tasks

Process a Data Feed

This will request our servers to parse your data feed again. Our servers will enqueue your request and will attend it as soon as possible. In case you don't have any data feed defined, an empty index will be created anyway.

POST https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' -XPOST https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process

or

POST https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process?force=true HTTP/1.1

curl -H 'Authorization: Token {token}' -XPOST https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process?force=true

Notice: If the force parameter is true, the system will process the feed even if it hasn't changed since the last processing, or even if it does not contain items.

Response

If your task has been accepted, the server will respond with an indication of where to find details about the task progress:

HTTP/1.1 201 Created
Content-Type: application/json; charset=UTF-8
Link: <https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/{task_id}>

{
   "state": "PENDING",
   "link": "https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/{task_id}",
   "message": "Your request has been accepted for processing",
   "task_name": "process"
}

If the same kind of task is running for you, then the server will tell you so, and where to look for details about it

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Link: <https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process>

{
   "state": "WAITING",
   "link": "https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process",
   "message": "A previous process is still running. Check the link",
   "task_name": "process"
}

Get Task details

Get information about a specific task by providing its id

GET https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/{task_id} HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/{task_id}

Or you can also get information about the last running 'process' task

GET https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' https://{search_zone}-api.doofinder.com/v1/{hashid}/tasks/process

Both request has the same responses

Response

If the task was run successfully the response will look like this one:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
   "state": "SUCCESS",
   "message": "The task has successfuly finished",
   "task_name": "process"
}

In case it is still running the response will reflect that state:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
   "state": "STARTED",
   "message": "Your task is being processed",
   "task_name": "process"
}

If there was some failure during the task execution you will get something like this:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
   "state": "FAILURE",
   "message": "Hints to determine the cause of the failure",
   "task_name": "process"
}

If the server can not find info on that task, it will return:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
   "state": "NOTFOUND",
   "message": "No info on that task",
   "task_name": "process"
}

List Tasks Logs

Each time a feed is processed, a log is created with details about the process. This request asks for a list of the processing tasks logs.

GET https://{search_zone}-api.doofinder.com/v1/{hashid}/logs HTTP/1.1
Authorization: Token {token}

curl -H 'Authorization: Token {token}' https://{search_zone}-api.doofinder.com/v1/{hashid}/logs

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

[
    {
        "date": "2013-07-16T11:02:02.080",
        "level": "WARNING",
        "msg": "Item does not have key field: 'id' at line number: 2"
    },
    {
        "date": "2013-07-16T11:02:02.080",
        "level": "WARNING",
        "msg": "Item does not have key field: 'id' at line number: 2"
    }
]

Tasks messages

Some tasks messages can give you a hint.

Feed is empty

This error happens when, after processing a feed, the system detect that no items have been parsed (i.e. the feed was "empty"). You can override this error by using the force parameter when starting a processing task.

No Changes in Feeds

When the system detects that a feed hasn't changed since the last processing task, it won't parse it again, because supposedly the resulting index is not to change. You can override this behavior by using the force parameter when starting a processing task.

API Errors

If your response comes with a status code different from 2xx, something went wrong. In most cases, a short message should give you a hint on what's going on. Here you can find a more explanatory description of some of the possible error messages.

401 status code: Authentication Errors

These errors are a classic! and they means you're not authenticated yourself to the API. Please refer to the authentication section

Possible messages:

  • Autentication credentials where not provided

429 status code: Requests Limit

This error response is obtained when you're surpassed your monthly allowed requests limit and try to make any further API operation. Friendly remainder: every API request counts as 1 request of the ones you have in your plan.

Possible messages:

  • Requests Limit Reached

409 status code: Conflict

This error response indicates you're trying to perform an API op that may conflict with some setting in your account, or an operation that can't be performed right at that moment

Possible messages:

  • Indexing operation in progress: There's an index being created right now, so no extra indexing operations are allowed until the new index is created.

  • Undefined type. Please create it first: You're trying to create, delete or update an item within a non existant type. You should create the type first.

  • The type is already created: The type you're trying to create already exists.

  • You can not modify the <type> type using the API: The feed definition binded to that type is one that doesn't allow type modification.

  • Some items don't have required 'id' field: When updating items in bulk, every item needs to have an 'id' attribute.

413 status code: Too many items

Doofinder management API allows bulk operations for indexed items (insert, update or delete indexed several items in one single API request). However no more than 100 items can be updated/inserted/deleted in one single API operation. This error response indicates you're trying to do a bulk operation with more than 100 items per request.

Possible messages:

  • Too many items in bulk operation

406 status code: Wrong Request Data

The operation you're trying to perform is "legit", but the request data you're providing is wrong for that operation.

Possible messages:

  • Please specify some item to index: You're trying to perform a single item index operation, but you've not specified any item.

  • There are no items to add: You're trying to performa bulk item index operation, but you're not specified any items.

  • put/post operation needs proper JSON to be submitted: You need to post a proper JSON. i.e. '{"name": "test"}' , '{}', '["abc", "def"]' (mind the double quotes)

  • Bulk operatio needs a list of items: You're not submitting a proper list for a bulk operation

  • Single item op attempted with a list of items: You've submitted a list of items for a single-item operation.

  • A list of ids should be provided: When performing a bulk delete operation you need to provide the ids of the items to be deleted.

status code 405: Not Allowed method

During some maintenance tasks, some HTTP methods may be temporary not allowed. This is the response you get in such cases.

Possible messages:

  • That operation is not allowed

status code 400: Unindexed Items

This error response shows up when there are some items that haven't been indexed during a bulk operation

Possible messages:

  • Some items where not indexed.