Search API (v5)

or send an email to support@doofinder.com

In this article

Doofinder comes with a simple yet powerful API to perform searches.

You can also control Doofinder's features with a number of parameters you send along with the GET request.

We have developed some client libraries which you can use:

Or you can simply write your own, implementing our API which is explained below.

Show me the money!

You want to search for something. You want it quick and you want it accurate. Just provide the search engine's id ({hashid}) and the search term.

GET http://eu1-search.doofinder.com/5/search?hashid={hashid}&query=some+query HTTP/1.1

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "query_counter":1,
  "results_per_page":12,
  "page":1,
  "total":31,
  "query": "some query",
  "hashid": "{hashid}",
  "max_score":1.1112283,
  "results":[
    {
      "description": "Antena. 5.2 dBi. omnidireccional…",
      "dfid": "523093f0ded16148dc005362",
      "title": "Cisco Aironet Pillar Mount Diversity Omnidirectional Antenna",
      "url": "http://www.example.com/product_description.html",
      "image_url": "http://www.example.com/images/product_image.jpg",
      "type": "{type}",
      "id": "{item_id}"
    },
    {
      "description": "Teclado. USB. España…",
      "dfid": "…",
      "…"
    },
    {"…": "…"},
    {
      "description": "Pila para cámara de vídeo portátil…",
      "dfid": "5230943aded16148dc005c52",
      "title": "Canon BP 514",
      "url": "http://www.example.com/alt_product_description.htm",
      "image_url": "http://www.example.com/images/alt_product_image.jpg",
      "type": "{type}",
      "id": "{item_id}"
    }
  ],
  "query_name": "fuzzy"
}

As you can see, the results attribute of the returned json contains an array with the search results. There is also some other search-related info, we'll go through that later

Notice: This sample response doesn't contain facets data.

Basics

You have a bunch of indexed data into Doofinder search engine. Let's call that data indexed items. They are items (products, articles, web pages content, etc…) that you now have indexed into your Doofinder search engine.

This api will let you search through those indexed items .

All search request are made issuing GET requests to a URL (both HTTP and HTTPS protocols supported) like this:

http://eu1-search.doofinder.com/5/search

The name of the search server may vary depending on your location and it usually is something like:

<zone>-search.doofinder.com

Authorization

Every Search API call must be authorized. There are two ways to authentify the requests.

  • Origin: This is used when the request is performed from a browser (from the client side). This header is filled in by the browser with the url you are requesting from and you must configure this url in your Allowed Domains whitelist (Security section inside SearchEngine Configuration in your Doofinder Admin Panel). By default your site url is configured as Allowed Domain.
  • Authorization: When you make a request from your server or device app, you must authentify it with the secret token you will find as part of your API Key (you can generate an API Key from My Account section in your Doofinder Admin Panel). When this header is sent you must use HTTPS protocol. There are two types of API key: search and management. For search only is recommended to use a search API key because it has search-only privileges.

cURL Example

Suppose that your search API key is:

eu1-14f327ad4389828d22f25bdb78dfff9b027a445d

That means:

  • Zone: eu1
  • Token: 14f327ad4389828d22f25bdb78dfff9b027a445d

And suppose that your search engine's hashid is:

16ddffffcf151caa1234caaa89d02374

This would request the first 20 gloves found in the index:

curl -XGET -H 'Authorization: 14f327ad4389828d22f25bdb78dfff9b027a445d' \
'https://eu1-search.doofinder.com/5/search?hashid=16ddffffcf151caa1234caaa89d02374&rpp=20&page=1&query=gloves'

So you need at least a token, a zone, a hashid and a query:

curl -XGET -H 'Authorization: <token>' \
'https://<zone>-search.doofinder.com/5/search?hashid=<hashid>&rpp=20&page=1&query=<query>'

Conventions

Along most of the code samples you will find placeholders for some common variable values. They are:

  • {hashid}: The search engine's unique id. i.e.: d8fdeab7fce96a19d3fc7b0ca7a1e98b
  • {type}: The type of the data you're searching at. i.e.: 'product'
  • {item_id}: Unique identificator of an indexed item within its type . i.e.: AOX11302

For the search zone we will always use eu1. Remember to replace it in your tests by the proper value (you will find it at the beginning of your API key).

The Request

Quite a few things in Doofinder' search are controlled through request parameters

Basic parameters

  • {hashid}: identificator of the search engine . Mandatory
  • query: the search term. must be escaped Mandatory
  • rpp: "results per page" how many results per page to return. it can't be bigger than 100
  • page: page number of the results to return
  • transformer: the transformation to apply to the results. If none specified, the results shall have the same fields the indexed items have. The one most commonly used is basic.

Notice: You can't fetch more than 1000 search results in total, so rpp*page must always be less than 1000

Available transformers

With the transformer parameter, you can choose between two transformers:

  • basic (default) : Optimized to be used by our instant results search layer. Returns id, title, description, link, image_link fields.
  • onlyid: Optimized to provide very light results. returns only the id field.

Dark magic parameters

  • type: the type of the items to search for. i.e.: 'product' or 'page'. you can use multiple values with this.
  • timeout: if you want to specify a timeout for Doofinder to close the connection with the client afterwards
  • jsonp: If you want the results to be wrapped in a function. instead of returning {"query": "some query", "results":[], ..}, return someFunction({"query": "some query", "results":[], ..})
  • query_name: Doofinder tries several types of querying the index to get the better results possible. Use this to instruct it to only use the "type of querying" specified by this parameter. Click here to see the available values for this parameter
  • nostats: when present in any search request, this search won't compute in stats reports.

Filter parameters

This is tricky somehow. Please bear with me:

Doofinder expects the filter parameter grouped like this:

filter: {
    color: ['blue', 'red'],
    size: ['M', 'S'],
    price: {
        gte: 4.36,
        lt: 99
    }
}

For the range filters you can use the following:

  • gte Greater than or equal
  • lte Less than or equal
  • gt Greater than
  • lt Less than

Notice: When you're applying filters, you need to make sure Doofinder uses the same query_name all the time, so your results are indeed "filtered". To help you with this, the response comes with the query_name property too, so you can instruct Doofinder to use that query_name only. Bear in mind that query_name makes a first filter on how the search term are matched against the index and afterwards filters are applied.

Writing that down in request parameters language is:

filter[color]=blue&filter[color]=red&filter[size]=M&filter[size]=S&filter[price][gte]=4.36&filter[price][lt]=99

Notice: The [ and ] characters should be escaped (%5B and %5D)

By default the filters make an AND query when they are for different fields, and an OR query when they are for the same field. So, the previous filter would be read as: Search for items which (color is blue OR color is red) AND (size is M OR size is S) AND (price is greater than or equal to 4.36 AND price is less than 99).

If you want to make AND queries into the same field, just add the following parameter to the url: filter_execution=and. Adding this parameter the query would be read as: Search for items which (color is blue AND color is red) AND (size is M AND size is S) AND (price is greater than or equal to 4.36 AND price is less than 99).

Go the distance: Geo Distance filter

If you have items with geographical position data, you can also filter results by the distance of the items to a certain origin. You need to prepare your data a little bit for that:

Tell Doofinder about your field.

You need to tell Doofinder which fields are expected to contain positional data. You can do that by specifing that field as as a Geo point field in your Doofinder Admin Panel at the configuration / advanced settings / field types section.

Make sure it has the right format.

Let's say 41.12 is the lattitude and -71.34 is the longitude. Your positional field (say its named "position" should have the following format:

  • API:
 {
    "id": "1",
    "title": "my title",
    "position": "41.12,-71.34"
  }
  • xml file:
  <item>
    <id>1</id>
    <title>my title</title>
    <position>41.12,-71.34</position>
  </item>
  • text file:
  id|title|position
  1|my title|41.12,-71.34

Go filter!

Filter format is quite simple. Say you want to filter red or blue items which are within 200 km from the (40, -70) coords.

filter: {
    color: ['blue', 'red'],
    geo_distance: {
        distance: "200km",
        position: "40,-70"
    }
}

Notice: Both geo_distance keys are mandatory, the distance and the name of your Geo point field.

Writing that down in request parameters language:

filter[color]=blue&filter[color]=red&filter[geo_distance][distance]=200km&filter[geo_distance][position]=40,-70

Notice: The query_name and the [,] notices are to be considered, too.

Exclusion parameters

As you can apply normal filters, you can exclude items using negative filters:

Doofinder expects the exclude parameter grouped like this:

exclude: {
    color: ['blue', 'red'],
    size: ['M', 'S'],
    price: {
        gte: 4.36,
        lt: 99
    }
}

For the range filters you can use the following:

  • gte Greater than or equal
  • lte Less than or equal
  • gt Greater than
  • lt Less than

Notice: When you're applying negative filters, you need to make sure Doofinder uses the same query_name all the time, so your results are indeed "filtered". To help you with this, the response comes with the query_name property too, so you can instruct Doofinder to use that query_name only.

Writing that down in request parameters language is:

exclude[color]=blue&exclude[color]=red&exclude[size]=M&exclude[size]=S&exclude[price][gte]=4.36&exclude[price][lt]=99

Notice: The [ and ] characters should be escaped (%5B and %5D)

By default the negative filters make an OR query when they are for different fields, and an OR query when they are for the same field. So, the previous filter would be read as: Search for items which (color is not blue OR color is not red) OR (size is not M OR size is not S) OR (price is not greater than or equal to 4.36 OR price is less than 99).

If you want to make AND queries for exclude filters in different fields, just add the following parameter to the url: exclude_execution=and. Adding this parameter the query would be read as: Search for items which (color is not blue OR color is red) AND (size is not M OR size is not S) AND (price is not greater than or equal to 4.36 AND price is not less than 99).

Go the distance: Geo Distance negative filter

As you can filter by geo location, the same you can exclude a circle around a point with a determined radius. Look at the Geo Distance Field

Excluding by geo-location

Filter format is quite simple. Assuming location is a Geo point field, say you want to exclude red and blue items which are within 200 km from the (40, -70) coords.

exclude: {
    color: ['blue', 'red'],
    geo_distance: {
        distance: "200km",
        location: "40,-70"
    }
}

Notice: Both geo_distance keys are mandatory.

Writing that down in request parameters language:

exclude[color]=blue&exclude[color]=red&exclude[geo_distance][distance]=200km&exclude[geo_distance][position]=40,-70

Notice: The query_name and the [,] notices are to be considered, too.

Sort parameters

Also tricky. What a happy hour we're having!

You can command Doofinder to return results sorted by certain fields. Doofinder expects sort parameters like any of these:

sort: {price: 'asc'}
sort: price
sort: [{price: 'asc'}, {brand: 'desc'}]

Notice: _score is a special field name that represents the "quality" Doofinder has assigned to that result. The higher the score, the best result it is.

Of course, you need to send this as request parameters:

sort[price]=asc
sort=price
sort[0][price]=asc&sort[1][brand]=desc

Geo Distance Sorting

Assuming location is a field of type Geo point:

sort[_geo_distance][location]=40,-70
sort[_geo_distance][order]=asc

The Response

You've already have a glimpe of a basic response. You know it's a json object with a bunch of properties. Here are some more details to whet your appetite:

Basic response properties

  • page: Page number of the results being returned in this response
  • total: The total number of results obtained
  • max_score: For the best result Doofinder could find, this is a numeric estimation of "how good" it is according to Doofinder.
  • query_name: In order to get the best possible results, Doofinder tries several types of querying. This is the type of the query Doofinder made to obtain these results.
  • query: The search term
  • results_per_page: How many results shown per page.
  • results: List of search results

Response with facets

When you have facets defined in your search engine, along with the response comes information about different groupings that can be made for certain fields in the search results.

This info comes in the response json, in the facets property:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
    "query":"chair",
    "total":"6",
    "..":"..",
    "results":[
        {"…":"…"},
        {"…":"…"}
    ],
    "facets":{
        "color":{
            "doc_count": 6
            "missing": {"doc_count": 1},
            "terms":{
              "buckets": [
                {"key":"Green","doc_count":2},
                {"key":"Blue","doc_count":2},
                {"key":"Red","doc_count":1},
                {"key":"Yellow","doc_count":1}
               ],
               "doc_count_error_upper_bound": 0,
               "sum_other_doc_count": 1
             }
        },
        "brand":{
            "doc_count":6,
            "missing": {"doc_count": 0},
            "terms": {
              "buckets": [
                {"key":"Red castle","doc_count":5},
                {"key":"Maclaren","doc_count":1}
              ],
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0
            }
        },
        "price":{
            "doc_count": 6,
            "range":{
               "buckets": [
                {   "doc_count": 6,
                    "key": "0.0-*",
                    "from": 0,
                    "from-as-string": "0.0",
                    "stats": {
                        "from":0,"count":6,
                        "min":30,"max":94.9000015258789, "avg":84.0833346048991
                    }
                }
              ]
            }
        },
        "categories":{
            "doc_count":12,
            "missing": {"doc_count": 0},
            "terms": {
               "buckets": [
                {"key":"walking chairs","doc_count":6},
                {"key":"Bags walking chairs","doc_count":5},
                {"key": "Blankets, covers","doc_count":1}
               ],
               "doc_count_error_upper_bound": 0,
               "sum_other_doc_count": 0
            }
        }
    }
}

As you can see, Doofinder returns info about the groupings for each facet field in the search results. Therefore, we know that , within our search results:

  • You can pick 2 results that are "green", 5 that are from the brand "Red castle", or 6 results that belongs to the "walking chairs" category.
  • The minimum price is 30, the maximum is 94.9

Notice: As you can see in the example above, we currently support two types of grouping or facets: terms type (like categories or color) and range type (like price).

Response with filters

If you make a search request with filters, the response , along with the basic parameters and the facets information, comes also with a property named filter that contains the applied filters in the search request.

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
    "query":"chair",
    "total":"6",
    "..":"..",
    "results":[
        {"…":"…"},
        {"…":"…"}
    ],
    "facets":{
       "…":"…"
    },
    "filter":{
        "terms":{
            "categories":["Walking chairs"],
            "color":["Green"]
        }
    }
}

Advanced

Registering clicks

With the management api you can obtain stats info about clicks made in search results. Usually our javascript layer takes care of that, but if you plan to use our search API with your "own devices", then you should "register" the clicks made to a search result item.

Say a user search for "potato" and obtains 3 results: "Raw Potatoes, Mr. Potato accesories, Fish & Chips". Then the user clicks on "Mr. Potato accesories" result and you want to register that click. To register that click, you have to do this request:

GET https://eu1-search.doofinder.com/5/stats/click?dfid={dfid}&query={query}&random={random_value}

Where:

  • {dfid} Is the doofinder result "doofinder id". It comes in every doofinder results for every item. it has the form {hashid}@{type}@{md5id}: i.e.: 6a96504dc173514cab1e0198af92e6e9@product@e19347e1c3ca0c0b97de5fb3b690855a
  • {query} is the query used to get the result that is being clicked. it's not mandatory, but important for the "clicktrough" stats
  • {random_value} in case you're doing the request from within a browser, you should add a random parameter with a random value which changes in every request. This is to avoid meddling with the browser's cache.

The server will answer with a simple OK message.

There is also a support article about how to register clicks programatically

The "query_name" parameter

As you read before, query_name is the type of search Doofinder does in order to obtain results.

The default behavior is to let Doofinder try several and return the first one that matches better, but you may want to instruct doofinder to try one particular type of search by passing the query_name parameter.

The possible values for the query_name parameter are:

  • match_and Doofinder will return results that contain all the search terms
  • match_or Doofinder will return results that contain any of the search terms. Of course, results that contain all the terms are scored better.
  • fuzzy Doofinder will try to apply some "fuzzy logic" to determine whether a result, even if it's not an exact match, is "close enough" to the search terms.
  • phonetic_text Doofinder will try to match search terms not according to how they're written but how they're pronounced. i.e. "cat" and "kat" would be a match.