Advanced Data Feed Concepts

or send an email to support@doofinder.com

In this article

You've started dealing with feeds and now that all is getting easy, you think you can do more things with them… You're right.

Data Types

As you will know, by default Doofinder provides a product data type by default. That's because the vast majority of our users owns an ecommerce site, so we implemented some tweaks to better manage that kind of documents on search.

But Doofinder also allows you to create additional data types or even completely replace the product type.

Additional Data Types

To create a new data type from the Admin, use the New Type button and provide a significant name for your datatype. For example, if you want to index plain text articles, you could create an article or a dessert_recipe type if you're going to index dessert recipes.

Just take into account that you should not use special characters nor blank spaces, to separate words use underscore symbols ( _ ) and everything will go smooth.

Once you have your new datatype you only have to add a feed to it and save.

NOTICE: Data types with no feed will be deleted when you save feeds configuration.

Removing Data Types

To remove a data type from the Admin just remove all feeds and save.

Working with Data Types Programmatically

If you're asking yourself if you can manage data types from your website, the answer is affirmative. You can use our API to request changes in your search engines.

Learn more about the Management API.

Data Feeds' Advanced Usage

Additional Feeds

Just as we can manage multiple data types, we can manage several feeds for each type. That's useful if your data comes from different sources.

You can add additional data feeds directly from the Admin, by pressing the Add Feed button in the data type feeds configuration section.

WARNING: Each indexed item must have it's own unique id. Values for the id field can be repeated between different data types but not in the same data type. If you use multiple feeds under the same data type remember to assign different values for the id field, so there're no duplicates for a single data type.

Removing Feeds

To remove a feed from the Admin just press the Remove red button next to it and save.

Working with Feeds Programmatically

The same way you can manage data types via API, you can configure feeds for your search engine.

Learn more about the Management API.

Combining Feeds and API

The default behavior in Doofinder is to process your feeds in a daily basis. That's enough for most sites that update their products at night but, if you're ecommerce manages stock on-the-fly, you may want to update or remove some products when they become out of stock, or add new ones when you create them in your site.

This is specially indicated if you have lots of documents in your search index, so you don't have to re-index all of them again and again to get just a few updates.

You can use our API to perform single (or batch) operations to update certain documents in Doofinder. Those updates will be applied to your search index immediatelly.

This way, your data feed acts as the source of truth. Each time it is processed, all data in your index is replaced by the contents of the feed, which should be updated with the latest content in your site.

Learn more about the Management API.

WARNING: If you added items via API and you process and outdated data feed, your index will also be out of date. This can affect you too if you have configured your feed to be automatically processed.

Switching between API and Feeds

Business can change, and the way you feed your search engines can change too. You may need to change the way you add items to your search results.

You can start working with your search engine by adding items via API and then change to a URL-based data feed. There's no problem in that, but remember that, when you process your file feed, you lose all existing products, including those indexed via API, because you decided to use data feed as the source of truth.

NOTICE: If you plan to change the way you feed your search engine, we strongly recommend you to start with a new search engine and make tests there before switching to a production environment. You might not need all the configurations that you've already added to your old search engine.