Paginating the data feed is an excellent way of making sure the server requests run smoothly and effectively. The server does not need to be tasked with an overload of requests to maintain your website’s peak performance.

You can feed Doofinder data directly via API or allow Doofinder to read the data from a URL.

This way, URLs can point to already generated URLs (so they’re basically downloads) or to server endpoints that dynamically return the most up-to-date data directly from the database. Such endpoints can be expensive in terms of resources. Therefore, Doofinder splits the download into multiple requests to maintain performance by enabling feed pagination.

Pagination, in this case, refers to the ordinal numbering of pages system.

Doofinder’s Feed Pagination

To activate Doofinder’s feed pagination, you need to contact the supporthttp://server.com/doofinder.cfm, which has 6300 products in its data feed. We then set its pagination limit to 2000 items.

The first server request to our parser would then be:

http://server.com/doofinder.cfm?limit=2000&offset=0

This would fetch the first 2000 items under the following conditions:

  • When the feed is an XML, this would also fetch the declarations at the beginning of the XML file.
  • When the feed is TXT, this would also fetch the file’s header. 

Consequently, the next request would be:

http://server.com/doofinder.cfm?limit=2000&offset=2000

This would fetch the items between 2000 – 3999.

⚠️ Warning: Special care should be taken when the feed is an XML or CSV feed such that the beginning of the XML is not resent with the data. 

This would be under another request with offset 4000:

http://server.com/doofinder.cfm?limit=2000&offset=4000

A request with the offset set at 6000 would fetch the last 300 items. When the feed is in XML, this would also fetch the XML closing tags.

http://server.com/doofinder.cfm?limit=2000&offset=8000

 The final request would then get an empty response, signalling Doofinder’s downloader to end the download process. 

Should you encounter any problems, please contact support@doofinder.com

Related articles:

Last Updated: Feb, 2022