Icon

Pagination on the example of Google Books API

Pagination on the example of Google Books API
This workflow demonstrates how to send a GET request with pagination on the example of Google Books API.In the case of Google Books API, the key parameter is "startIndex". The maximum number of returned results is 40, the default one is 10.First, the workflow gets the total number of items for the requested search term.Next, the workflow starts to get the titles and the authors for the books corresponding to the provided search terms from the "startIndex" = 0. At each iteration, the "startIndex" is increased by thenumber of results returned in this iteration. Next iteration starts from the increased "startIndex". This process is repeated till "startIndex" is higher or equal than the maximum number of items.The duplicates are removed in the end. Get the total number of items for the search term Loop over the pages and get the books metadata page by page ExtracttotalItemsChange startIndexparameter in the URLSend requestto APIExtractbooks metadataUngrouptitles &authorsNumber ofresults inthis iterationIncrement startIndexby # of resultsin this iterationCreate variable startIndex=0Collect the resultsuntill startIndex>= totalItemsSend requestto APIIterate resultpages (max 40results per page)JSON Path String Manipulation(Variable) Table Rowto Variable GET Request JSON Path Ungroup Extract TableDimension Math Formula(Variable) Variable Creator Variable ConditionLoop End GET Request Generic Loop Start DuplicateRow Filter This workflow demonstrates how to send a GET request with pagination on the example of Google Books API.In the case of Google Books API, the key parameter is "startIndex". The maximum number of returned results is 40, the default one is 10.First, the workflow gets the total number of items for the requested search term.Next, the workflow starts to get the titles and the authors for the books corresponding to the provided search terms from the "startIndex" = 0. At each iteration, the "startIndex" is increased by thenumber of results returned in this iteration. Next iteration starts from the increased "startIndex". This process is repeated till "startIndex" is higher or equal than the maximum number of items.The duplicates are removed in the end. Get the total number of items for the search term Loop over the pages and get the books metadata page by page ExtracttotalItemsChange startIndexparameter in the URLSend requestto APIExtractbooks metadataUngrouptitles &authorsNumber ofresults inthis iterationIncrement startIndexby # of resultsin this iterationCreate variable startIndex=0Collect the resultsuntill startIndex>= totalItemsSend requestto APIIterate resultpages (max 40results per page)JSON Path String Manipulation(Variable) Table Rowto Variable GET Request JSON Path Ungroup Extract TableDimension Math Formula(Variable) Variable Creator Variable ConditionLoop End GET Request Generic Loop Start DuplicateRow Filter

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Lada.Rudnitckaia
Created at: 2022-09-14
On NodePit since: 2022-09-16
Last update: 2024-04-27
Created with KNIME version: v4.6.1
Tags: paginationAPIGET requestREST

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!