Icon

01b Data Collection with Web Interaction - Exercise

Data Collection: Web Interaction


Learning objective: In this exercise, you will learn how to use the nodes of the Web Interaction Extension to interact with a browser through KNIME and obtain the content of a website.


Workflow description: In this exercise we are going to connect to the Amazon website, search for a product and store the information about it. While the workflow is executing, you can see what is happening in the browser window that automatically opens.


You'll find the instructions to the exercises in the yellow annotations.

Step 1. Start the interaction

Use the Web Interaction Start node to open your browser and start the session.



Step 2. Connect to the website

Add the Navigator node and provide the desired url https://www.amazon.co.uk/



Step 3. Select the search bar

Use the Clicker node to select the search bar in the Amazon homepage.

  • Access method: CSS Selector

  • Access query: #twotabsearchtextbox


Note that the access query must be adapted if you connect to a different website.

Step 4. Add text

Configure the Text Sender node to write some text in the search bar

  • Select the search bar as in the previous step

  • Text to send: ergonomic office chairs



Step 5. Click the search button

Configure the Clicker node to click the search button

  • Access method: CSS Selector

  • Access query: #nav-search-submit-button



Step 6. Retrieve the content

Use the Content Retriever node to read the elements of the HTML page inside the workflow.

  • Retrieve tag type: Headings



Optional. Wait...

Add a Wait... node between all the web interaction nodes. This will increase the waiting time between two subsequent actions and reduce the errors




Step 7. Filter results

  1. Only filter the headings where the Heading tag is h2

  2. Remove the headings with less than 100 chars

  3. Keep only the Heading text column

  4. Remove the rows with missing values



Step 8. Export

Write the results to an Excel file.



Optional. End session

Use the Web Interaction End node to close the browser session.




Nodes

  • No nodes found

Extensions

  • No modules found

Links