Icon

Web Scrapping & Selenium

Forking Angel Molina's workflow, adjusting some nodes and scraping prices for olive oil instead of sunflower oil.

URL: Web Scrapping & Selenium by Ángel Molina https://hub.knime.com/mola_data/spaces/Public/Web%20Scrapping%20&%20Selenium%20by%20%C3%81ngel%20Molina~UqLvaRaJTQD6ZBQU/current-state

Connect to and interact with website
Extract product and prices
Add timestamp and store results

Web scraping of products and prices

This workflow interacts with the Spanish website of a supermarket to scrape olive oil products, prices and price/l.

Create Date&Time Range
Split original and discounted pricesand calculate price/l
Wrangle prices
Extract text in <h> tag-> in this case: products
Content Retriever
Extract text in <p> tag-> in this case: prices
Content Retriever
Column Appender
- ZIP codeof El Prat de Llobregat, Spain- Search input: "Aceite de girasol 5L"
Variable Creator
Validate ZIP code
Clicker
Only "*aceite*"
Row Filter
Only prices
Row Filter
Click filter
Clicker
Input website ofsupermarket
Navigator
Start web browser
Web Interaction Start
Area where toinput ZIP code
Clicker
Reject cookies
Clicker
Search bar
Clicker
Input ZIP code
Text Sender
Rename columns
Table Manipulator
Confirm ZIP code
Clicker
Provide searchinput text text:"Aceite de girasol 5L"
Text Sender
Store scraped results
Excel Writer
Cross Joiner

Nodes

Extensions

Links