Find Elements

Extracts WebElements based on different criteria. The input can either be a WebDriver (representing a browser’s window), or a WebElement (a subpart of the window). The following search criteria are available:

Find by	Explanation
Class Name	Finds elements based on their `class` attributes. In case an element has multiple class values assigned, matching a single value is sufficient.
CSS Selector	Finds elements based on a CSS selector expression.
ID	Finds an element based in its `id` attribute.
Link Text	Finds links through an exact match of their text.
Name	Finds elements based on their `name` attributes.
Partial Link Text	Finds links by a partial match of the given text.
Tag Name	Finds elements based on their tag names.
XPath	Finds elements based on an XPath expression. (see e.g. here for a introduction to XPath)

You can use the “Outline” view to easily create selector queries from the current DOM tree: Find the element you want to extract and double click on the corresponding line. A matching selector expression will be automatically inserted into the “Query” field.

Alternatively, you can select elements directly in the browser (works for Chromium-based browsers Chrome, Chromium): Click the “Select” button beside the “Query” input, switch to your browser window, and select any element on the page. A selector expression will be inserted into the “Query” field.

Please note that these feature require (1) a running browser instance which must show the actual data you want to extract, (2) the “full input data” to the node to be available.

To search for a text snippet within the DOM tree, use the search bar below the source.

For each extracted WebElement a new row is added to the output table.

Note that you cannot only search within the given input, but also navigate the entire DOM tree. For example, you can use the XPath query .. to find a WebElement’s parent.

Also note that you can only find on a DOM element level here, but not text content or attributes. To extract text, use this node to select the surrounding element, and then connect an “Extract InnerHTML” node to get the text. To extract attribute values, use an “Extract Attribute” node accordingly.

Options

Find in: The input column providing the starting point where to search.
By: The search criterion (see above).
Query: The query (in case you allow KNIME to open the dialog with “full input data” and the Selenium browser is running, the dialog will give you a live update with the number of matches for the entered query. The query input will show suggestions based on the currently loaded page e.g. for ID, class name, tag name, and tag name. Hint: To show updates instantly, make sure to disable the “Implicit wait” in the Start WebDriver node!).
Set result column name: Enable to change the name of the appended column.
Extract first match only: If enabled, extract only first match per input row instead of all.
Return missing values when nothing is found: If enabled, return a “missing value” cell in case nothing could be found, or an ignored exception was catched (see below).
Wait up to X s, poll every Y ms: Specify the maximum amount of time in seconds to wait in case elements are searched and are not immediately available. Additionally, the poll interval in milliseconds can be specified. Selenium will wait at most for the given amount of time and keep checking in the given poll interval.
Ignore exceptions: Do not fail the node’s execution in case an exception is encountered (typically, in case the wait timeout was reached without finding an element). Note, that when used together with the “wait” option and multiple input rows, this will cause the node to wait for each row sequentially (i.e. a worst case of X seconds × N rows).
Append additional WebElement information: If enabled, additional columns with the WebElement’s properties are appended (tag name, text content, selected, enabled, displayed state, position and size). This may (1) slow down extraction or (2) not be supported by specific WebDriver implementations. In latter case MissingValues will be returned.

Input Ports

: Table with a column providing a WebDriver or WebElements in which to search

Output Ports

: Table with the found WebElements (and optionally additional columns with more information)

Popular Predecessors

Popular Successors

Views

This node has no views

Workflows

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.

Installation

To use this node in KNIME, install the extension Selenium Nodes for KNIME from the below update site following our NodePit Product and Node Installation Guide:

v5.3

A zipped version of the software site can be downloaded here.

Plugin provider: seleniumnodes.com

Plugin version: 5.9.0.202411012330

On NodePit since: 2024-07-10

Last update: 2024-11-23

KNIME versions: Since v3.6

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!