Elasticsearch Writer (ES 6)

Go to Product

Writes data to Elasticsearch.

This node takes documents as JSON and sends them one by one to the Elasticsearch cluster for indexing. Document metadata, such as the target index can either be provided globally via the dialog or on a document-level as column.

If your documents are not yet in JSON format (but a KNIME table), we highly recommend using the “Table to JSON” node that allows you to select columns and transform these into one JSON object per row that then easily can be used as input for this node.

Options

Protocol
The protocol used to connect to Elasticsearch. The following strategies are available:
  • HTTP: Connect via HTTP. Be aware: Data transfer is send unencrypted and vulnerable to man-in-the-middle attacks.
  • HTTPS: Connect via HTTPS. Data transfer is send encrypted. The Elasticsearch server has to support this mode.
Host
Host name.
Port
Network port under which Elasticsearch can be reached. By default Elasticsearch uses port 9200 for communication.
Accept all certificates
If enabled, all SSL certificates will be accepted. Caution: Only enable this option if you really know what you are doing!
Authentication
The protocol used to connect to Elasticsearch. The following strategies are available:
  • None: No authentication is used. Elasticsearch can be accessed without any access restrictions.
  • Credentials: Credentials are injected via KNIME credentials. Select the proper credential via drop-down menu.
  • Username & Password: Username and password are provided within the dialog. Enter username and password in the provided text fields.
Index
The Elasticsearch index the data is written to. Fetch existing indices via "Select an index…". This option might not be available if you have specified the index via the "Elasticsearch Index" node already. Besides you can also provide an index for each document to be added/updated. Therefore specify a column that holds the index on a document basis. Selecting a column will overwrite the index you set manually. In case the specified column does not provide an index for a particular document, index selection falls back to the manually provided index.
Type
The Elasticsearch type the data is written to. Fetch existing indices via "Select a type…". Please keep in mind, Elasticsearch 6 supports only one type per index and will throw an exception else wise. This option might not be available if you have specified the type via the "Elasticsearch Type" node already. Besides you can also provide a type for each document to be added/updated. Therefore specify a column that holds the type on a document basis. Selecting a column will overwrite the type you set manually. In case the specified column does not provide a type for a particular document, type selection falls back to the manually provided type.
Routing
Specify a custom routing, if you like the operation to be executed to a particular shard. This option might not be available if you have specified the route via the "Elasticsearch Route" node already.
Column with documents
The column that holds documents to be written to the Elasticsearch index. If the specified column contains missing or invalid values, the Elasticsearch Writer skips this row. If your documents are not yet in JSON format, we recommend first using the “Table to JSON” node to convert your input table into a proper JSON structure that can then be inserted into your cluster using this node.
Column with unique identifiers
If you like to overwrite already existing documents or manually provide an unique identifier for each document, please specify a column that holds the document identifier. If no column is specified, Elasticsearch will auto-generate a document identifier for you. If the specified column contains missing values, the Elasticsearch Writer skips this row.
Append result columns
Enable this option to append result columns to the output table. Result columns contain information about the index, type, id, version, and the result of the write operation for each individual document.
Fail on error
Disable this option if you want to continue after an error.

Input Ports

Icon
An Elasticsearch connection.
Icon
Documents and meta information to be written to Elasticsearch.

Output Ports

Icon
The input data table with additional columns, if checked in the dialog. The results of the write operation. Additional columns contain information about index, type, id, version, and result of the write operation for each individual document.

Views

This node has no views

Workflows

  • No workflows found

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.