HTTP Retriever

Go to Product

This node allows to perform different HTTP methods: GET, POST, HEAD, PUT, DELETE, PATCH. The node allows to transfer content, which must be given as binary or string data. It handles cookies and allows to specify arbitrary HTTP headers.

Results of the “HTTP Retriever” node are provided as “HTTP Result” cell type. The HTTP Result type bundles the actual binary content of the result, status code, and all HTTP response headers. In case you want to extract header information from an HTTP Result, use the “HTTP Result Data Extractor” node.

Form-encoded requests

In case you want to send content with your HTTP requests (typically for POST and PUT), you can select a Binary cell as input in the HTTP Retriever node’s configuration. For building a form-encoded request, use the “Form Encoded HTTP Entity Creator” node, where you can transform string columns to encoded key-value data. Do not forget to specify HTTP entity content type in HTTP Retriever’s configuration afterwards.

Multipart-encoded requests

Multipart-encoded requests can be created using the “Multipart Encoded HTTP Entity Creator” node. It requires one or more binary input columns and creates (1) a combined multipart-encoded column, (2) column with the content type header, including the delimiter. Connect an HTTP Retriever node and select appended binary column as HTTP entity input, and use a flow variable to set the proper HTTP entity content type.

Cookies

Cookies which are created during the node’s execution are output to the HTTP Retriever’s second output port. In case you want to send cookies with a request, use the second (optional) input port of the HTTP Retriever node. When performing sequential requests with multiple HTTP Retriever nodes, you can simply chain the cookie in- and out-ports to hand the cookies through the workflow.

“Infotising”: This node is only intended for “static” HTML structures. If you need to work with interactive web pages and web apps which are dynamically generated client-side in the browser, have a look at our plugin “Selenium Nodes”.

Options

General

URL input
The column in the input table which contains the URLs to retrieve.
HTTP method input
The (optional) column in the input table which contains the HTTP methods to execute. In case no column is selected, a GET will be assumed for all URLs. Supported methods are: GET, POST, HEAD, PUT, DELETE, PATCH.
HTTP entity input
The (optional) column in the input table which contains the HTTP entity to send. The data must be supplied as binary object or string cell.
HTTP entity content type
The content type of the HTTP entity.
Maximum file size
The limit for file sizes in kilobytes when downloading. After the limit is reached, the download is cancelled.

Headers

Header columns
Select string columns in the input table which will be sent as HTTP headers.

Advanced

# retries after error
The number of retries to perform for each URL in case the retrieval fails. A value of zero means no retrying at all.
Connection timeout (ms)
The timeout in milliseconds to wait for a connection to be established.
Socket timeout
The timeout in milliseconds to wait for new packets to arrive.
User agent
The HTTP user agent string.
Accept all certificates
If enabled, all SSL certificates will be accepted. Caution: Only enable this option if you really know what you are doing!
Fail on non-success HTTP status code
Fail execution, in case an HTTP status of >= 400 is returned. (this is a logical error, so no retries are executed).
Fail on network error
Fail execution in case a network error occurs (and the specified # of retries have been reached)

Proxy

Use custom proxy server
Enable this to override the default proxy server (as configured in KNIME’s preferences → General → Network Connections). This is useful, if you want to use a proxy server only for specific requests.
Host
Hostname or IP address of the proxy
Port
The port of the proxy
Username (*)
Username for proxy authentication
Password (*)
Password for proxy authentication

Input Ports

Icon
Table with HTTP URLs to be retrieved. Optionally a column with a HTTP method to perform for each URL.
Icon
Table with cookies to use during execution.

Output Ports

Icon
Table with downloaded HTTP Results.
Icon
Table with cookies which were set during execution, in case a cookie table was provided as input, new cookies will be appended, existing cookies replaced.

Popular Predecessors

  • Variable to Table Row6 %
  • Table Creator3 %
  • Table Row To Variable Loop Start3 %
  • Chunk Loop Start3 %
  • Excel Reader (XLS)3 %
  • String Manipulation2 %
  • Wait...2 %
  • URL Extractor2 %
  • XML Reader2 %
  • Cross Joiner2 %
  • Excel Reader (XLS)2 %
  • Row Filter1 %
  • XLS Reader1 %
  • String Replacer1 %
  • Download1 %
  • HTTP Retriever1 %
  • Python Source< 1 %
  • URL to File Path< 1 %
  • Constant Value Column< 1 %
  • List Files< 1 %
  • Web Searcher< 1 %
  • Recursive Loop Start< 1 %
  • Counting Loop Start< 1 %
  • Column Filter< 1 %
  • URL Resolver< 1 %
  • Start WebDriver< 1 %
  • Loop End< 1 %
  • Loop End< 1 %
  • Reference Row Filter< 1 %
  • Top k Selector< 1 %
  • Variable to Table Column< 1 %
  • Partitioning< 1 %
  • Empty Table Creator< 1 %
  • OAuth< 1 %
  • HTTP Retriever< 1 %
  • File Reader< 1 %
  • Column Combiner< 1 %
  • String Input< 1 %
  • URL Normalizer< 1 %
  • Variable to Table Row< 1 %
  • Duplicate Row Filter< 1 %
  • String Manipulation (Variable)< 1 %
  • Regex Extractor< 1 %
  • Table Column to Variable< 1 %
  • Concatenate< 1 %
  • Column Expressions< 1 %
  • Table View< 1 %
  • UNIX Timestamp to Date&Time< 1 %
  • HTML Parser< 1 %
  • Binary Objects to Strings< 1 %
  • URI to String< 1 %
  • URL Domain Extractor< 1 %
  • Table Column to Variable< 1 %
  • Split Collection Column< 1 %
  • Strings to Binary Objects< 1 %
  • Java Snippet< 1 %
  • Column List Loop Start< 1 %
  • Column Splitter< 1 %
  • Ungroup< 1 %
  • Rule-based Row Splitter< 1 %
  • Empty Table Switch< 1 %
  • End IF< 1 %
  • Credentials Input< 1 %
  • JSON to Table< 1 %
  • POST Request< 1 %
  • XSLT< 1 %
  • HTML Node to Text< 1 %
  • Extract Attribute< 1 %
  • CSV Reader< 1 %
  • Python Script< 1 %
  • OAuth Connector [BETA]< 1 %
  • String to Path (Variable)< 1 %
  • Date Extractor< 1 %
  • String to URI< 1 %
  • Try (Data Ports)< 1 %
  • Java Edit Variable< 1 %
  • Variable Condition Loop End< 1 %
  • Group Loop Start< 1 %
  • Column Aggregator< 1 %
  • Row Splitter< 1 %
  • GroupBy< 1 %
  • Missing Value< 1 %
  • Column Rename< 1 %
  • RowID< 1 %
  • Row Sampling< 1 %
  • Sorter< 1 %
  • Rule-based Row Filter< 1 %
  • Parallel Chunk Start< 1 %
  • TimeDelay Loop Start< 1 %
  • JPython Function< 1 %
  • Java Edit Variable (simple)< 1 %
  • String to JSON< 1 %
  • Table to JSON< 1 %
  • JSON Path< 1 %
  • GET Request< 1 %
  • PUT Request< 1 %
  • String To XML< 1 %
  • XPath< 1 %
  • HTTP Result Data Extractor< 1 %
  • Form Encoded HTTP Entity Creator< 1 %
  • Multipart Encoded HTTP Entity Creator< 1 %
  • Chunk Loop Run-for-time Loop Start< 1 %
  • Variable to Table Column< 1 %
  • IF Switch< 1 %
  • RSS Feed Reader< 1 %
  • Webpage Retriever< 1 %
  • Web Page Content Extractor< 1 %
  • URL Decode< 1 %
  • Variable to Table Column< 1 %
  • Table Row to Variable< 1 %
  • Variable To Table Row< 1 %
  • Table Reader< 1 %
  • Variable Creator< 1 %
  • Counting Loop Start< 1 %
  • Generic Loop Start< 1 %
  • Table Row To Variable Loop Start< 1 %
  • Column Appender< 1 %
  • Joiner< 1 %
  • Excel Reader< 1 %
  • Path to String< 1 %
  • Cell Splitter< 1 %
  • Cookies< 1 %
  • Try (Variable Ports)< 1 %
  • Extract Attribute< 1 %
  • Catch Errors (Data Ports)< 1 %
  • Inject Variables (Data)< 1 %
  • Database Reader< 1 %
  • Table Reader< 1 %
  • Generic Loop Start< 1 %
  • Recursive Loop Start (2 ports)< 1 %
  • Concatenate (Optional in)< 1 %
  • Column Appender< 1 %
  • Joiner< 1 %
  • Transpose< 1 %
  • Rule Engine< 1 %
  • Python Script (1?1)< 1 %
  • Bulk RemoteWebDriver Factory< 1 %
  • URL Encode< 1 %
  • Table Row to Variable< 1 %
  • Missing Value Column Filter< 1 %
  • Tika Parser URL Input< 1 %
  • R Snippet< 1 %
  • Current URL< 1 %
  • Strings To Document< 1 %
  • String To Number< 1 %
  • Column Merger< 1 %
  • Pivoting< 1 %
  • Shuffle< 1 %
  • Unpivoting< 1 %
  • Math Formula (Variable)< 1 %
  • Excel Reader (XLS)< 1 %
  • Tika Parser< 1 %
  • Python Source< 1 %
  • ChEBI Retriever< 1 %
  • Dataset Reader< 1 %
  • Navigate< 1 %
  • Quit WebDriver< 1 %
  • Data Generator< 1 %
  • String Widget< 1 %
  • Create Date&Time Range< 1 %
  • Rule-based Row Filter (Dictionary)< 1 %
  • Window Loop Start< 1 %
  • GET Resource< 1 %
  • Chunk Loop Run-to-time Loop Start< 1 %
  • Load Text Files< 1 %
  • HTTPS Connection< 1 %
  • Extract Variables (Data)< 1 %
  • Database Looping< 1 %
  • File Reader< 1 %
  • Line Reader< 1 %
  • MultiLayerPerceptron Predictor< 1 %
  • Container Input (Table)< 1 %
  • Date&Time-based Row Filter< 1 %
  • Feed Parser< 1 %
  • Find Elements< 1 %
  • Download / Upload from List< 1 %
  • Add Empty Rows< 1 %
  • Number To String< 1 %
  • Merge Variables< 1 %
  • Extract Color< 1 %
  • DB Reader< 1 %
  • Stop Word Filter< 1 %
  • Document Grabber< 1 %
  • Column Selection Configuration< 1 %
  • Container Input (Credentials)< 1 %
  • Container Input (Variable)< 1 %
  • Ranking Services< 1 %
  • CASE Switch Data (Start)< 1 %
  • DB Query Reader< 1 %
  • GET Resource (Batch)< 1 %
  • Extract URI Info< 1 %
  • 列中行到变量< 1 %
  • CSV Reader< 1 %
  • Interval Loop Start< 1 %
  • Recursive Loop End< 1 %
  • Recursive Loop Start< 1 %
  • Cell Splitter< 1 %
  • Create Table Structure< 1 %
  • Nominal Value Row Filter< 1 %
  • Reference Row Splitter< 1 %
  • One Row to Many< 1 %
  • Math Formula< 1 %
  • Java Snippet (simple)< 1 %
  • Google Sheets Reader< 1 %
  • File Chooser< 1 %
  • JSON Row Combiner< 1 %
  • Don't Save Start< 1 %
  • R Source (Table)< 1 %
  • Date&Time to String< 1 %
  • Wait< 1 %
  • MapQuest Geocoder< 1 %
  • Wait< 1 %
  • Create REST Representation< 1 %
  • Clean HTML Retriever< 1 %
  • XML Sitemap Reader< 1 %
  • Flow Variable IF Switch (Flow Variable Value)< 1 %
  • Generic Run-for-time Loop Start< 1 %
  • Table Row To Variable Run-to-time Loop Start< 1 %
  • Wait-for-time< 1 %
  • Random Numbers Generator< 1 %
  • String to Table< 1 %
  • Delete Files< 1 %
  • List Remote Files< 1 %
  • Catch Errors (Generic Ports)< 1 %
  • Read Images< 1 %
  • Loop End (2 ports)< 1 %
  • Variable Loop End< 1 %
  • Column Auto Type Cast< 1 %
  • Extract Column Header< 1 %
  • Column Resorter< 1 %
  • Double To Int< 1 %
  • Reference Column Filter< 1 %
  • Filter Apply< 1 %
  • Regex Split< 1 %
  • Rule Engine Variable< 1 %
  • Data Explorer (JavaScript)< 1 %
  • Cache< 1 %
  • HiLite Table< 1 %
  • Parallel Chunk End< 1 %
  • TimeDelay Loop End< 1 %
  • Variable Expressions< 1 %
  • External Tool< 1 %
  • Read Excel Sheet Names (XLS)< 1 %
  • Markup Tag Filter< 1 %
  • PDF Parser< 1 %
  • RSS Feed Reader< 1 %
  • String Configuration< 1 %
  • Column Filter< 1 %
  • Boolean Input< 1 %
  • Container Input (Row)< 1 %
  • Columns to JSON< 1 %
  • Image Converter< 1 %
  • Image Reader (Table)< 1 %
  • MongoDB Reader< 1 %
  • Call Workflow (Table Based)< 1 %
  • Python Script (1⇒1)< 1 %
  • Python Script (1⇒2)< 1 %
  • Python Edit Variable< 1 %
  • Date Field Extractor (legacy)< 1 %
  • Column To XML< 1 %
  • Table to HTML String< 1 %
  • Latitude/Longitude to Coordinate< 1 %
  • AP Calculator< 1 %
  • ContentExtractor< 1 %
  • Feed Discovery< 1 %
  • NekoHtmlParser (deprecated)< 1 %
  • Click< 1 %
  • Execute JavaScript< 1 %
  • Variable Loop End< 1 %
  • Extract Missing Value Cause< 1 %
  • Table Manipulator< 1 %
  • String Manipulation (Multi Column)< 1 %
  • Random Number Assigner< 1 %
  • HTTP(S) Connector< 1 %
  • String To Term< 1 %
  • Term To String< 1 %
  • List Files/Folders< 1 %
  • String to Path< 1 %
  • Transfer Files< 1 %
  • Table Editor< 1 %
  • Container Input (JSON)< 1 %
  • XML Reader< 1 %
  • Find Elements< 1 %

Popular Successors

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.

Go to Product

Installation

To use this node in KNIME, install the extension Palladian for KNIME from the below update site following our NodePit Product and Node Installation Guide:

v5.12

A zipped version of the software site can be downloaded here.

Plugin provider: palladian.ws
Plugin version: 3.4.0.202601041906
On NodePit since: 2026-07-07
Last update: 2026-07-22
KNIME versions: Since v3.6
NodePit Exclusive Only available on NodePit

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!