log_parser_jobnames

log_parser_jobnames

This workflow basically retrievs log files from the KNIME Server, parses them in a certain way, to combine their contents and stores the result in an AWS bucket.

Therefore, the workflow only considers the knime.log file and the localhost.yyyy-mm-dd.log files. The idea behind this workflow is to add the workflow names, which are generally a bit complex to extract, to the jobIDs in the output. Thus, the user can interpret the server log information more easily and knows in which workflow to look for errors, if there are any.

The 'download log files' metanode retrieves the KNIME Server log files via a GET Request. It unzips the downloaded archive and stores the log files in a temporary folder, which is removed after the workflow is executed.

Since this workflow is meant to be running constantly every x hours on the KNIME Server, some date&time handling is necessary. Therefore, the 'read log time from last iteration' metanode generates a dummy date for initial execution. On each future exectution, it reads the date from a file, which is written in the 'extract & write last log time' metanode. Here, the workflow extracts the most current date from the currently read log files and saves it to a physical file, which persists, until the workflow is executed the next time. It will be overwritten each time, the workflow ran again.

The component "read server logs" utilises this date to apply temporal filtering on the KNIME Server log files. However, past development caused the workflow to be developed in a way, that only localhost.yyyy-mm-dd.log files from the last two days are read. After the upper section processes the workflow names inside the logs the outcome is joined together with the filtered knime.log file content.

Outside of the component, the result is written to a temporary CSV file, which is then uploaded into an AWS bucket in the 'upload to AWS' metanode.

log_​parser_​jobnames

Nodes

Extensions

Links

Download

log_parser_jobnames