NodePit Runner

Run Your Workflows Around The Clock!

Changelog

This log gives an overview of the most prominent changes in each release. Minor fixes, changes “under the hood” and refactoring changes are not listed here. We follow the Semantic Versioning scheme and the guidelines from “Keep a Changelog” as close as possible.

version-1.17.0 (2024-12-07)

Add
Add support for KNIME 5.4.0. Please make sure to update NodePit Runner when using KNIME 5.4.
Change
Executors are now identified by their container name. As a result, you can now update your executors or scale them up or down (with docker compose scale executor=2) without getting completely new executor entries in the executor list.
Change
Set KNIME 5.4.0 as the default for new environments.
Remove
Remove the SHARED_VOLUME_NAME environment variable for executors. Executors don't need to explicitly configure the shared volume, as this is now derived programmatically. You still need to mount a data volume into the executor container with target /opt/data (as already done previously).

version-1.16.0 (2024-11-26)

Add
Executors can now be paused. Paused executors still finish any assigned runs, but will not execute new runs. This can be useful if you want to temporarily stop an executor from running schedules, but don't want to remove or unregister it.
Add
Executors can now be modified by admins via UI (e.g. change the executor's description or modify tags).
Add
Show run count for each executor.
Add
Show run list per executor.
Change
You can now set environment variables INITIAL_DESCRIPTION and INITIAL_TAGS for executor containers. These values will only be honored when the executor registers against the NodePit Runner API. Changing descriptions and tags afterwards can now be done via the UI.
Deprecate
The environment variables DESCRIPTION and TAGS for executor containers are now deprecated and are currently mapped to INITIAL_DESCRIPTION and INITIAL_TAGS.

version-1.15.0 (2024-11-20)

Add
Add tags to executors, environments, and schedules to control where runs are executed. You can now assign tags to executors via the TAGS environment variable (e.g. TAGS=large,gpu). Runs of environments and schedules that require a certain tag (e.g. large) can now only be executed by executors that are tagged accordingly. Runs without tags can also be executed by executors with tags.
Add
Add new configuration option “Cancel Pending Runs After” that automatically cancels long pending runs (e.g. due to missing executors).
Add
Add search functionality for the log output of a run.
Add
Add an option to copy the entire log output to the clipboard. This eliminates the need to manually select particularly long logs.
Add
Add an option to download the entire log as file.
Add
Add highlighting of log lines and the ability to create links to highlighted lines. Select individual log lines by clicking on the line number. Hold down the Shift key and click on the line numbers to select a group of lines. Copy the URL which will now also contain information about the selected lines.
Add
Add timeout configuration. Global timeouts can now be set by administrators in the Settings menu. No timeout is configured by default.
Add
Add timeout configuration for schedules. Schedules can now define their own timeout, which overrides any global timeout that may exist. By default, schedules respect the global timeout.
Add
Add search input field to runs.
Add
Add sorting to overview tables for executors, users, environments, files, and schedules.
Change
Set KNIME 5.3.3 as the default for new environments.
Change
Don't show unconfirmed users as suggestions for project members.
Change
All confirmed users now have read-only access to non-sensitive information in the “Executor” tab. This allows users to gain more insight into the available executors and understand why runs may not be starting.
Change
Show more information about runs on overview and details pages (e.g. duration, queue time, assigned executor).
Remove
Remove obsolete configuration options EXECUTOR_INACTIVE_TIMEOUT and EXECUTOR_ORPHANED_RUN_TIMEOUT. Previously, executers reported their status less frequently and less reliably, requiring timeouts configurable via environment variables. Recent fixes have removed the need to set these variables to increase our configured defaults.
Remove
Remove the RUN_TIMEOUT configuration option in favour of a configuration via the Settings menu.
Fix
During a run only scroll to the end of the log section when new log lines are available. Previously every five seconds the run view page had been scrolled to the bottom for active runs.
Fix
Canceling runs is now much faster as operations such as pulling, building or pushing worker images can now be aborted and don't need to be finished.

version-1.14.0 (2024-10-17)

Add
Add project role Maintainer. Maintainers have read and write access to a project's items, but are prohibited from changing the project configuration (e.g. rename project, delete project, add or remove members).
Add
Add project role Operator. Operators can read project items, but don't have permission to modify them. Operators can trigger runs to execute already pre-configured workflows and scripts. Operators can also abort and delete runs they have started.
Add
Add link to this changelog to settings page for easy access.
Change
Add the project name of the schedule to email notifications. This makes it easier to identify which schedule the notification belongs to, especially if you have multiple schedules with the same name in different projects.
Change
Run archives now contain knwf and knar files of executed workflows, which can be easily opened in KNIME by double-clicking. The previously stored unexecuted workflow is no longer part of the archive, but can be downloaded from the File menu.
Change
Show if a run has been triggered manually or by schedule on run details page.
Change
Show the name of the user that has manually triggered a run on run details page.
Fix
Fix an issue that let to new users beeing auto-confirmed.

version-1.13.0 (2024-09-17)

Add
Experimental support for Xvfb within KNIME Environments.
Change
Set KNIME 5.3.2 as the default for new environments.

version-1.12.3 (2024-08-07)

Fix
Fix an issue that let to failing runs in case the log output became to big.
Fix
Fix an issue that let to a failing request when loading an empty list of runs.

version-1.12.2 (2024-07-23)

Change
Set KNIME 5.3.0 as the default for new environments.
Fix
Fix an issue that prevented seeing members of a project in the project details view.

version-1.12.1 (2024-07-02)

Change
Simplify OpenAPI schema.
Fix
Fix updating environments and schedules.

version-1.12.0 (2024-06-27)

Add
Add quick filters to the schedule overview for filtering KNIME and Shell schedules.
Add
Add additional icons to the schedule overview to more easily distinguish between KNIME and Shell schedules without having to go to the details page.
Add
Add a details page for executors to see additional details and configuration instructions.
Add
Add uptime information to health endpoint.
Add
Allow admins to easily clean up all inactive and unused executors on executor overview.
Change
Set KNIME 5.2.5 as the default for new environments.
Change
Allow the environment cache to be cleared on the “Edit Environment” page. Clearing the environment cache forces the environment to be rebuilt the next time it is run, and can be used to force the installation of new versions of plugins and extensions that would otherwise have been cached.
Change
Apply additional validity checks and improve error messages for enhanced robustness of NodePit Runner API.
Change
Implement request-based retry functionality in the Executor component to compensate for short-term network outages.
Change
Consolidate API endpoints and flatten the structure of resources returned via the API. Depending on if and how you use the NodePit Runner API, please update to the latest version of NodePit Runner Nodes and/or check the updated API specification.
Change
Remove existing file upload limitations. Previously only a limited list of file extensions were allowed (e.g. knwf, sh, epf). With support for additionalFiles in environments and schedules, user are now able to upload any kind of file.
Fix
Fix an issue that caused a large number of runs to be created when the schedule fell into the daylight saving time switchover.
Fix
Fix a problem that caused inconsistent data when changing the type of an existing and already used environment. In future, it will no longer be possible to change the type of an environment.

version-1.11.1 (2024-05-13)

Change
Update JS runtime for improved reliability.
Change
Improve wording for run cancellation to be less confusing.
Change
Add instructions for scaling Executors to web interface.
Fix
Stop impersonation when logging out.
Fix
Fix server configuration in Swagger UI.

version-1.11.0 (2024-04-11)

Add
Suggest existing preference entries from EPF file.
Add
Show confirm dialog when there are unsaved changes.
Add
Add “Follow Log” button to run view.
Change
Set KNIME 5.2.3 as the default for new environments.
Change
Move “Save” button to top bar of page.

version-1.10.3 (2024-03-15)

Fix
Running executor on Apple Silicon processors - ensure to install Rosetta as there is currently no ARM version of KNIME available.
Fix
Further improve handling of large file uploads and provide additional log output.

version-1.10.2 (2024-02-29)

Change
Show additional icons for files and environments in UI.
Change
Improve explanations for cron expressions.
Fix
Allow to select “no file” where applicable (e.g. for KNIME preferences file).
Fix
Improve relative date formatting.
Fix
Improve sorting of lists.

version-1.10.1 (2024-02-25)

Add
Add a quick filter to display only environments of a particular type (e.g. KNIME or Shell).
Fix
Fixed an issue that led to page reloads when trying to upload a file.
Fix
Formatting of license information in UI.

version-1.10.0 (2024-02-22)

Add
Add “Projects” to NodePit Runner. Projects are a new way to group and isolate environments, schedules, files and runs (e.g. by department, business unit, topic) and make them available to other users. Invite users to your project and assign them a role to determine their access right (e.g. owner for read and write access and reader for read-only access).
Change
More logging output from Executor.
Change
Remove file upload restriction based on extension.
Change
Improve readability of date and timestamps.
Change
Misc. UI and UX improvements.
Fix
Improve handling of large file uploads.

version-1.9.0 (2024-02-13)

Add
Add Docker health check for Executor container.
Add
Show icons for environment types.
Change
Show additional details if sending a test email fails.
Change
Set KNIME 5.2.1 as the default for new environments.
Fix
Properly show favicons in all browsers.

version-1.8.0 (2024-02-07)

Add
Allow to override the Executor’s network mode via environment variable DOCKER_NETWORK_MODE. This way you can connect to existing Docker networks during your runs, e.g. to access a database, etc.
Change
Don't apply CPU and RAM resource limits to the Executor's worker containers by default. Previously, we limited the worker containers to using a maximum of 85% of CPU and 1.5 GB of RAM. If required, use the environment variables DOCKER_CPU_PERIOD, DOCKER_CPU_QUOTA, and DOCKER_MEMORY on the Executor container to set appropriate limits for your host machine. More information about resource constraints can be found here.
Fix
Prevent deleting environments which are still referenced by runs.
Fix
Don’t remove schedule settings when switching environments.
Fix
Docker health check.

version-1.7.0 (2024-01-11)

Add
Allow to add plugin extensions by searching for a contained node name (e.g. searching for “Column Expressions” will add org.knime.features.expressions.feature.group from the matching update site).
Add
Allow to add plugin extensions by searching for a product name (e.g. searching for “Palladian” will add the corresponding update site and all required features).
Add
Show notification about available NodePit Runner updates in menu and settings.
Add
Add passwordrules to allow compatible browsers to auto-generate passwords which conform to the rules when signing up.

version-1.6.1 (2024-01-05)

Add
Add Docker health check for API container.
Change
More user-friendly error loggging from Executor.
Fix
Prevent accidental submission when using Plugin Search when editing environments.
Fix
Ensure that Executor status is continuously reported during long workflow runs.

version-1.6.0 (2023-12-21)

Add
Allow to set preference key/value pairs additionally to entire .epf file.
Add
Color highlighting of log rows based on severity (error, warn, debug).
Fix
Wrap log lines in run logs instead of overflowing the text area.
Fix
Sorting of file list by filename.

version-1.5.0 (2023-12-14)

Change
Allow compatible workflows created with a future version of KNIME to run in a previous version of KNIME (e.g. workflows created with KNIME 5.2 will also run in KNIME 5.1).
Change
The artifacts archive now contains an unzipped version of the executed workflow.
Change
Remove file type limitations (i.e. all types of files can be uploaded in the “Additional Files” section).
Fix
Fixed an issue with paths relative to the current mountpoint, current workflow or current workflow's data area. Previously loading files with the "Relative to" node option failed with an exception (e.g. in the Excel Reader and Excel Writer nodes).
Fix
Keep runs’ log line breaks and whitespace such as indentation.
Fix
Prevent log appender exception Appender 'logfile' does not exist.

version-1.4.0 (2023-12-07)

Add
Added support for additional files in schedules. You can now upload additional files (e.g. JDBC drivers, licence files, models) that you want to be available on the executor during the execution of workflows or scripts. They are available under /nodepit with their original file names.
Change
Decrease log verbosity to put more focus on important messages in run logs.
Change
Set KNIME 5.2.0 as the default for new environments.
Fix
Prevent run exception when an invalid email configuration is given.

version-1.3.1 (2023-10-23)

Change
Set KNIME 5.1.2 as the default for new environments.
Fix
Fixed an issue that let to a failed run in case the workflow name contained whitespaces.

version-1.3.0 (2023-08-17)

Add
Added functionality to duplicate Environments and Schedules.
Change
Provided more files in the artifacts archive. The run configuration is now automatically included in the artifact archive.
Change
Improved explanation for license key input.
Change
Restyled license banner.
Fix
Fixed a problem that prevented double values from being configured as workflow parameters.
Fix
Prevented a permission error when running NodePit Runner as non-root user.
Fix
Prevented banners to obscure controls.

version-1.2.0 (2023-08-03)

Add
Show version information for API, Web, and Executors (see “Settings” and “Executor” sections).
Fix
Increased upload limit for large log data.
Fix
Improved error logging in Executor.
Fix
Prevented permission error when running as non-root user.

version-1.1.0 (2023-07-21)

Add
Added search input fields to executors, environments, files, schedules, and users.
Add
Added support for additional files in environments. You can now upload additional files (e.g. JDBC drivers, licence files, models) that you want to be available on the executor during the execution of workflows or scripts. They are available under /nodepit with their original file names.
Add
Added support for output files. Files that your workflow or script writes to /nodepit are now included in the artifact archive.
Change
Set KNIME 5.1.0 as the default for new environments.
Change
Provided more files in the artifacts archive. Additional files, preference files, and the actual run command are now automatically included in the artifact archive.
Fix
Fixed a cosmetic bug that caused the currently selected preference file to not be properly displayed in the drop-down menu.

version-1.0.1 (2023-05-30)

Add
Added an option to easily select your current time zone in the Settings menu. You no longer need to select your time zone from a long list of time zones.
Change
Restyled UI elements to increase the contrast between active and inactive elements.
Change
Improved executors to start new runs immediately. Previously it could take up to a minute for an executor to start a run.
Change
Set KNIME 4.7.3 as the default for new environments.
Change
Environments are now cached by default. Set the environment variable DOCKER_REMOVE_IMAGES to true at the executor level if you don't want to use caching for faster builds.
Fix
Fixed a problem with environment caching when using an external Docker registry. If you cache your environments via a Docker registry (e.g. Docker Hub), you will now experience more cache hits and therefore faster run times.
Fix
Fixed an issue when trying to push environment images to an insecure Docker registry.
Fix
Improved the description for some input fields.
Fix
Improved the log output for certain errors.

version-1.0.0 (2023-04-06)

Info
Initial release of NodePit Runner – Hello, world!