Icon

XX - Backup Knime Workspace

BACKUP READ ME Backup - How it works? 1. Set your Workspace root folder i.e. "E:/Knime-Workspace" 2. Indexes the workspace and filters out workflows with lock files. 3. Reads workflow-lastmod.table for delta processing. If none exists or the "Full" option is selected, all workflows are archived. 4. Generates 7-Zip commands based on configuration. 5. Uses a Python script to create archives. Restore - How it works? 1. Configfure the Interactive View and set the parent folder of your Knime Worksapce i.e. "E:\" 2. Restore-Workflow checks for "workflow-lastmod.table" and previews workflows based on config Note: If restore from AWS S3 was chosen, files get downloaded. This overwrites, if file is older, any identical file already present in this workflows data directory! 3. Auto creates missing parent folder if not exisitng, then restores archives Features - Auto download 7Zip if not existing - Uses workflow/workspace name for archive naming and 1:1 replication during restore. - Supports individual workflow archives or a single archive for the entire workspace - Skips workflows with lock files - Allows excluding data directories and node port data. - Delta processing by default, even with compression, thanks to 7-Zip. - Password protection, compression methods, and archive integrity validation (CRC check by default). - CRC-Archive Validation can be skipped (saved time when backing up large workspaces!) - Output compression and validation results - Restore from local and remote archives in AWS S3 - Preview of found Workflows in archives - Parallel Processing when chosing individual archive files - Moves files to final local destination Tested and confirmed on - Windows 11, Knime 5.4 in Classic UI - With and w/o data, password, local, remote, entire workspace and individual workflow archives Solution designed with Linux in mind BUT NOT TESTED on Linux / OSX! AWS S3 as sole transfer method as it meets my personal use case! Local option should work with NAS/DAS (Network/Direct attached Storage) How to Backup? Recommendation: Create a dedicated Backup Workspace! 1. Import workflow into your workspace 2. Configure (Interactive View) the Component "Get and set Values" & execute everything 2nd Component Bottom Port Workflows with lock files If Knime got force quit, there might be lock files in place you wnat to manually remove or just open, save and close these workflows! Quick Access Workflow Data-Directory RESTORE Remarks "Cannot connect to Python Process" Just retry. The Compress Files/Folder Node from Knime seems the have some issues. Hence, a Python-based approach, to trigger a cmd/bash command, is used. Export without Data Upon opening a transferred workflow an error is thrown about missing data which can be sasfely ignored Cause: Missing modifications in settings.xml: 1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED 2. Change <entry key="hasContent" type="xboolean" value="true"/> to false 3. And likely more related to entries under //config[@key="ports"]/config How to Extract Archives manually Extraction can trigger "File Path/Name Lenght exceeded" error. Most often when Windows File Explorer is involded by Drag & Drop the archive contents, which results in them getting extracted into a temp-folder prior to moving. Option 1: Install 7Zip > Right click on archive > Extract to Option 2: Using command line Without Password Schema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"[Path-TO-DESTINATION]" -spf2 Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"D:\" -spf2 With Password Schema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"E:\" -spf2 "-p[PASSWORD]}" Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"E:\" -spf2 "-pKnim3IsAw3som}" How to Restore? 1. Configure Interactive View 2. Execute everything Check Bottom Port for excluded Workflows! Create Archive Path and Command Read workflow-lastmod.table and create Rule for Delta Processsing Compress Workflows or whole Workspace List Workflows and get last modified date Transfer Archives Execute to auto-open Worklflow Data-Directory Get and set Values Get and set Values De-Compress Backups Download from AWS S3 BACKUP READ ME Backup - How it works? 1. Set your Workspace root folder i.e. "E:/Knime-Workspace" 2. Indexes the workspace and filters out workflows with lock files. 3. Reads workflow-lastmod.table for delta processing. If none exists or the "Full" option is selected, all workflows are archived. 4. Generates 7-Zip commands based on configuration. 5. Uses a Python script to create archives. Restore - How it works? 1. Configfure the Interactive View and set the parent folder of your Knime Worksapce i.e. "E:\" 2. Restore-Workflow checks for "workflow-lastmod.table" and previews workflows based on config Note: If restore from AWS S3 was chosen, files get downloaded. This overwrites, if file is older, any identical file already present in this workflows data directory! 3. Auto creates missing parent folder if not exisitng, then restores archives Features - Auto download 7Zip if not existing - Uses workflow/workspace name for archive naming and 1:1 replication during restore. - Supports individual workflow archives or a single archive for the entire workspace - Skips workflows with lock files - Allows excluding data directories and node port data. - Delta processing by default, even with compression, thanks to 7-Zip. - Password protection, compression methods, and archive integrity validation (CRC check by default). - CRC-Archive Validation can be skipped (saved time when backing up large workspaces!) - Output compression and validation results - Restore from local and remote archives in AWS S3 - Preview of found Workflows in archives - Parallel Processing when chosing individual archive files - Moves files to final local destination Tested and confirmed on - Windows 11, Knime 5.4 in Classic UI - With and w/o data, password, local, remote, entire workspace and individual workflow archives Solution designed with Linux in mind BUT NOT TESTED on Linux / OSX! AWS S3 as sole transfer method as it meets my personal use case! Local option should work with NAS/DAS (Network/Direct attached Storage) How to Backup? Recommendation: Create a dedicated Backup Workspace! 1. Import workflow into your workspace 2. Configure (Interactive View) the Component "Get and set Values" & execute everything 2nd Component Bottom Port Workflows with lock files If Knime got force quit, there might be lock files in place you wnat to manually remove or just open, save and close these workflows! Quick Access Workflow Data-Directory RESTORE Remarks "Cannot connect to Python Process" Just retry. The Compress Files/Folder Node from Knime seems the have some issues. Hence, a Python-based approach, to trigger a cmd/bash command, is used. Export without Data Upon opening a transferred workflow an error is thrown about missing data which can be sasfely ignored Cause: Missing modifications in settings.xml: 1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED 2. Change <entry key="hasContent" type="xboolean" value="true"/> to false 3. And likely more related to entries under //config[@key="ports"]/config How to Extract Archives manually Extraction can trigger "File Path/Name Lenght exceeded" error. Most often when Windows File Explorer is involded by Drag & Drop the archive contents, which results in them getting extracted into a temp-folder prior to moving. Option 1: Install 7Zip > Right click on archive > Extract to Option 2: Using command line Without Password Schema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"[Path-TO-DESTINATION]" -spf2 Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"D:\" -spf2 With Password Schema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"E:\" -spf2 "-p[PASSWORD]}" Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"E:\" -spf2 "-pKnim3IsAw3som}" How to Restore? 1. Configure Interactive View 2. Execute everything Check Bottom Port for excluded Workflows! Create Archive Path and Command Read workflow-lastmod.table and create Rule for Delta Processsing Compress Workflows or whole Workspace List Workflows and get last modified date Transfer Archives Execute to auto-open Worklflow Data-Directory Get and set Values Get and set Values De-Compress Backups Download from AWS S3

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Mike Wiegand
Created at: 2024-09-04
On NodePit since: 2024-10-05
Last update: 2026-11-13
Created with KNIME version: v5.4.2

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!