Icon

XX - Backup Knime Workspace

BACKUP READ MEBackup - How it works?1. Set your Workspace root folder i.e. "E:/Knime-Workspace"2. Builds an index of the selected workspace & identifies workflows with lock files to filter them out3. Reads the "workflow-lastmod.table" for delta process and, if none exists or Full option was selected,passes all Workflows though for archiving4. Creates 7zip commands based on configuration5. Uses a python script to create archivesRestore - How it works?1. Configfure the Interactive View and set the parent folder of your Knime Worksapce i.e. "E:\"2. Restore-Workflow checks for "workflow-lastmod.table" and previews workflows based on configNote: If restore from AWS S3 was chosen, files get downloaded. This overwrites, if file is older, any identicalfile already present in this workflows data directory!3. Auto creates missing parent folder if not exisitng, then restores archivesFeatures- Auto download 7Zip if not existing- Uses workflow / worspace name for archive name and during restore for 1-to-1 replication- Creates an archive for each individual workflow- or a single one containing your entire Knime workspace- Skips workflows with lock files- Allows to exclude data-directory and node port data- Delta processing by default, also when compression is active thanks to 7Zip!- Ability to set password, compression method, test archive integrity CRC-Check by default!- Output compression and validation results- Restore from local and remote archives in AWS S3- Preview of found Workflows in archives- Parallel Processing when chosing individual archive filesTested and confirmed on- Windows 11, Knime 5.4 in Classic UI- With and w/o data, password, local, remote, entire workspace and individual workflow archivesSolution designed with Linux in mind BUT NOT TESTED on Linux / OSX! AWS S3 as sole transfer methodas it meets my personal use case! Local option should work with NAS/DAS (Network/Direct attachedStorage) How to Backup?Recommendation: Create a dedicated BackupWorkspace!1. Import workflow into your workspace2. Configure (Interactive View) the Component "Get andset Values"& execute everything2nd Component Bottom PortWorkflows with lock filesIf Knime got force quit, there might be lock files in placeyou wnat to manually remove or just open, save and closethese workflows! Quick AccessWorkflow Data-Directory RESTORE Remarks"Cannot connect to Python Process"Just retry. The Compress Files/Folder Node from Knime seems the have some issues. Hence, a Python-based approach, to trigger a cmd/bash command, is used.Export without DataUpon opening a transferred workflow an error is thrown about missing data which can be sasfely ignoredCause: Missing modifications in settings.xml:1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED2. Change <entry key="hasContent" type="xboolean" value="true"/> to false3. And likely more related to entries under //config[@key="ports"]/configHow to Extract Archives manuallyExtraction can trigger "File Path/Name Lenght exceeded" error. Most often when Windows File Explorer isinvolded by Drag & Drop the archive contents, which results in them getting extracted into a temp-folder priorto moving.Option 1: Install 7Zip > Right click on archive > Extract toOption 2: Using command lineWithout PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"[Path-TO-DESTINATION]" -spf2Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"D:\" -spf2With PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"E:\" -spf2 "-p[PASSWORD]}"Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"E:\" -spf2 "-pKnim3IsAw3som}" How to Restore?1. Configure Interactive View2. Execute everything Check Bottom Portfor excluded Workflows!Create ArchivePath and Command Read workflow-lastmod.table andcreate Rule for Delta Processsing Compress Workflowsor whole Workspace List Workflows and getlast modified date Transfer Archives Execute to auto-openWorklflow Data-Directory Get and set Values Get and set Values De-Compress Backups Downloadfrom AWS S3 BACKUP READ MEBackup - How it works?1. Set your Workspace root folder i.e. "E:/Knime-Workspace"2. Builds an index of the selected workspace & identifies workflows with lock files to filter them out3. Reads the "workflow-lastmod.table" for delta process and, if none exists or Full option was selected,passes all Workflows though for archiving4. Creates 7zip commands based on configuration5. Uses a python script to create archivesRestore - How it works?1. Configfure the Interactive View and set the parent folder of your Knime Worksapce i.e. "E:\"2. Restore-Workflow checks for "workflow-lastmod.table" and previews workflows based on configNote: If restore from AWS S3 was chosen, files get downloaded. This overwrites, if file is older, any identicalfile already present in this workflows data directory!3. Auto creates missing parent folder if not exisitng, then restores archivesFeatures- Auto download 7Zip if not existing- Uses workflow / worspace name for archive name and during restore for 1-to-1 replication- Creates an archive for each individual workflow- or a single one containing your entire Knime workspace- Skips workflows with lock files- Allows to exclude data-directory and node port data- Delta processing by default, also when compression is active thanks to 7Zip!- Ability to set password, compression method, test archive integrity CRC-Check by default!- Output compression and validation results- Restore from local and remote archives in AWS S3- Preview of found Workflows in archives- Parallel Processing when chosing individual archive filesTested and confirmed on- Windows 11, Knime 5.4 in Classic UI- With and w/o data, password, local, remote, entire workspace and individual workflow archivesSolution designed with Linux in mind BUT NOT TESTED on Linux / OSX! AWS S3 as sole transfer methodas it meets my personal use case! Local option should work with NAS/DAS (Network/Direct attachedStorage) How to Backup?Recommendation: Create a dedicated BackupWorkspace!1. Import workflow into your workspace2. Configure (Interactive View) the Component "Get andset Values"& execute everything2nd Component Bottom PortWorkflows with lock filesIf Knime got force quit, there might be lock files in placeyou wnat to manually remove or just open, save and closethese workflows! Quick AccessWorkflow Data-Directory RESTORE Remarks"Cannot connect to Python Process"Just retry. The Compress Files/Folder Node from Knime seems the have some issues. Hence, a Python-based approach, to trigger a cmd/bash command, is used.Export without DataUpon opening a transferred workflow an error is thrown about missing data which can be sasfely ignoredCause: Missing modifications in settings.xml:1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED2. Change <entry key="hasContent" type="xboolean" value="true"/> to false3. And likely more related to entries under //config[@key="ports"]/configHow to Extract Archives manuallyExtraction can trigger "File Path/Name Lenght exceeded" error. Most often when Windows File Explorer isinvolded by Drag & Drop the archive contents, which results in them getting extracted into a temp-folder priorto moving.Option 1: Install 7Zip > Right click on archive > Extract toOption 2: Using command lineWithout PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"[Path-TO-DESTINATION]" -spf2Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"D:\" -spf2With PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"E:\" -spf2 "-p[PASSWORD]}"Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"E:\" -spf2 "-pKnim3IsAw3som}" How to Restore?1. Configure Interactive View2. Execute everything Check Bottom Portfor excluded Workflows!Create ArchivePath and Command Read workflow-lastmod.table andcreate Rule for Delta Processsing Compress Workflowsor whole Workspace List Workflows and getlast modified date Transfer Archives Execute to auto-openWorklflow Data-Directory Get and set Values Get and set Values De-Compress Backups Downloadfrom AWS S3

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Mike Wiegand
Created at: 2024-09-04
On NodePit since: 2024-10-05
Last update: 2025-01-20
Created with KNIME version: v5.4.0

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!