Icon

XX - Backup Knime Workspace

BACKUP READ MEHow it works1. Builds an index of the current workspace2. Identifies workflows with lock files and filters them out3. Reads the lastmodified table and, if none exists or Full option was selected, passes all though4. Creates 7zip commands based on configuration5. Uses a python script to craete archivesTested and confirmed on- Windows 11- Knime 5.3.3 in Classic UI- With and w/o data, password, local, remote, entire workspace and individual workflow archivesRemarks1. Solution designed with Linux in mind BUT NOT TESTED on Linux / OSX!2. AWS S3 is sole transfer method as it meets my personal use case!Features- Manually select Workspace Location- Create an archive for each individual workflow- or one the entire Knime workspace- Skips workflows with lock files- Allows to exclude data-directory and node port data- Delta processing by default, also when compression is active thanks to 7Zip!- Ability to set password- Choose compression Method- Archive Integrity CRC-Check by default thanks to 7Zip!- Output of compression and validation- Restore from local archives- Parallel Processing when chosing individual archive files- Designed to work with little to no extra extensions (only Vernalis bcs. it`s awesome!)- Dynamically uses workflow / worspace name- Always includes, as I did not find a solution, some settings like workspace tab arrangement, recent workflows or SeleniumLicense (bug, not a feature!?!) Bottom Port - Workflowswith lock filesIf Knime got force quit, theremight be lock files in place youwnat to manually remove or justopen, save and close theseworkflows! Additional RemarksA. Export without Data: Upon opening a transferred workflow an error is thrown about missing data which can be sasfelyignoredCause: Missing modifications in settings.xml:1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED2 Change <entry key="hasContent" type="xboolean" value="true"/> to false3. And likely more related to entries under //config[@key="ports"]/configB. Export skips Workflows with lock file triggers: Error "Unable to open workflow XYZ. Location does not exist" once uponopening Knme. Cause not identified yet.C. Workspacse settings such as workspace tab arrangement, recent workflows or Selenium License are transfered as part ofthe workflow. Cause partially identified but not fully understood ... so far, a feature, nto a bug ;-)Reset Perspective if necessary: Knime Menu "View">"Resset Perspective"D. The Compress Files/Folder Node from Knime seems the have some issues. Hence, a Python-based approach, to trigger acmd/bash command, is used. Quick AccessWorkflow Data-Directory REQUIREDExecute to download 7Zip CMD Executable intoWorkflow Data Directory Process1. Import workflow to your workspace2. Close all workflows except this3. Configure (Interactive View) the Component "Get and set Values"& execute everything5. Transfer files to your backup drive or extract into workspace of another systemHow to Extract Archives manuallyExtraction can trigger "File Path/Name Lenght exceeded" error. Most often when Windows File Explorer is involded by Drag &Drop the archive contents, which results in them getting extracted into a temp-folder prior to moving.Option 1: Install 7Zip > Right click on archive > Extract toOption 2: Using command lineWithout PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"[Path-TO-DESTINATION]" -spf2Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"D:\" -spf2With PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"E:\" -spf2 "-p[PASSWORD]}"Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"E:\" -spf2 "-pKnim3IsAw3som}" RESTORE CheckBotom PortSourcehttps://7-zip.org/download.htmlCreate ArchivePath and Command Read workflow-lastmod.table andcreate Rule for Delta Processsing Compress Workflowsor whole Workspace List Workflows and getlast modified date Transfer Archives Execute to auto-openWorklflow Data-Directory Transfer Files Get and set Values Get and set Values De-Compress Backups BACKUP READ MEHow it works1. Builds an index of the current workspace2. Identifies workflows with lock files and filters them out3. Reads the lastmodified table and, if none exists or Full option was selected, passes all though4. Creates 7zip commands based on configuration5. Uses a python script to craete archivesTested and confirmed on- Windows 11- Knime 5.3.3 in Classic UI- With and w/o data, password, local, remote, entire workspace and individual workflow archivesRemarks1. Solution designed with Linux in mind BUT NOT TESTED on Linux / OSX!2. AWS S3 is sole transfer method as it meets my personal use case!Features- Manually select Workspace Location- Create an archive for each individual workflow- or one the entire Knime workspace- Skips workflows with lock files- Allows to exclude data-directory and node port data- Delta processing by default, also when compression is active thanks to 7Zip!- Ability to set password- Choose compression Method- Archive Integrity CRC-Check by default thanks to 7Zip!- Output of compression and validation- Restore from local archives- Parallel Processing when chosing individual archive files- Designed to work with little to no extra extensions (only Vernalis bcs. it`s awesome!)- Dynamically uses workflow / worspace name- Always includes, as I did not find a solution, some settings like workspace tab arrangement, recent workflows or SeleniumLicense (bug, not a feature!?!) Bottom Port - Workflowswith lock filesIf Knime got force quit, theremight be lock files in place youwnat to manually remove or justopen, save and close theseworkflows! Additional RemarksA. Export without Data: Upon opening a transferred workflow an error is thrown about missing data which can be sasfelyignoredCause: Missing modifications in settings.xml:1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED2 Change <entry key="hasContent" type="xboolean" value="true"/> to false3. And likely more related to entries under //config[@key="ports"]/configB. Export skips Workflows with lock file triggers: Error "Unable to open workflow XYZ. Location does not exist" once uponopening Knme. Cause not identified yet.C. Workspacse settings such as workspace tab arrangement, recent workflows or Selenium License are transfered as part ofthe workflow. Cause partially identified but not fully understood ... so far, a feature, nto a bug ;-)Reset Perspective if necessary: Knime Menu "View">"Resset Perspective"D. The Compress Files/Folder Node from Knime seems the have some issues. Hence, a Python-based approach, to trigger acmd/bash command, is used. Quick AccessWorkflow Data-Directory REQUIREDExecute to download 7Zip CMD Executable intoWorkflow Data Directory Process1. Import workflow to your workspace2. Close all workflows except this3. Configure (Interactive View) the Component "Get and set Values"& execute everything5. Transfer files to your backup drive or extract into workspace of another systemHow to Extract Archives manuallyExtraction can trigger "File Path/Name Lenght exceeded" error. Most often when Windows File Explorer is involded by Drag &Drop the archive contents, which results in them getting extracted into a temp-folder prior to moving.Option 1: Install 7Zip > Right click on archive > Extract toOption 2: Using command lineWithout PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"[Path-TO-DESTINATION]" -spf2Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"D:\" -spf2With PasswordSchema: ./7zr.exe x "[PATH-TO-7ZIP-ARCHIVE]" -y -aoa -o"E:\" -spf2 "-p[PASSWORD]}"Example: ./7zr.exe x "E:\Knime-Backup\Data-Included-No\Workflow-Name.7z" -y -aoa -o"E:\" -spf2 "-pKnim3IsAw3som}" RESTORE CheckBotom PortSourcehttps://7-zip.org/download.htmlCreate ArchivePath and Command Read workflow-lastmod.table andcreate Rule for Delta Processsing Compress Workflowsor whole Workspace List Workflows and getlast modified date Transfer Archives Execute to auto-openWorklflow Data-Directory Transfer Files Get and set Values Get and set Values De-Compress Backups

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Mike Wiegand
Created at: 2024-09-04
On NodePit since: 2024-10-05
Last update: 2024-12-17
Created with KNIME version: v5.3.3

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!