Icon

XX - Backup Knime Workspace

Ideas- Restore from remote source - Check Java Command from Knime source code to trigger Export- Command line compress for CRC check- Commit to repository- Generate report with details about size of data-dirctory and port-data- Parallel Processing if individual Workflows are compresssedNote: Possible bug reported.See https://forum.knime.com/t/parallel-chunk-nodes-could-not-connect-to-the-python-process/83585 Write data of recently archivedworkflows, for delta processing,after sucessful processing. READ ME Tested and confirmed on- Windows 11- Knime 5.3.2 in Classic UI- With and w/o data- Full workspace and individual workflow archives1. Solution designed with Linux in mind BUT NOT TESTED on Linux / OSX!2. AWS S3 is sole transfer method as it meets my personal use case!Features- Create individual workflow- or one single workspace archive- Skips workflows with lock files- Allows to exclude data-directory and / or node port data- Delta processing- Designed to work with little to no extra extensions (only Vernalis bcs. it`s awesome!)- Dynamically uses workflow / worspace name- Always includes, as I did not find a solution, some settings like workspace tabarrangement, recent workflows or Selenium License (bug, not a feature!?!)Process1. Import workflow to your workspace2. Close all workflows except this3. Configure (Interactive View) the Component "Get and set Values"4. Execute everything5. Transfer files to your backup drive or extract into workspace of another systemHow to Extract ArchivesExtraction can trigger "File Path/Name Lenght exceeded" error i.e. when using WinRar.Hence, exctraction via CMD is required:Schema: tar -xvzf [PATH to Archive].tar.gz -C [Path to destination]Example: tar -xvzf Knime-Workspace.tar.gz -C D:/How it works1. Builds an index of the current workspace2. Identifies workflows with lock files and filters them out3. Reads the lastmodified table and if none exists passes all though4. Creates tar commands based on configuration5. Uses a python script to craete archives Bottom Port - Workflowswith lock filesIf Knime got force quit, theremight be lock files in place youwnat to manually remove or justopen, save and close theseworkflows! Additional RemarksA. Export without Data: Upon opening a transferred workflow an error is thrown aboutmissing data which can be sasfely ignoredCause: Missing modifications in settings.xml:1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED2 Change <entry key="hasContent" type="xboolean" value="true"/> to false3. And likely more related to entries under //config[@key="ports"]/configB. Export skips Workflows with lock file triggers: Error "Unable to open workflow XYZ.Location does not exist" once upon opening Knme. Cause not identified yet.C. Workspacse settings such as workspace tab arrangement, recent workflows orSelenium License are transfered as part of the workflow. Cause partially identified but notfully understood ... so far, a feature, nto a bug ;-)Reset Perspective if necessary: Knime Menu "View">"Resset Perspective"D. The Compress Files/Folder Node from Knime seems the have some issues. Hence, aPython-based approach, to trigger a cmd/bash command, is used. Get and set Values Create ArchivePath and Command Read workflow-lastmod.table andcreate Rule for Delta Processsing Compress Workflowsor whole Workspace List Workflows and getlast modified date Transfer Archives Ideas- Restore from remote source - Check Java Command from Knime source code to trigger Export- Command line compress for CRC check- Commit to repository- Generate report with details about size of data-dirctory and port-data- Parallel Processing if individual Workflows are compresssedNote: Possible bug reported.See https://forum.knime.com/t/parallel-chunk-nodes-could-not-connect-to-the-python-process/83585 Write data of recently archivedworkflows, for delta processing,after sucessful processing. READ ME Tested and confirmed on- Windows 11- Knime 5.3.2 in Classic UI- With and w/o data- Full workspace and individual workflow archives1. Solution designed with Linux in mind BUT NOT TESTED on Linux / OSX!2. AWS S3 is sole transfer method as it meets my personal use case!Features- Create individual workflow- or one single workspace archive- Skips workflows with lock files- Allows to exclude data-directory and / or node port data- Delta processing- Designed to work with little to no extra extensions (only Vernalis bcs. it`s awesome!)- Dynamically uses workflow / worspace name- Always includes, as I did not find a solution, some settings like workspace tabarrangement, recent workflows or Selenium License (bug, not a feature!?!)Process1. Import workflow to your workspace2. Close all workflows except this3. Configure (Interactive View) the Component "Get and set Values"4. Execute everything5. Transfer files to your backup drive or extract into workspace of another systemHow to Extract ArchivesExtraction can trigger "File Path/Name Lenght exceeded" error i.e. when using WinRar.Hence, exctraction via CMD is required:Schema: tar -xvzf [PATH to Archive].tar.gz -C [Path to destination]Example: tar -xvzf Knime-Workspace.tar.gz -C D:/How it works1. Builds an index of the current workspace2. Identifies workflows with lock files and filters them out3. Reads the lastmodified table and if none exists passes all though4. Creates tar commands based on configuration5. Uses a python script to craete archives Bottom Port - Workflowswith lock filesIf Knime got force quit, theremight be lock files in place youwnat to manually remove or justopen, save and close theseworkflows! Additional RemarksA. Export without Data: Upon opening a transferred workflow an error is thrown aboutmissing data which can be sasfely ignoredCause: Missing modifications in settings.xml:1. Change <entry key="state" type="xstring" value="EXECUTED"/> to CONFIGURED2 Change <entry key="hasContent" type="xboolean" value="true"/> to false3. And likely more related to entries under //config[@key="ports"]/configB. Export skips Workflows with lock file triggers: Error "Unable to open workflow XYZ.Location does not exist" once upon opening Knme. Cause not identified yet.C. Workspacse settings such as workspace tab arrangement, recent workflows orSelenium License are transfered as part of the workflow. Cause partially identified but notfully understood ... so far, a feature, nto a bug ;-)Reset Perspective if necessary: Knime Menu "View">"Resset Perspective"D. The Compress Files/Folder Node from Knime seems the have some issues. Hence, aPython-based approach, to trigger a cmd/bash command, is used. Get and set Values Create ArchivePath and Command Read workflow-lastmod.table andcreate Rule for Delta Processsing Compress Workflowsor whole Workspace List Workflows and getlast modified date Transfer Archives

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Mike Wiegand
Created at: 2024-09-04
On NodePit since: 2024-10-05
Last update: 2024-10-18
Created with KNIME version: v5.3.2

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!