Icon

Challenge 17 - Prepare KNIME Training File Set

<p>Challenge 17 - Prepare KNIME Training File Set</p><p><strong>Level:</strong> Medium<br><br><strong>Description</strong>: You are responsible for creating an educational program for KNIME to be conducted within your company. You plan to create a workflow where participants can learn how to use the "<strong>List Files/Folders</strong>" node and its utility. <strong>Hint:</strong> When generating random numbers, the <strong>Vernalis KNIME Nodes Extension</strong> might be helpful.<br><br>As preparation, you need to create dummy data that meets a few requirements, described as follows.<br><br><em>Beginner-friendly objective</em>s:<br>- Create a specified number of folders (up to 5,000) with unique random numbers of up to 6 digits within a designated directory.<br>- Use zero-padding to ensure the numbers have consistent digit length.<br><br><em>Intermediate-friendly objective:</em><br>- Inside each folder, create one file with a random number of up to 5 digits as its name. Zero-padding is not required, and the file names do not need to be unique.<br>- The file extension should be (1) the fictional file extension <strong>.fzbz</strong> if the folder name is a multiple of 3 <strong>and</strong> the file name contains the number 5, or (2) <strong>.csv</strong> for all other cases. The contents of each file can be empty.</p><p><strong>Solution Summary: </strong>To solve this challenge, we create a combination of directories and files, then assign the appropriate extensions based on the conditions to generate file paths. After that, create a dummy file without an extension and copy it according to the designated file paths.</p><p></p><p><strong>Solution Details</strong>:First, two Random Number Generator Nodes are used to generate random numbers for folders and files. The Joiner Node then merges them into a single table, and the Expression Node executes three expressions.<em>Exp1.K-AI: The following script is entered and executed:"If the value in the Directory column is a multiple of 3 and the filename column contains '5', set the extension to '.fzbz'; otherwise, set it to '.csv'."This determines the appropriate file extension.Exp2.K-AI: The following script is entered and executed:"Format Directory column values with leading zeros to make them 6 digits."This ensures that the folder name is fixed at six digits.Exp3: Combines the existing columns with the results from Exp1 and Exp2 to create folder/file paths.These file paths are created under the directory specified later by the Local File System Connector.</em>Since the generated file paths are in String format, they are converted to Path type using the String to Path Node.Additionally, one dummy file is created using the following process.<em>Create Temp Folder Node: Creates a temporary folder that will be deleted upon reset. Variables for specifying the dummy file path are also created.Table Creator Node: Creates an empty table.CSV Writer Node: Saves the table as a dummy file at the path specified by theVariables.Path to String (Variable) &amp; URL to File Path (Variable) Nodes: Convert the dummy file path to String format.</em>Next, a table containing Training File Paths is processed. The Constant Value Column Node is used to assign the dummy file path to all rows. The String to Path Node is then used to convert the dummy file path back to Path format.Finally, the Local File System Connector Node sets the root directory, and the Transfer Files (Table) Node creates the Training File Set.The Source column is set to the dummy file path.The Destination column is set to "From Table (Destination column: file path)".</p><p>This ensures that the files are created in the desired structure.</p><p></p><p></p>

Nodes

Extensions

Links