Icon

02_​calculate_​InceptionV3_​features

01_caption_preprocessing
Point to folder containingcoco train2014 images: Seeworkflow description for adownload link. KNIME Deep Learning - Simple Image CaptioningIn this series of workflows, we want to demonstrate how to caption images using KNIME Deep Learning - Keras Integration. In image captioning the goalis to create a small textual description from just raw image data without additional information.This example is based on the COCO dataset (see http://cocodataset.org/#home), specifically on a subset of the 2014 data. This subset is defined in thefile coco_annotations_sample.csv located in the data folder of this workflow group. The 2014 data can be downloaded here: http://images.cocodataset.org/zips/train2014.zipThe workflows in this series need to be ran in order as they create data required for the execution of later workflows.Please note: The workflow series is heavily inspired by the great blog-post of Harshall Lamba (see https://towardsdatascience.com/image-captioning-with-keras-teaching-computers-to-describe-pictures-c88a46a311b8)1. Workflow 01 Caption Preprocessing2. Workflow 02 Calculate InceptionV3 Features: In this workflow we create image feature vectors for each image in the training data using a pretrainedInceptionV3 (https://arxiv.org/abs/1512.00567) model. These feature vectors will then be used in workflow 4. to train our caption network. This way we donot need to include a computationally expensive convolutional branch in our network architecture. As output, this workflow will write a table containing afeature vector for each image to the data folder.3. Workflow 03 Create GLOVE Vector Dictionary4. Workflow 04 Train Model5. Workflow 05 InferenceIn order to run the example, please make sure you have the following KNIME extensions installed:- KNIME Deep Learning - Keras Integration (Labs)- KNIME Image Processing (Community Contributions Trusted)- KNIME Image Processing - Deep Learning Extension (Community Contributions Trusted)- KNIME Streaming Execution (Labs)- KNIME TextprocessingYou also need a local Python installation that includes Keras. Please refer to https://www.knime.com/deeplearning#keras for installationrecommendations and further information. Read coco image IDs(predefined sample)Calculate imagefeaturesWrite image featuresto dada dirExtract file nameDownload InceptionV3network from Keras List Files Joiner DuplicateRow Filter CSV Reader Chunk Loop Start Keras NetworkExecutor Loop End Table Writer Joiner Column Expressions Read andPreprocess Images Create ImageFeauture Collections DL PythonNetwork Creator Point to folder containingcoco train2014 images: Seeworkflow description for adownload link. KNIME Deep Learning - Simple Image CaptioningIn this series of workflows, we want to demonstrate how to caption images using KNIME Deep Learning - Keras Integration. In image captioning the goalis to create a small textual description from just raw image data without additional information.This example is based on the COCO dataset (see http://cocodataset.org/#home), specifically on a subset of the 2014 data. This subset is defined in thefile coco_annotations_sample.csv located in the data folder of this workflow group. The 2014 data can be downloaded here: http://images.cocodataset.org/zips/train2014.zipThe workflows in this series need to be ran in order as they create data required for the execution of later workflows.Please note: The workflow series is heavily inspired by the great blog-post of Harshall Lamba (see https://towardsdatascience.com/image-captioning-with-keras-teaching-computers-to-describe-pictures-c88a46a311b8)1. Workflow 01 Caption Preprocessing2. Workflow 02 Calculate InceptionV3 Features: In this workflow we create image feature vectors for each image in the training data using a pretrainedInceptionV3 (https://arxiv.org/abs/1512.00567) model. These feature vectors will then be used in workflow 4. to train our caption network. This way we donot need to include a computationally expensive convolutional branch in our network architecture. As output, this workflow will write a table containing afeature vector for each image to the data folder.3. Workflow 03 Create GLOVE Vector Dictionary4. Workflow 04 Train Model5. Workflow 05 InferenceIn order to run the example, please make sure you have the following KNIME extensions installed:- KNIME Deep Learning - Keras Integration (Labs)- KNIME Image Processing (Community Contributions Trusted)- KNIME Image Processing - Deep Learning Extension (Community Contributions Trusted)- KNIME Streaming Execution (Labs)- KNIME TextprocessingYou also need a local Python installation that includes Keras. Please refer to https://www.knime.com/deeplearning#keras for installationrecommendations and further information. Read coco image IDs(predefined sample)Calculate imagefeaturesWrite image featuresto dada dirExtract file nameDownload InceptionV3network from KerasList Files Joiner DuplicateRow Filter CSV Reader Chunk Loop Start Keras NetworkExecutor Loop End Table Writer Joiner Column Expressions Read andPreprocess Images Create ImageFeauture Collections DL PythonNetwork Creator

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: marcel
Created at: 2017-07-11
On NodePit since: 2020-05-15
Last update: 2024-04-24
Created with KNIME version: v4.1.2
Tags: image processingimage analysisimage captioningdeep learningneural networksGLOVEtransfer learningword embedding

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!