Icon

03_​create_​glove_​vector_​dictionary

01_caption_preprocessing
KNIME Deep Learning - Simple Image CaptioningIn this series of workflows, we want to demonstrate how to caption images using KNIME Deep Learning - Keras Integration. In image captioning the goalis to create a small textual description from just raw image data without additional information.This example is based on the COCO dataset (see http://cocodataset.org/#home), specifically on a subset of the 2014 data. This subset is defined in thefile coco_annotations_sample.csv located in the data folder of this workflow group. The 2014 data can be downloaded here: http://images.cocodataset.org/zips/train2014.zipThe workflows in this series need to be ran in order as they create data required for the execution of later workflows.Please note: The workflow series is heavily inspired by the great blog-post of Harshall Lamba (see https://towardsdatascience.com/image-captioning-with-keras-teaching-computers-to-describe-pictures-c88a46a311b8)1. Workflow 01 Caption Preprocessing2. Workflow 02 Calculate InceptionV3 Features3. Workflow 03 Create GLOVE Vector Dictionary: In this workflow we look-up a GLOVE embedding vector (https://nlp.stanford.edu/projects/glove/) for eachword in our caption vocabulary. Like the previous workflow, these can be used as word features to train our caption model and remove the need to train anown embedding. First, the GLOVE model is downloaded and extracted to the data folder. The vectors are simply contained in a text file. The actual look-upis then performed in a Python Script node, that creates a Python dictionary mapping from each word of the vocabulary to the corresponding GLOVE vector.This Python dictionary will then directly be written to the data folder within the Python Script, as we need this format in the next workflow during modeldefinition.4. Workflow 04 Train Model5. Workflow 05 InferenceIn order to run the example, please make sure you have the following KNIME extensions installed:- KNIME Deep Learning - Keras Integration (Labs)- KNIME Image Processing (Community Contributions Trusted)- KNIME Image Processing - Deep Learning Extension (Community Contributions Trusted)- KNIME Streaming Execution (Labs)- KNIME TextprocessingYou also need a local Python installation that includes Keras. Please refer to https://www.knime.com/deeplearning#keras for installationrecommendations and further information. Select 200 dimGLOVE modelRead GLOVEmodel paths Read vocabularyLook up GLOVE vectorfor each vocabulary wordand save as .pickle fileConstruct GLOVEdict pathRow Filter List Files Table Rowto Variable Table Reader Python Script (1⇒1) Column Expressions DownloadGLOVE Model KNIME Deep Learning - Simple Image CaptioningIn this series of workflows, we want to demonstrate how to caption images using KNIME Deep Learning - Keras Integration. In image captioning the goalis to create a small textual description from just raw image data without additional information.This example is based on the COCO dataset (see http://cocodataset.org/#home), specifically on a subset of the 2014 data. This subset is defined in thefile coco_annotations_sample.csv located in the data folder of this workflow group. The 2014 data can be downloaded here: http://images.cocodataset.org/zips/train2014.zipThe workflows in this series need to be ran in order as they create data required for the execution of later workflows.Please note: The workflow series is heavily inspired by the great blog-post of Harshall Lamba (see https://towardsdatascience.com/image-captioning-with-keras-teaching-computers-to-describe-pictures-c88a46a311b8)1. Workflow 01 Caption Preprocessing2. Workflow 02 Calculate InceptionV3 Features3. Workflow 03 Create GLOVE Vector Dictionary: In this workflow we look-up a GLOVE embedding vector (https://nlp.stanford.edu/projects/glove/) for eachword in our caption vocabulary. Like the previous workflow, these can be used as word features to train our caption model and remove the need to train anown embedding. First, the GLOVE model is downloaded and extracted to the data folder. The vectors are simply contained in a text file. The actual look-upis then performed in a Python Script node, that creates a Python dictionary mapping from each word of the vocabulary to the corresponding GLOVE vector.This Python dictionary will then directly be written to the data folder within the Python Script, as we need this format in the next workflow during modeldefinition.4. Workflow 04 Train Model5. Workflow 05 InferenceIn order to run the example, please make sure you have the following KNIME extensions installed:- KNIME Deep Learning - Keras Integration (Labs)- KNIME Image Processing (Community Contributions Trusted)- KNIME Image Processing - Deep Learning Extension (Community Contributions Trusted)- KNIME Streaming Execution (Labs)- KNIME TextprocessingYou also need a local Python installation that includes Keras. Please refer to https://www.knime.com/deeplearning#keras for installationrecommendations and further information. Select 200 dimGLOVE modelRead GLOVEmodel paths Read vocabularyLook up GLOVE vectorfor each vocabulary wordand save as .pickle fileConstruct GLOVEdict pathRow Filter List Files Table Rowto Variable Table Reader Python Script (1⇒1) Column Expressions DownloadGLOVE Model

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: dave
Created at: 2020-03-31
On NodePit since: 2020-05-15
Last update: 2024-04-23
Created with KNIME version: v4.1.2
Tags: image processingimage analysisimage captioningdeep learningneural networksGLOVEtransfer learningword embedding

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!