Ollama KNIME - Multimodal Prompting - Car Accident Severity Analysis

Import textual data (car accident insurance claims) and images (accident images)and combine them into a multimodal prompt

Set up a dummy OpenAI API key, and select the local LLM using Ollama

For each claim, sendthe multimodal prompt to an LLM to assess severity

Multimodal Prompting - Car Accident Severity Analysis

This workflow demonstrates how to prompt an LLM using both text and images to assess car accident severity.

Textual data from insurance claims and corresponding accident images are imported, combined into row-wise multimodal prompts, and sent to a connected LLM. The model then returns a severity assessment for each accident based on both the text and image inputs.

ollama pull qwen3:8b

ollama pull mistral-small3.1:latest
actually seem to provide descriptions of the images (https://ollama.com/library/mistral-small3.1)

adapted from "02 Multimodal prompting" - you will find also more data there

https://hub.knime.com/s/8acGwIQTQmAJM1q9

see discussion on the KNIME forum - also how to install Ollama and configure it for KNIME usage

https://forum.knime.com/t/llm-prompter-node-genai-data-workflow-error/90020/3?u=mlauber71

Extract and Format the response from the JSON file created by the LLM