Icon

Explain KPI report with multimodal LLMs

<p><strong>KPIs report explanation using Gen AI</strong></p><p>This workflow uses multimodal local LLMs (LLaMa3.2:1b and LLaMa3.2-vision) to explain a KPI report containing information (in the form of different plots and charts) regarding a simulated software company.</p><p></p><p><strong>IMPORTANT</strong>: These local models are significantly simpler compared to other available models (like OpenAI's), hence also their performance are not on the same level. However, the fact that they can be run locally ensures <strong>protection of the company data in every processing step</strong>.</p><p></p><p>Following the instructions below, you can learn how to use the workflow and how to modify it if you need to adapt this workflow to other use cases.</p><p><strong>HOW TO USE THE WORKFLOW:</strong></p><ol><li><p>Run "ollama run <strong>llama3.2:1b" </strong>on your command prompt line to enable the text parsing from a PDF report in Step 1</p></li><li><p>Run "ollama run <strong>llama3.2-vision</strong>"<strong> </strong>always on your command prompt line to enable the plot and chart interpretation in Step 2</p></li><li><p>As suggested, in Step 3 refresh the session before the refinement prompt</p></li><li><p>Export the <strong>report </strong>with the KPIs interpretations and recommendations</p></li></ol><p><strong>Notes if you need a different use case:</strong></p><p>If you need to change the workflow for a different use case, the following recommendations may help:</p><ul><li><p>Modify the text extraction (both cleaning steps and prompt) to fit the schema of the new document</p></li><li><p>Explore different models to optimize the performance in the new task</p></li><li><p>Adapt the following prompts for the explanation</p></li><li><p>Modify the output report as needed to fit your use case</p></li></ul>

URL: How to leverage open source LLMs locally via Ollama https://www.knime.com/blog/how-to-leverage-open-source-llms-ollama

1) AI-parsed content from PDF report

In the TIKA parser node, configure the path to the pdf folder containing the for the KPIs. If other files are present, they will also be read.

KPIs report explanation using GenAI


This workflow uses multimodal local LLMs (LLaMa3.2:1b and LLaMa3.2-vision) to explain a KPI report containing information (in the form of different plots and charts) regarding a simulated software company.

2) AI-generated plot interpretations

The prompt asks the model (LLaMa3.2-vision) to interpret the plots and charts extracted from the KPI report.

Subdividing the task in simpler reasoning steps within the same prompt via Chain of Thought helps improve model responses.

4) Report writing

Here the output of the model is inserted with the corresponding image inside the report.

Parameterized prompt

Since the models are local, execution may require long time. This is particularly true when processing images.

Notes on the workflow:

In order to correctly run the workflow, ensure the following:

  1. Run "ollama run llama3.2:1b on your command prompt line to enable the text parsing from a PDF (Step 1)

  2. Run "ollama run llama3.2-vision" always on your command prompt line to enable the graphts interpretation in Step 2

  3. As suggested in section 3, refresh the session before the refinement prompt, so use the commands "/bye" and once more "ollama run llama3.2-vision"

  4. Export the report with the KPIs interpretations

3) AI-generated recommendations

The prompt asks the model (llama3.2-vision) to summarize the plot interpretations and provide recommendations.

Subdividing the task in multiple simpler steps via Prompt Chaining helps improve model responses.

Refinement prompt

Recommendation: Start again the session with llama3.2-vision by using /bye and then restarting the session a second time. This cleans the context.

Suggestions and refinement
Warning: long executiontime
Plot interpretations
Read KPI reportTop: Textual dataBottom: Images
Tika Parser
Joiner
Report Template Creator
Report creation
Read plots and charts
Parse KPIs definitions
Report PDF Writer

Nodes

Extensions

Links