Icon

Detect LLM vulnerabilities

<p><strong>Detect LLM vulnerabilities in a medical diagnoser</strong></p><ul><li><p>This workflow relies on an LLM to provide medical diagnoses to patients based on a set of symptoms.</p><p></p></li><li><p>The generative workflow segment is then scanned for risks and vulnerabilities using Giskard.</p><p></p></li><li><p>An interactive report of identified issues is output for users to take action.</p></li></ul>

URL: KNIME for Generative AI https://hub.knime.com/c/D4ckx2q_J5FPBQXu
URL: Giskard LLM Scanner https://docs.giskard.ai/en/stable/knowledge/llm_vulnerabilities/index.html#how-does-the-llm-scan-work

Detect LLM vulnerabilities in a medical diagnoser

  • This workflow relies on an LLM to provide medical diagnoses to patients based on a set of symptoms.

  • The generative workflow segment is then scanned for risks and vulnerabilities using Giskard.

  • An interactive report of identified issues is output for users to take action.

Model selection

Capture workflow segment to scan

Data Access

Scan LLM vulnerabilities

Address vulnerabilities

  • Selection of more capable and sophisticated LLM

  • Improved prompt to explicitly address vulnerabilities



Get diagnosis
LLM Prompter
Model: gpt-3.5-turbo
OpenAI Chat Model Connector
Capture Workflow Start
Patientsand symptoms
Table Reader
Capture Workflow End
Capture Workflow Start
OpenAI Authenticator
OpenAIAPI key
Credentials Configuration
Model: gpt-4
OpenAI Chat Model Connector
OpenAIAPI key
Credentials Configuration
OpenAI Authenticator
Model: gpt-4
OpenAI Chat Model Connector
Capture Workflow End
Patientsand symptoms
Table Reader
Scan the LLM forrisks and vulnerabilitiesMind: this is not deterministicand results may vary at each execution
Giskard LLM Scanner
Get diagnosis
LLM Prompter
Model: gpt-4
OpenAI Chat Model Connector
Double-click to open
Improved prompt
Create prompt to requestdiagnosis of diseasesbased on symptoms
Expression
Create prompt to requestdiagnosis of diseasesbased on symptoms
Expression
Tabular view
View diagnosis
Tabular view
View diagnosis
Scan the LLM forrisks and vulnerabilitiesMind: this is not deterministicand results may vary at each execution
Giskard LLM Scanner

Nodes

Extensions

Links