Icon

Evaluation of Text Generation Workflows with Giskard

<p></p><p>This workflow uses the <strong>Giskard LLM Scanner </strong>node to scan a text generation workflow for common vulnerabilities. Giskard’s LLM scan is designed to conduct in-depth assessments of domain-specific models.</p><p>You can download and run the workflow directly in your KNIME Analytics Platform. For optimal performance, we recommend using the latest version of KNIME AP.</p><p>In order to run the workflow you need an OpenAI API key. If you don't have one yet, register with OpenAI and create a new API key at https://platform.openai.com/account/api-keys</p><p>Please keep in mind that OpenAI will deprecate models on a regular basis. For more information about available models visit: https://platform.openai.com/docs/models For information about the cost of using OpenAI models visit https://openai.com/pricing</p>

URL: LLM Vulnerabilities https://docs.giskard.ai/en/stable/knowledge/llm_vulnerabilities/index.html
URL: Giskard LLM Scan https://docs.giskard.ai/en/stable/knowledge/llm_vulnerabilities/index.html

Evaluate LLM Models

  • This workflow shows how to use Giskard to detect potential vulnerabilities in LLM models

  • In order to run the workflow you need an OpenAI API key. If you don't have one yet, register with OpenAI and create a new API key under https://platform.openai.com/account/api-keys

  • To learn more about the workflow, click the left bar and check the description section.

Workflow segment to be evaluated

  • The segment could contain more nodes, e.g. chained LLM prompters or a RAG agent

Capture Workflow End
Authenticate
OpenAI Authenticator
Capture Workflow Start
OpenAIAPI Key
Credentials Configuration
Dataset that is used byGiskard to generate domain-specific probes
Table Creator
Choose a modelfor generationand configure it
OpenAI Chat Model Connector
Create report onpotential vulnerabilitiesof the LLM model
Giskard LLM Scanner
Generates emails based on the product informationin the prompt
LLM Prompter
Choose a modelfor evaluationand configure it
OpenAI Chat Model Connector
Promptengineering
Expression

Nodes

Extensions

Links