KNIME Hub LLM Selector

Options

Model

Select the ID of the LLM to use. If set via flow variable, the ID can be obtained from the KNIME Hub AI Model Lister node.

Model Parameters

Maximum response length (token)

The maximum number of tokens to generate.

This value, plus the token count of your prompt, cannot exceed the model's context length.

Temperature

Sampling temperature to use, between 0.0 and 2.0.

Higher values will lead to less deterministic answers.

Try 0.9 for more creative applications, and 0 for ones with a well-defined answer. It is generally recommended altering this, or Top-p, but not both.

Number of concurrent requests

Maximum number of concurrent requests a single node (e.g. LLM Prompter (Table)) can make to the GenAI Gateway. The more requests a node can make in parallel, the faster it executes. Too many requests might get rate-limited by some GenAI providers.

Top-p sampling

An alternative to sampling with temperature, where the model considers the results of the tokens (words) with top_p probability mass. Hence, 0.1 means only the tokens comprising the top 10% probability mass are considered.

Views

This node has no views

Workflows

• No workflows found

Links

• No links available

Developers