This node establishes a connection with an Azure OpenAI Chat Model. After successfully authenticating using the Azure OpenAI Authenticator node, enter the deployment name of the model you want to use. You can find the models on the Azure AI Studio at 'Management - Deployments'.
Note that chat models can also be used as LLMs because they are actually a subcategory of LLMs that are optimized for chat-like applications.
The name of the deployed model to use. Find the deployed models on the Azure AI Studio.
The maximum number of tokens to generate.
The token count of your prompt plus max_tokens cannot exceed the model's context length.
Sampling temperature to use, between 0.0 and 2.0. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 for ones with a well-defined answer. It is generally recommend altering this or top_p but not both.
How many chat completion choices to generate for each input message. This parameter generates many completions and can quickly consume your token quota.
Set the seed parameter to any integer of your choice to have (mostly) deterministic outputs. The default value of 0 means that no seed is specified.
If the seed and other model parameters are the same for each request, then responses will be mostly identical. There is a chance that responses will differ, due to the inherent non-determinism of OpenAI models.
Please note that this feature is in beta and only currently supported for gpt-4-1106-preview and gpt-3.5-turbo-1106 [1].
[1] OpenAI Cookbook
Maximum number of concurrent requests to LLMs that can be made, whether through API calls or to an inference server. Exceeding this limit may result in temporary restrictions on your access.
It is important to plan your usage according to the model provider's rate limits, and keep in mind that both software and hardware constraints can impact performance.
For OpenAI, please refer to the Limits page for the rate limits available to you.
An alternative to sampling with temperature, where the model considers the results of the tokens (words) with top_p probability mass. Hence, 0.1 means only the tokens comprising the top 10% probability mass are considered.
You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.
To use this node in KNIME, install the extension KNIME Python Extension Development (Labs) from the below update site following our NodePit Product and Node Installation Guide:
A zipped version of the software site can be downloaded here.
Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.
Try NodePit Runner!Do you have feedback, questions, comments about NodePit, want to support this platform, or want your own nodes or workflows listed here as well? Do you think, the search results could be improved or something is missing? Then please get in touch! Alternatively, you can send us an email to mail@nodepit.com, follow @NodePit on Twitter or botsin.space/@nodepit on Mastodon.
Please note that this is only about NodePit. We do not provide general support for KNIME — please use the KNIME forums instead.