This node establishes a connection to a specific chat model hosted on the Hugging Face Hub. The difference to the HF Hub LLM Connector is that this node allows you to provide prompt templates which are crucial for obtaining the best output from many models that have been fine-tuned for chatting. To use this node, you need to successfully authenticate with the Hugging Face Hub using the HF Hub Authenticator node.
Provide the name of the desired chat model repository available on the Hugging Face Hub as an input.
The model name to be used, in the format <organization_name>/<model_name>
. For example, Writer/camel-5b-hf
.
You can find available models at the Hugging Face Models repository.
Task type for a given model.
Please ensure that the model capabilities align with the chosen task.
Available options:
text-generation: A popular variant of Text Generation where the model predicts the next word given a sequence of words. GPT-based models, such as GPT-3, are commonly used for this task. Available text generation models
text2text-generation: Task used for mapping between pairs of texts, such as translation from one language to another. Available text-to-text generation models
Model specific system prompt template. Defaults to "%1". Refer to the Hugging Face Hub model card for information on the correct prompt template.
Model specific prompt template. Defaults to "%1". Refer to the Hugging Face Hub model card for information on the correct prompt template.
The number of top-k tokens to consider when generating text.
The typical probability threshold for generating text.
The repetition penalty to use when generating text.
The maximum number of tokens to generate in the completion.
The token count of your prompt plus max new tokens cannot exceed the model's context length.
Sampling temperature to use, between 0.0 and 100.0. Higher values will make the output more random, while lower values will make it more focused and deterministic.
An alternative to sampling with temperature, where the model considers the results of the tokens (words) with top_p probability mass. Hence, 0.1 means only the tokens comprising the top 10% probability mass are considered.
You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.
To use this node in KNIME, install the extension KNIME AI Extension (Labs) from the below update site following our NodePit Product and Node Installation Guide:
A zipped version of the software site can be downloaded here.
Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.
Try NodePit Runner!Do you have feedback, questions, comments about NodePit, want to support this platform, or want your own nodes or workflows listed here as well? Do you think, the search results could be improved or something is missing? Then please get in touch! Alternatively, you can send us an email to mail@nodepit.com, follow @NodePit on Twitter or botsin.space/@nodepit on Mastodon.
Please note that this is only about NodePit. We do not provide general support for KNIME — please use the KNIME forums instead.