LLM Chat Prompter

This node prompts a chat model with the provided user message.

An optional table containing a Message column representing the conversation history can be provided. Existing messages will be used as context, and new messages generated by the model are appended to the conversation by default.

An optional table containing a JSON column with tool definitions can be provided to enable tool calling.

Conversation history:

The conversation history table will be used as context when sending the new message to the chat model. To use only the conversation history table for prompting (without a new message), leave the new message setting empty and ensure that the last entry in the table is from either User or Tool.

Tool use:

In order to enable tool calling, a table containing tool definitions must be connected to the corresponding optional input port of the node.

If the model decides to call a tool, the node appends a new 'AI' message containing said tool call. Information like Tool Call ID and Tool Call Arguments, normally necessary for processing a tool call, can be extracted from the message using the Message Part Extractor node. This can then be used to route the downstream portion of the workflow appropriately.

The output of the tool should then be turned into a "Tool" message, appended to the conversation, and fed back into the node. It is crucial to ensure that this "Tool" message has the same Tool Call ID as the request it is responding to, which allows the model to link the two.

During the next node execution, the model will use the messages in the conversation as context, including the original "User" request, the "AI" response containing the tool call, and the corresponding "Tool" message, to generate the final "AI" message, thus completing the tool-calling loop.

A tool definition is a JSON object describing the corresponding tool and its parameters. The more descriptive the definition, the more likely the LLM to call it appropriately.

Example:

{
    "title": "number_adder",
    "type": "object",
    "description": "Adds two numbers.",
    "properties": {
        "a": {
            "title": "A",
            "type": "integer",
            "description": "First value to add"
        },
        "b": {
            "title": "B",
            "type": "integer",
            "description": "Second value to add"
        }
    },
    "required": ["a", "b"]
}