Create Eval Run

Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation.

Options

Eval Id

The ID of the evaluation to create a run for.

Body

Result Format

Specify how the response should be mapped to the table output. The following formats are available:

Structured Table: Returns a parsed table with data split into rows and columns.

• Object: The type of the object. Always "eval.run".
• Id: Unique identifier for the evaluation run.
• Eval Id: The identifier of the associated evaluation.
• Status: The status of the evaluation run.
• Model: The model that is evaluated, if applicable.
• Name: The name of the evaluation run.
• Created At: Unix timestamp (in seconds) when the evaluation run was created.
• Report Url: The URL to the rendered evaluation run report on the UI dashboard.
• Result Counts: Counters summarizing the outcomes of the evaluation run.
• Per Model Usage: Usage statistics for each model during the evaluation run.
• Per Testing Criteria Results: Results per testing criteria applied during the evaluation run.
• Data Source: Information about the run's data source.
• Metadata:

  Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

  Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

• Error: An object representing an error response from the Eval API.

Raw Response: Returns the raw response in a single row with the following columns:

• body: Response body
• status: HTTP status code

Views

This node has no views

Workflows

• No workflows found

Links

• No links available

Developers