Evaluation Metrics

Creating a fundamental LLM application may be not too complicated, the challenge lies in its ongoing maintenance and continuous enhancement. This features was created to provide an overview of performance in whole LLM build-in system.

Based on reliable sources, valuable research topics, etc., this approach suggests crucial metrics over time, providing effective insights in order to evaluate system performance.

In general, there are two main steps that an evaluator needs to take to assess whether the use of AI is feasible.

Step 1 - Gathering Essential Information:

Relying on the metrics used for evaluation, the key information have to collect before processing next step.

As a best practice, all information used in the flow should be collected and stored.

Step 2 - Statistical Evaluation:

Each metric needs a different group metadata, depending on how much metadata we have, the corresponding metrics will be generated.