Paper Image

Evaluating AI Assistants for Business Process Analysis

Paper Authors:

Alessandro Berti,

Humam Kourani,

Hannes Hafke,

Chiao-Yun Li,

Daniel Schuster


Key Details

Reviews how large language models are currently used in process mining across areas like anomaly detection and root cause analysis

Proposes key capabilities needed for process mining tasks: long context, visuals, coding, factuality

Introduces general and process mining specific benchmarks to evaluate language models on relevant skills

Suggests automatic, human, and self-assessment methods to measure quality of model outputs and mitigate issues like hallucination

AI generated summary

Evaluating AI Assistants for Business Process Analysis

This paper discusses strategies to evaluate the capabilities and benchmark the performance of large language models when applied to process mining tasks like discovering process models from event logs, detecting anomalies, ensuring fairness, and enhancing processes. It proposes required abilities like reasoning, coding, visual and factual understanding, and measuring quality across tasks through precision, recall and correctness. The goal is to develop comprehensive benchmarks tailored to process mining that build confidence in AI assistant outputs.

Answers from this paper


No comments yet, be the first to start the conversation...

Sign up to comment on this paper

Sign Up