Comparing Trace Similarity Metrics Across Logs and Evaluation Measures
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Trace similarity is a prerequisite for several process mining tasks, e.g. identifying process variants and anomalies. Many similarity metrics have been presented in the literature, but the similarity metric itself is seldom subject to controlled evaluation. Instead, they are usually demonstrated in conjunction with downstream tasks, e.g. process model discovery, and often evaluated qualitatively or with limited comparison. In this paper, we isolate similarity metrics from downstream tasks and compare them wrt. evaluation measures adapted from metric learning and clustering literature. We present a comparison of 18 similarity metrics across 4 evaluation measures and 12 event logs. Friedman and Nemenyi tests for statistical significance show that certain similarity metrics consistently outperform on some evaluation measures, but their mean rank varies across evaluation measures. One similarity metric based on a weighted eventually-follows relation does stand out as consistently outperforming, and the simplest n-gram similarity metrics also perform well. Our results demonstrate that choice of evaluation measures will determine the contours of the metric that are revealed. This study may be harnessed as a baseline for benchmarking future work on trace similarity, and describes tools for quantitative evaluation that we hope will inspire empirical rigor in future work.
Original language | English |
---|---|
Title of host publication | Advanced Information Systems Engineering : 35th International Conference, CAiSE 2023, Zaragoza, Spain, June 12–16, 2023, Proceedings |
Editors | Marta Indulska, Iris Reinhartz-Berger, Carlos Cetina, Oscar Pastor |
Number of pages | 17 |
Publisher | Springer |
Publication date | 2023 |
Pages | 226-242 |
ISBN (Print) | 978-3-031-34559-3 |
ISBN (Electronic) | 978-3-031-34560-9 |
DOIs | |
Publication status | Published - 2023 |
Event | 35th International Conference on Advanced Information Systems Engineering, CAiSE 2023 - Zaragoza, Spain Duration: 12 Jun 2023 → 16 Jun 2023 |
Conference
Conference | 35th International Conference on Advanced Information Systems Engineering, CAiSE 2023 |
---|---|
Land | Spain |
By | Zaragoza |
Periode | 12/06/2023 → 16/06/2023 |
Series | Lecture Notes in Computer Science |
---|---|
Volume | 13901) |
ISSN | 0302-9743 |
Bibliographical note
Supported by Innovation Fund Denmark as part of DIREC initiative
- Empirical Evaluation, Process Mining, Similarity Metric
Research areas
ID: 387382847