Alexandra Wins

Rybinski, Fabian

Large Language Models (LLMs) are increasingly being used to analyze, document, and interpret existing process models. These tasks require that the LLM accurately capture the specific model at hand. However, due to their training, LLMs possess extensive implicit knowledge about how typical business processes—such as ordering, approval, or invoicing processes—usually unfold. A well-known phenomenon in LLM-supported process model generation is “knowledge-driven hallucination”: LLMs supplement or alter process content based on their pre-trained domain knowledge rather than strictly adhering to the given input.

This phenomenon has hardly been studied in the context of analyzing existing process models, yet it poses significant risks: If a real-world process model deviates from the “typical” schema—for example, due to company-specific sequences, intentionally omitted verification steps, or unusual responsibilities—an LLM could “correct” these deviations unnoticed, overlook them, or declare them as errors. The analysis results would then not reflect the actual model, but rather a mixture of model content and prior knowledge. It is unclear under what conditions LLMs fall back on pre-trained process schemas when interpreting models, how this effect can be reliably measured, and what countermeasures (e.g., prompting strategies, forms of representation) can reduce it.

This thesis therefore aims to investigate whether and to what extent knowledge-driven hallucination occurs in LLM-based analysis of existing process models. The target is to determine, through systematic experiments with specifically manipulated process models—such as controlled deviations from domain-typical workflows—when LLMs interpret the given model faithfully and when they rely on their prior knowledge.

The thesis will cover the following steps:

Literature Review: First, an initial literature review will be conducted to identify existing approaches in the literature on hallucinations in LLMs—particularly knowledge-driven hallucinations in process modeling—as well as methods for measuring and detecting hallucinations. This will also involve identifying suitable process model datasets.

Analysis: The study will analyze which types of model deviations (e.g., atypical activity sequences, missing standard steps, reversed responsibilities) are particularly prone to knowledge-driven misinterpretations. Building on this, a classification system will be developed to categorize types of hallucinations in model interpretation and distinguish them from other types of errors (e.g., purely syntactic reading errors).

Design: Based on the analysis, an experimental framework will be developed that systematically induces and measures knowledge-driven hallucinations. In this process, suitable metrics for quantifying interpretation fidelity will be developed.

Implementation & Evaluation: Finally, a pipeline will be implemented that enables the experiments to be conducted with various configurations. The results are expected to reveal the conditions under which LLMs interpret process models faithfully and to provide practical recommendations on how to reduce knowledge-driven misinterpretations in LLM-based BPM tools.

Please apply via the following link: https://portal.wiwi.kit.edu/forms/form/Bewerbung_Abschlussarbeit_AIFB-BIS

Rybinski, Fabian

Large Language Models (LLMs) are increasingly being used in Business Process Management (BPM) to create, analyze, and interpret process models. Current research shows that the form of representation of a process model—such as a textual description, a rendered diagram, or structured serialization—has a significant impact on the quality of interpretation by LLMs. However, these studies keep the underlying modeling language constant and vary only the representation of the same model.

In practice, however, there is a wide variety of process modeling languages—ranging from Petri nets to UML activity diagrams to BPMN. Each language has its own syntactic constructs, formal semantics, and expressive power: Petri nets offer a mathematically precise execution semantics, while BPMN provides a comprehensive, practice-oriented catalog of elements. It remains unclear whether and how the choice of modeling language influences LLMs’ understanding of processes and whether this effect depends on the type of task (e.g., structural questions, trace validation, deadlock detection).

This thesis will therefore systematically investigate how different process modeling languages, when used as input for LLMs, affect the understanding of process models. The target is to determine, through controlled experiments with substantively equivalent models in different languages, which modeling languages enable the best LLM understanding for which analysis tasks.

The thesis will cover the following steps:

Literature Review: First, an initial literature review will be conducted to identify existing approaches in the LLM-based division of process model analysis, as well as the characteristics and comparability of common process modeling languages. This will also involve researching available datasets as well as methods for semantically preserving conversions between modeling languages.

Analysis: The selected modeling languages will be examined and evaluated based on defined criteria to assess their suitability as LLM input. In particular, it must be ensured that the compared models are information-equivalent, so that any observed differences can be attributed to the language rather than the content.

Design: Based on the analysis, an experimental framework will be developed that enables the systematic evaluation of process model understanding across different modeling languages. In this process, suitable task types and metrics will be developed to operationalize different levels of understanding (syntactic, semantic, inferential).

Implementation & Evaluation: Finally, a pipeline will be implemented to enable the execution of experiments with various configurations. The results will provide insights into which modeling languages are particularly suitable for LLM-based process analysis and what implications this has for the choice of modeling language in practice.

If you are interested, please apply via the following link: https://portal.wiwi.kit.edu/forms/form/Bewerbung_Abschlussarbeit_AIFB-BIS

Kruse, Nadja

Kruse, Nadja

Alexander Dregger, Andreas Oberweis

Description

A chatbot is to be implemented based on the model of artificial personality (Dregger, 2023) using a large language model. Methods such as Personality Infusion (Kovacevic, 2024) can be used here. Users should be able to use sliders, for example, to adjust the personality of a chatbot that they subsequently use. Based on the personality setting, the language of the system should vary in order to create the impression of a customizable artificial personality (Dregger, Seifermann & Oberweis, 2024). The users then have to complete two different tasks and evaluate the personality using the questionnaire for measuring artificial personality (Dregger et al., in press). Here, the effects of the customized artificial personality on the user experience are to be examined in more detail. In this context, the similarity hypothesis can be considered, which assumes that people like to interact with social actors who are similar to them, e.g. in their personality. However, this hypothesis is also criticized and it is necessary to examine whether the expectations of a personality are also influenced by the social role independently of the similarity.

Procedure

• Review of the literature on the design and adaptation of personality in conversational agents
• Development of an approach for the adaptation of artificial personality in conversational agents
• Development of an experiment for measurement, e.g. using an LLM-based chatbot
• Conducting the experiment with users
• Evaluation of the experiment using statistical methods

What you bring with you

• You are studying computer science, business informatics, industrial engineering, economics or a comparable course of study
• You are interested in UX and the design of conversational agents
• You enjoy working independently
• Ideally, you have basic knowledge of literature research
• Ideally, you have basic knowledge in the construction of questionnaires
• Ideally, you have basic knowledge of LLM and chatbot development
• You have a very good command of written and spoken German and English
• You have basic knowledge of statistics and relevant software, e.g. R or SPSS
• You are motivated and committed

What we offer you:

• Motivated and competent support is important to us. For us, this includes taking enough time for you and supporting you with helpful feedback.
• You will gain exciting insights into interdisciplinary research topics.
• You can flexibly organize the implementation (working hours, remote)

Please send your application to dregger@fzi.de with your CV and current transcript of records.

Literature

Dregger, A. More than Big Five? Towards Modeling and Defining Artificial Personality for Conversational Agents. Conversations, Oslo, 2023

Dregger, A. (2025). Artificial Personality: Model and Questionnaire Development (unpublished manuscript).

Dregger, A., Seifermann, M., & Oberweis, A. (2024). Language Cues for Expressing Artificial Personality: A Systematic Literature Review for Conversational Agents. Proceedings of the 6th ACM Conference on Conversational User Interfaces, 1-17. https://doi.org/10.1145/3640794.3665559

Kovacevic, N., Boschung, T., Holz, C. Gross, M., & Wampfler, R. (2024). Chatbots with Attitude: Enhancing Chatbot Interactions through Dynamic Personality Infusion, Proceedings of the 6th ACM Conference on Conversational User Interfaces, 1-16. https://doi.org/10.1145/3640794.3665543

Kathrin Janowski, Hannes Ritschel, and Elisabeth André. 2022. Adaptive Artificial Personalities. In The Handbook on Socially Interactive Agents (1st ed.), Birgit Lugrin, Catherine Pelachaud and David Traum (eds.). ACM, New York, NY, USA, 155-194. https://doi.org/10.1145/3563659.3563666

Jiang, G., Xu, M., Zhu, S.-C., Han, W., Zhang, C., & Zhu, Y. (2023). Evaluating and Inducing Personality in Pre-trained Language Models. https://doi.org/10.48550/ arXiv.2206.07550

Sebastian Schneider and Franz Kummert. 2021. comparing Robot and Human guided Personalization: Adaptive Exercise Robots are Perceived as more Competent and Trustworthy. Int J of Soc Robotics 13, 2 (April 2021), 169-185. https://doi.org/10.1007/s12369-020-00629-w

Völkel, S. T. & Kaya, L. (2021). Examining User Preference for Agreeableness in Chatbots. In CUI 2021 - 3rd Conference on Conversational User Interfaces (pp. 1-6). ACM. https://doi.org/10.1145/3469595.3469633

Völkel, S. T., Schödel, R., Buschek, D., Stachl, C., Winterhalter, V., Bühner, M., & Hussmann, H. (2020). Developing a Personality Model for Speech-based Conversational Agents Using the Psycholexical Approach. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1-14. https://doi.org/10.1145/3313831.3376210

Kruse, Nadja; Ullrich, Meike

Folgende drei Themen stehen für die externe Abschlussarbeit zur Auswahl:

THEMA 1: Entwicklung und Anwendung einer intuitiven Root Cause Analysis für nicht-technische Nutzer im Kontext von Process Mining

Root Cause Analysis (RCA) ist ein zentraler Bestandteil moderner Process-Mining-Technologien und ermöglicht es, auf prozessuale Fragestellungen wie "Warum tritt ein Problem auf?" konkrete Antworten und Ansatzpunkte zur Verbesserung oder Vermeidung zu liefern. Aktuelle Ansätze konzentrieren sich jedoch häufig auf technisch versierte Anwender und setzen ein hohes Maß an Fachwissen über Dashboarding, Datenanalyse sowie Algorithmen voraus oder sind so einfach gehalten, dass die Ergebnisse nur begrenzte Relevanz zur Problemlösung haben. Ziel dieser Forschungsarbeit ist es daher, die Essenz des Process Minings für nichttechnische Fachanwender intuitiv und werthaltig zugänglich zu machen. Im Rahmen dieser Masterarbeit soll untersucht werden, wie eine anwenderfreundliche Root Cause Analysis gestaltet werden kann.

THEMA 2: Entwicklung und Anwendung von Semantic Conformance Checking in Audit- und Compliance-Szenarien

Conformance Checking wird derzeit primär zur Überprüfung des Kontrollflusses eines Prozesses eingesetzt, indem Soll-Prozessvarianten mit Ist-Prozessvarianten verglichen werden. Diese rein syntaktische Betrachtung
beschränkt sich jedoch auf die Struktur der Prozessabläufe und lässt inhaltliche Aspekte wie Kosten, Zeiten oder Ressourcenzuweisungen unberücksichtigt. Das Konzept des Semantic Conformance Checking erweitert diese Methodik um eine ganzheitliche Soll-Ist-Analyse, die zusätzliche Dimensionen wie Sollund Ist-Kosten, Soll- und Ist-Zeiten sowie Ressourcenzuweisungen einbezieht. Im Rahmen dieser Masterarbeit soll untersucht werden, wie Semantic Conformance Checking technisch und methodisch umgesetzt werden kann.

THEMA 3: Optimized Data Models for Object Centric Process Mining

Object-Centric Process Mining gewinnt zunehmend an Bedeutung, da es eine differenzierte und umfassende Analyse komplexer Prozesse ermöglicht. Um OCPM als neuen Standard in kommerziellen Process Mining Werkzeugen zu implementieren, werden alle Software Vendoren die traditionellen Process Mining Datenmodelle bestehend aus Event-Log und Case-Tabelle auf Objekt-zentrierte Sichten anpassen müssen. Als führende Process-Mining-Plattform haben wir das Ziel, Branchenstandards aktiv zu fördern und die Nutzung von OCPM bei unseren Kunden zu etablieren. Bisher nutzen wir dafür ein sehr stark am traditionellen Log-Case-Datenmodell
angelehntes OCPM-Datenmodell. Im Rahmen dieser Masterarbeit soll untersucht werden, wie ein OCPMDatenmodell aufgebaut sein sollte um (1.) die technischen Möglichkeiten der Process-Mining-Lösung optimal hinsichtlich Performance und Filtermöglichkeiten zu nutzen, (2.) die Erstellung eines solchen Datenmodells für self-service Process Mining maximal einfach zu machen und (3.) die Nutzung und Wiederverwendung des Datenmodells für self-service Analytics und AI-Initiativen transparent und einfach zu gestalten.