Taking Process Mining to the Next Level

How Did It Start?

I started to work on process mining in the late 1990s, having observed that hand-made process models did not capture reality well. As a result, most workflow-management projects failed.

The two-year sabbatical I took working at the University of Georgia and the University of Colorado in 1998 gave me time to think about the problem of discovering accurate process models from example executions. It was there that I started to develop the first algorithms to learn Petri nets from event logs and wrote the project proposal "Process Design by Discovery" that was later funded by the research school BETA. My proposal defined process mining as the "method to distill process models from real process executions" and can be seen as the start of the process mining field.

We developed an open-source platform, ProM, that allowed us to combine many different processes in a short period because it was immediately apparent to me that the scope of process mining was much broader. These included techniques we developed for conformance checking, performance analysis, decision & organizational mining, and process prediction.

However, I deliberately did not financially participate in these process-mining startups and remained independent until now. To explain why I'm joining Celonis, I first need to describe the development of the discipline and the changing role of process mining software.

What Changed?

Over the last two decades, process mining matured as a field. Now there are research groups all over the globe working on process mining, and there are dedicated conferences such as the International Conference on Process Mining (ICPM).

The uptake of process mining transformed Business Process Management (BPM) research completely. It is no longer acceptable to waste time and money on creating BPMN models that say little about the actual processes. Moreover, there are dozens of commercial process mining tools available, and many of the largest organizations have adopted process mining.

Where Celonis Fits In

Initially, process-mining tools focused on process analysts with a data science mindset. Process mining was done in the context of improvement projects and was not a continuous activity. Celonis was the first process mining vendor that enabled non-experts to use process mining results continuously. The vision that process mining should be used by many people, every day, helped Celonis to become the undisputed market leader.

Many of the process mining problems I started to work on over 20 years ago (process discovery, conformance checking, and process prediction, for instance) are still being researched. These problems are not fully solved and one can still witness significant improvements each year. However, as the field matures, the focus is shifting (1) from insights to actions and (2) from structured event logs and processes to complex entangled data sources and processes.

Process-mining research, then, is also shifting from the offline analysis of event logs in university labs ("in vitro") to applications and experiments inside organizations ("in vivo"). The latter form of research requires a fully deployed process mining pipeline and production-strength process mining software. This necessitates a closer collaboration between research groups at universities, software providers, and end-users.

It is unrealistic to assume that researchers can create and maintain such a pipeline using open-source software, so I'm really looking forward to intensifying the collaboration between my PADS research group at RWTH Aachen University and Celonis.

What's Next in Process Mining?

As Niels Bohr said, "it is difficult to predict, especially the future”. Nevertheless, there are a few clear developments.

From backward-looking to forward-looking process mining. The initial focus of process mining was on analyzing historical event data to detect and diagnose performance and compliance problems. This is extremely valuable and helps capture the low-hanging fruit (i.e. obvious and easy to achieve process improvements). However, new performance and compliance problems may emerge unexpectedly, and it may not be so obvious what actions to take. Therefore, forward-looking forms of process mining are needed, including predictive techniques using machine learning and data-driven simulations to answer what-if questions.

These techniques build on traditional process discovery and conformance checking techniques. Therefore, innovations in core process mining technologies will also lead to more accurate predictions and more realistic simulations. Process mining will play a key role in creating digital twins of production lines, airports, supply chains, hospitals, and other organizations.

From insights to actions. The ultimate goal is to fundamentally improve operational processes, not just provide sophisticated analysis results, so process mining diagnostics (e.g., performance or compliance problems) need to be actionable. This requires forward-looking forms of process mining to respond quickly, and the ability to trigger workflows that take corrective actions.

Having a background in workflow automation, I'm excited to see the fields of process mining and workflow automation converge. Process mining helps to identify Robotic Process Automation (RPA) opportunities and can support workflows and apps using low-code, visual integration platforms.

From isolated processes to collections of processes. To do process mining, one needs to have a collection of events where each event has at least three attributes: case identifier, activity name, and timestamp. Therefore, process-mining opportunities are everywhere. That said, it may be very time-consuming to extract such data from dozens of database tables in multiple systems; moreover, different case notions (e.g., order, item, delivery, payment, and customer) may be entangled.

This means that process mining technology needs to move closer to the true fabric of processes and systems. Object-centric process mining connects different case notions in a holistic manner, helping to accelerate data extraction and enable a view of interconnected processes from different angles.

The scope of process mining will expand further both in terms of technologies used (e.g. the connection to automation, machine learning, simulation, and optimization) and novel applications. Initially, process mining was predominantly applied to standard processes like Procure-to-Pay (P2P) and Order-to-Cash (O2C). However, process mining can and must also be used to improve primary processes in production, materials handling, distribution, healthcare, education, and service delivery.

Why Celonis?

Celonis is the undisputed market leader in process mining and is well-positioned to take advantage of the three developments I mentioned. The Celonis Execution Management System (EMS) platform supports forward-looking forms of process mining such as prediction and simulation. The Celonis EMS is able to design and trigger action flows and this capability gained a lot of strengths through the acquisition of Integromat last year, which enables process mining insights to be turned into actions. In addition, capabilities such as the multi-event log can be used to connect different processes.

Celonis has always been one of the few vendors aiming for the continuous use of process mining by many stakeholders. Now, Celonis has many customers that apply process mining at an unprecedented scale, both in terms of data (handling billions of events) and numbers of users (up to thousands of active users).

As mentioned before, process-mining research will increasingly depend on applications and experiments inside organizations ("in vivo process mining"). Therefore, Celonis is a logical choice, and the time is right for it.

Having served as Chief Scientific Advisor for many years, I already know many great Celonauts. I'm deeply impressed by the amazing growth, technical innovations, positive corporate spirit and the three founders: Alex Rinke, Bastian Nominacher, and Martin Klenk. I believe that their complementary skills are the secret to Celonis's success. Alex studied mathematics and is great at connecting the dots and clearly communicating the strategy. Basti has a strong business and IT-management background, and is eager to govern Germany's fastest-growing IT company. Martin is a genuine computer scientist interested in building and architecting great software.

This combination of skills and characters helped to create an amazing success story. I'm glad to be part of it.