Bringing Data Insights and Automation Together: The Next Big Thing in Process Mining

Let me start with a short excursion to one of my biggest “failures” in research. Back in the early 90s, I wrote the first textbook on workflow management systems. The idea was that those software tools, in theory, would help streamline routine business processes for optimal efficiency. Every process, once defined, would always happen the same, standardized way.

I was convinced that workflow management technology would become a key technology in every organization: the next big thing like database management systems or Excel. Turns out I was wrong. Because I’ve overlooked one small, but important, factor.

People.

People, no matter which industry, company, or department they work in, have a “mind of their own.” They don’t always follow the prescribed path, policy, or work instruction. They take manual workarounds, send invoices back and forth between departments, buy without a PO, pay invoices twice. It’s a nightmare.

So I’ve come to the hard realization that my fellow academics and I just had a very naive idea of how standard processes were being executed. Process steps are nonlinear, unpredictable, and sometimes even change from one employee to another. There simply wasn’t just one happy path.

All of that became clear a couple of years later, when I introduced the concept of Process Mining in the late 1990s, which led to a wave of new Process Mining tools from software companies like Celonis a decade later. For the first time in history, companies had a data-driven, objective way to make those process and compliance issues visible.

The evolution of Process Mining: Bringing data and processes together

Today, Process Mining is a well-defined field that combines model-based process analytics with data-driven technologies. Before we started our research in this field in 1999, processes and data were completely disconnected. Usually, people would do one of two things: They would look at data, but not talk about processes or the people involved in those processes.

Or, they would talk about processes, but create process models that were entirely disconnected from the data. They looked good on paper but were nowhere close to reality. What makes Process Mining technology and companies like Celonis game-changing is that they combine process modelling and data analysis. This makes for an incredibly powerful mix to solve performance and compliance problems.

One of the reasons Celonis was more successful than many others is that at a very early stage, they didn’t see Process Mining and process analytics as something only data scientists could leverage. They knew it was useful for everybody working in these processes. By lowering the threshold to use and spread process (mining) knowledge across departments, for some companies, it became one of the key technologies to drive digital transformation, make data-driven decisions, and achieve better outcomes.

In countries like Germany and the Netherlands, the prevalence of Process Mining technology is quite high. If you look across the pond, the situation is completely different. While usually being the first ones to develop and implement new technologies, US companies have been significantly slower in adoption when it comes to Process Mining. In many ways, we’re only at the starting point. Process Mining technology could and should be applied in many more situations.

So why are companies still not using Process Mining?

The two biggest bottlenecks to a wider use of Process Mining are both people and technology related. Although the Process Mining market has grown exponentially over the last years (growing at around 140-160% from 2018 to 2019 to reach US$230-250 million, according to Everest Group), we have an awareness problem. Many people still don’t know it exists. And by extension, they don’t know what’s going wrong in their processes.

If people see all their process variants with Process Mining, and know what’s actually going on, it feels like they are seeing snow for the first time. They are completely amazed. 

The second roadblock is a problem that has plagued companies for decades: their data management.

Data quality and a fragmented system landscape are (still) the biggest issues

Their data is scattered across multiple systems, and there are hundreds of thousands of ways of executing even standard core processes like Order-to-Cash or Purchase-to-Pay. But due to their fragmented IT landscape, many don’t see this data quality problem to its full extent. All they see is that they don’t execute against their goals.

Of course, we can’t stop dealing with complicated systems and it’s unlikely that we’ll start processing fewer data points. We haven’t built thousands of tables in SAP because we wanted to overcomplicate our life. It’s a simple reality that organizations have many interconnected, interdependent processes that don’t always play well together.

Process Mining has exposed these problems. It has continually held up a mirror to companies who have failed to acknowledge for decades: These problems are really there and they need to be addressed. Only by addressing these issues can companies solve compliance and performance problems and overcome system complexity.

And this is where a promising new technology class comes in. The Execution Management System.

Where Process Mining is going next: Bringing Data Insights and Automation together

The recent move of Celonis to introduce an Execution Management System is a logical continuation of what we call “action-oriented Process Mining” in the academic world. It’s a system that acts as an intelligent layer on top of all transactional systems (like SAP, Oracle, Salesforce, or Workday) and desktop applications (like Microsoft, Excel, GSuite, Outlook) to not only make sense of the data but do something about it. So while Process Mining historically focused on process analytics, visualizing the process and all its variations, and identifying inefficiencies, an Execution Management System goes several steps further.

It takes action to close these process problems — Celonis calls them “execution gaps” — by sending alerts and assigning tasks to employees, so they can react in a timely manner.

And it automates these actions, through built-in automation tools or by triggering external ones (such as RPA bots) to make the necessary updates in the relevant transactional systems (ERP, CRM, SCM, HRM/HCM, etc.).

The release of the new Execution Management System and the development of the market shows a powerful convergence of Process Mining and process automation. A convergence that could indeed be the “next big thing”. Time will show if and when companies will adapt.

But one thing is for sure: It is not enough to simply see the problems. It’s time to act on them.

Read the second part of this series: The Self-Driving Enterprise: To Bring AI to Your Processes, Start With the EMS

Wil van der Aalst

Distinguished Humboldt Professor at RWTH Aachen University and Fraunhofer-Institut für Angewandte Informationstechnik

Prof.dr.ir. Wil van der Aalst is the Chief Scientist of Celonis and a full professor at RWTH Aachen University leading the Process and Data Science (PADS) group. He is also part-time affiliated with the Fraunhofer-Institut für Angewandte Informationstechnik (FIT) where he leads FIT’s Process Mining group and the Technische Universiteit Eindhoven (TU/e). He coined the field of Process Mining and his research interests include Process Mining, Petri Nets, business process management, workflow management, process modeling, and process analysis.