Bringing Data Insights and Automation Together: The Next Big Thing in Process Mining

I was convinced that workflow management technology would become a key technology in every organization: the next big thing like database management systems or Excel. Turns out I was wrong. Because I’ve overlooked one small, but important, factor.

People.

People, no matter which industry, company, or department they work in, have a “mind of their own.” They don’t always follow the prescribed path, policy, or work instruction. They take manual workarounds, send invoices back and forth between departments, buy without a PO, pay invoices twice. It’s a nightmare.

So I’ve come to the hard realization that my fellow academics and I just had a very naive idea of how standard processes were being executed. Process steps are nonlinear, unpredictable, and sometimes even change from one employee to another. There simply wasn’t just one happy path.

All of that became clear a couple of years later, when I introduced the concept of Process Mining in the late 1990s, which led to a wave of new Process Mining tools from software companies like Celonis a decade later. For the first time in history, companies had a data-driven, objective way to make those process and compliance issues visible.

The evolution of Process Mining: Bringing data and processes together

Today, Process Mining is a well-defined field that combines model-based process analytics with data-driven technologies. Before we started our research in this field in 1999, processes and data were completely disconnected. Usually, people would do one of two things: They would look at data, but not talk about processes or the people involved in those processes.

Or, they would talk about processes, but create process models that were entirely disconnected from the data. They looked good on paper but were nowhere close to reality. What makes Process Mining technology and companies like Celonis game-changing is that they combine process modelling and data analysis. This makes for an incredibly powerful mix to solve performance and compliance problems.

One of the reasons Celonis was more successful than many others is that at a very early stage, they didn’t see Process Mining and process analytics as something only data scientists could leverage. They knew it was useful for everybody working in these processes. By lowering the threshold to use and spread process (mining) knowledge across departments, for some companies, it became one of the key technologies to drive digital transformation, make data-driven decisions, and achieve better outcomes.

In countries like Germany and the Netherlands, the prevalence of Process Mining technology is quite high. If you look across the pond, the situation is completely different. While usually being the first ones to develop and implement new technologies, US companies have been significantly slower in adoption when it comes to Process Mining. In many ways, we’re only at the starting point. Process Mining technology could and should be applied in many more situations.

So why are companies still not using Process Mining?

The two biggest bottlenecks to a wider use of Process Mining are both people and technology related. Although the Process Mining market has grown exponentially over the last years (growing at around 140-160% from 2018 to 2019 to reach US$230-250 million, according to Everest Group), we have an awareness problem. Many people still don’t know it exists. And by extension, they don’t know what’s going wrong in their processes.

The second roadblock is a problem that has plagued companies for decades: their data management.

Data quality and a fragmented system landscape are (still) the biggest issues

Their data is scattered across multiple systems, and there are hundreds of thousands of ways of executing even standard core processes like Order-to-Cash or Purchase-to-Pay. But due to their fragmented IT landscape, many don’t see this data quality problem to its full extent. All they see is that they don’t execute against their goals.

Of course, we can’t stop dealing with complicated systems and it’s unlikely that we’ll start processing fewer data points. We haven’t built thousands of tables in SAP because we wanted to overcomplicate our life. It’s a simple reality that organizations have many interconnected, interdependent processes that don’t always play well together.

Process Mining has exposed these problems. It has continually held up a mirror to companies who have failed to acknowledge for decades: These problems are really there and they need to be addressed. Only by addressing these issues can companies solve compliance and performance problems and overcome system complexity.

And this is where a promising new technology class comes in. The Execution Management System.