New algorithm looks to help streamline data restoration of event logs

New tertiary research out of Korea has led to the development of an algorithm capable of streamlining the restoration of event data logs.

According to the researchers from Pusan National University, they have developed a data restoration algorithm that uses correlations between existing information to restore missing data in an event log with a high degree of accuracy.

They see the new technology as not only able to be utilised in current enterprises but also future AI applications.

Missing data can lead to a myriad of problems for multiple parties involved, and the increase in digitisation has increased the risk of failures in log keeping and development.

The algorithm has been designed to collect data from multiple perspectives in numerous information systems. Therefore there is a relationship between the collected data.

In most event logs, the researchers say that events have attributes linked to other events in "single event" or "multiple event relationships.

In the former case, each attribute of an event corresponds to a unique attribute in another event. Based on this relationship, the researchers developed a Systematic Event Imputation (SEI) method that restores a missing value by simply referring to the available value it is linked to.

In cases where attributes have multiple correspondences, however, matching of attributes is not possible. To help remedy this situation, researchers developed a multiple event imputation (MEI) method where missing events are first estimated and used to create event sequences or event chains.

These sequences can then be compared with an event log without missing data to restore the missing event attributes.

When conducting tests with real-world event logs, the researchers found that their algorithm improved restoration accuracy by 1030% compared to existing restoration algorithms. In addition, it could restore almost 90% of the data accuracy even when more than half of it was missing.

Dr. Sunghyun Sim and Prof. Hyerim Bae, along with Prof. Ling Liu from Georgia Institute of Technology are spearheading this new research. They say it will significantly improve working processes and solve many business challenges.

"Since data is collected from multiple perspectives in numerous information systems, there is a relationship between the collected data. Starting with this point, our study suggested a method of restoring missing event values by utilising the relationship among entities in the event log, which can overcome human error or system," explains Sim.

Hyerim elaborates that it can further enhance AI aspects, saying, "It is possible to improve the performance of artificial intelligence by improving the quality of data in its learning process. The algorithm will also help prevent model malfunction by improving the quality of data it collects in real-time in a real-time environment."

The title of the original paper published is "Bagging recurrent event imputation for repair of imperfect event log with missing categorical events." It is published in the online journal IEEE transactions on services computing.