UniClean: A Multi-Signal Fusion Pipeline for Optimizing Data Cleaning Workflow

Data quality issues are prevalent in information systems, making data cleaning a complex and time-consuming task, particularly with large-scale datasets and the lack of standardized automated processes. Existing cleaning pipelines often lack automated schemes to guide the execution of cleaning algorithms and the sequence of error corrections, limiting their practicality in real-world big data applications. To address the growing demand for advanced cleaning tools driven by the complexity of data activities, we propose the UniClean framework for on-demand big data cleaning. UniClean employs a unified cleaning operation (Uniop) from multiple cleaners to optimize the data cleaning workflow. It integrates a cleaning parameter generation pipeline, a cleaning parameter selection pipeline, and a module for cleaning process preparation and optimization, covering the entire workflow from cleaner modeling and data preparation to optimal cleaning operation generation. UniClean provides an adaptive (data-driven cleaning workflow generation) and flexible (multi-signal extension system) solution to meet the urgent need for high-quality data in today's data-driven decision-making environments. We demonstrate how UniClean effectively addresses the challenges of big data cleaning across diverse information system landscapes.