Revisiting data analysis with Pre-trained foundation models: Pre-trained foundation model-enhanced scalability

Data analysis focuses on harnessing advanced statistics, programming, and machine learning techniques to extract valuable insights from vast datasets. An increasing volume and variety of research emerged, addressing datasets of diverse modalities, formats, scales, and resolutions across various industries. However, experienced data analysts often find themselves overwhelmed by intricate details in ad-hoc solutions or attempts to extract the semantics of real-world data properly. This makes the infrastructure difficult to maintain system performance and scale to more complex analytical scenarios. Pre-trained foundation models (PFMs), grounded (reality-anchored) with a large amount of real-world data that previous data analysis methods cannot fully understand, leverage complete statistics that combine reasoning of an admissible subset of results and statistical approximations by surprising engineering effects, to automate and enhance the analysis process. It pushes us to revisit data analysis to make better sense of data with PFMs. This paper provides a comprehensive review of systematic approaches to improving data analysis through the power of PFMs, while critically identifying the limitations of PFMs and also discusses possible directions for future research.