Adaptive Statistical–Entropy Preprocessing (ASEP): A Novel Framework for Noise-Resilient Clinical Data Modeling

Abstract

Accurate air quality prediction depends heavily on the reliability of data preprocessing, as environmental datasets often contain noise, missing values, and inconsistent measurements. This study introduces a novel Adaptive Statistical–Entropy Preprocessing (ASEP) framework that enhances data quality before model training. The algorithm integrates entropy-based imputation, variance-guided noise removal, distribution-sensitive normalization, and correlation entropy-driven feature pruning to improve data uniformity and signal clarity. Using publicly available air quality data, ASEP demonstrates significant improvements in balance, consistency, and feature diversity compared to existing methods. The framework effectively minimizes redundancy and noise while preserving meaningful variability within pollutant readings. Designed to be scalable and interpretable, ASEP offers a robust and computationally efficient preprocessing solution for real-time air quality monitoring and prediction. Its results highlight the importance of adaptive, data-driven preprocessing as a foundation for accurate and sustainable environmental modeling.