Raqamli Transformatsiya va Sun’iy Intellekt
DATA PREPROCESSING METHODS FOR AGRICULTURAL STATISTICAL DATA: ANALYSIS AND COMPARISON
1 School 4
2 Jizzakh branch of the National University of Uzbekistan named after Mirzo Ulugbek
Zenodo DOI: 10.5281/zenodo.23343553
Keywords
Abstract
Agricultural statistical datasets combine measurements at different spatial and temporal scales and may contain repeated records, skewed inputs and inconsistent analysis units. This paper compares five preprocessing decisions using a public crop-yield dataset: key-based aggregation, handling of missing values, outlier treatment, feature scaling and temporal validation. The merged source contains 25,932 rows, but only 13,130 distinct country–year–crop units; 3,690 units have repeated yield values paired with different temperatures. In maize data, treating source rows as separate observations gives a temperature–yield Pearson correlation of −0.549, versus −0.481 after aggregation. Since the source has no missing cells, a controlled experiment masked 15% of test-year temperature and pesticide values over 20 random repetitions. Country-specific training medians reduced temperature reconstruction MAE from 6.09 °C (global median) to 0.63 °C. In a fixed 7-neighbor yield model evaluated on later years, min–max scaling reduced MAE from 1.780 to 0.733 t ha⁻¹. These results show that the analysis unit and scaling rule can substantially alter statistical conclusions and predictive behavior. They do not establish that one preprocessing method is universally optimal.
References
- Nodir, Rakhimov, Khasanov Dilmurod, Turakulov Olim, Parmonov Abdutolib, and Dusanov Xurshid. "Development of an algorithm for selecting repeat crops type and predicting yield in the conditions of agriculture in Uzbekistan." Plant Science Today 13, no. 3 (2026).
- D. Khasanov, N. Raximov, N. Raximov, "PREDICTION THE YIELD OF GRAIN CROPS USING BASIC MACHINE LEARNING ALGORITHMS," Raqamli Transformatsiya va Sun’iy Intellekt, 2(5), 40-46, 2024. https://dtai.uz/articles/prediction-the-yield-of-grain-crops-using-basic-machine-learning-algorithms/
- Shirinboyev, R., Rakhimov, N., Khasanov, D., & Xafizadinov, U. (2024). EVALUATING THE PERFORMANCE OF CONVOLUTIONAL NEURAL NETWORKS AND HYBRID CNN-SVM MODELS FOR SYMBOL RECOGNITION IN COMPLEX DATASETS. DTAI–2024, 1, 295-299.
- D. Khasanov, B. Daminova and M. Tojiyev, "The Impact of Normalization on Regression-Based Crop Yield Prediction: Accuracy and Efficiency Analysis," 2026 International Russian Smart Industry Conference (SmartIndustryCon), Sochi, Russian Federation, 2026, pp. 63-67, doi: 10.1109/SmartIndustryCon68821.2026.11493015.
- P. Arizo-García, S. Castiñeira-Ibáñez, E. Cruzado-Campos, B. Ricarte, C. Rubio, and A. San Bautista, “A standardized framework for cleaning non-normal yield data from wheat and barley crops, and validation using machine learning models for satellite imagery,” Agronomy, vol. 16, no. 3, 386, 2026, doi: 10.3390/agronomy16030386.
- N. Raximov, D. Khasanov, "THE IMPORTANCE OF FORECASTING CROP YIELDS AND REGIONAL CROP DISTRIBUTION BASED ON AGRO-CLIMATIC CHARACTERISTICS AND MARKET DYNAMICS," Raqamli Transformatsiya va Sun’iy Intellekt, 4(5), 1-5, 2026. https://dtai.uz/articles/the-importance-of-forecasting-crop-yields-and-regional-crop-distribution-based-on-agro-climatic-characteristics-and-market-dynamics/
- I. Laktionov, S. Semenov, and G. Diachenko, “Dataset for Crop Yield Prediction,” Zenodo, version 1, 2026, doi: 10.5281/zenodo.18761663.
- M. L. Zinzinhédo, M. F. Mitchozounnou, K. V. Salako, and R. Glèlè Kakaï, “Sensitivity of machine learning regression models to data structure and quality in crop yield prediction,” PLOS ONE, vol. 21, no. 9, e0353938, 2026, doi: 10.1371/journal.pone.0353938.
- Dilmurod, H., and R. Shirinboyеv. "THЕ MАIN THЕORY OF LINЕАR RЕGRЕSSION MODЕLS IN DАTА MINING." Sun’iy Intellekt Nazariyasi va Amaliyoti: Tajribalar, Muammolar va Istiqbollari (2024): 143-147.