An Incorrect Data Detection Method for Big Data Cleaning of Machinery Condition Monitoring
- 13 March 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 67 (3), 2326-2336
- https://doi.org/10.1109/tie.2019.2903774
Abstract
The presence of incorrect data leads to the decrease of condition-monitoring big data quality. As a result, unreliable or misleading results are probably obtained by analyzing these poor-quality data. In this paper, to improve the data quality, an incorrect data detection method based on an improved local outlier factor (LOF) is proposed for data cleaning. First, a sliding window technique is used to divide data into different segments. These segments are considered as different objects and their attributes consist of time-domain statistical features extracted from each segment, such as mean, maximum and peak-to-peak value. Second, a kernel-based LOF (KLOF) is calculated using these attributes to evaluate the degree of each segment being incorrect data. Third, according to these KLOF values and a threshold value, incorrect data are detected. Finally, a simulation of vibration data generated by a defective rolling element bearing and three real cases concerning a fixed-axle gearbox, a wind turbine, and a planetary gearbox are used to verify the effectiveness of the proposed method, respectively. The results demonstrate that the proposed method is able to detect both missing segments and abnormal segments, which are two typical incorrect data, effectively, and thus is helpful for big data cleaning of machinery condition monitoring.Keywords
Funding Information
- National Natural Science Foundation of China (61673311)
- NSFC-Zhejiang Joint Fund for the Integration of Industrialization and Informatization (U1709208)
- National Program for Support of Top-notch Young Professionals
This publication has 33 references indexed in Scilit:
- Big Data: A SurveyMobile Networks and Applications, 2014
- Service Innovation and Smart Analytics for Industry 4.0 and Big Data EnvironmentProcedia CIRP, 2014
- Validation of vibration measurements for heavy duty machinery diagnosticsMechanical Systems and Signal Processing, 2013
- Distance-based outlier detectionProceedings of the VLDB Endowment, 2010
- Anomaly detectionACM Computing Surveys, 2009
- Student t‐tests for potentially abnormal dataStatistics in Medicine, 2009
- Outlier Detection for Compositional Data Using Robust MethodsMathematical Geosciences, 2008
- A Survey of Outlier Detection MethodologiesArtificial Intelligence Review, 2004
- Variance-constrained filtering for uncertain stochastic systems with missing measurementsIEEE Transactions on Automatic Control, 2003
- OPTIMISATION OF BEARING DIAGNOSTIC TECHNIQUES USING SIMULATED AND ACTUAL BEARING FAULT SIGNALSMechanical Systems and Signal Processing, 2000