Change detection is an important task in remotely monitoring and diagnosing equipment and other processes. Specifically, early detection of differences that indicate abnormal conditions has the promise to provide considerable savings in averting secondary damage and preventing system outage. Of course, accurate early detection has to be balanced against the successful rejection of false positive alarms. In noisy environments, such as aircraft engine monitoring, this proves to be a difficult undertaking for any one algorithm. In this paper, we investigate the performance improvement that can be gained by aggregating the information from a set of diverse change detection algorithms. Specifically, we examine a set of change detectors that utilize a variety of different techniques such as neural nets, random forests, and support vector machines. The different techniques have different detection sensitivities and different regression technique that operates well for time series as well as averaging schemes, and a meta-classifiers. We provide results using illustrative examples from aircraft engine monitoring.
In this paper, we present a feature selection and classification approach that was used to assess highly noisy sensor data from a NDE field study. Multiple, heterogeneous NDT sensors were employed to examine the solid structure. The goal was to differentiate between two types of phenomena occurring in a solid structure where one phenomenon was benign, the other was malignant. Manual distinction between these two types is almost impossible. To address these issues, we used sensor validation techniques to select the best available sensor that had the least noise effects and the best defect signature in the region of interest. Hundreds of features were formulated and extracted from data of the selected sensors. Next, we employed separability measures and correlation measures to select the most promising set of features. Because the NDE sensors poorly described the different defect types under consideration, the resulting features also exhibited poor separability. The focus of this paper is on how one can improve the classification under these constraints while minimizing the risk of overfitting (the number of field data was small). Results are shown from a number of different classifiers and classifier ensembles that were tuned to a set true positive rate using the Neyman-Pearson criterion.
In design of partial discharge (PD) diagnostic systems, finding a set of features corresponding to an optimal classification performance (accuracy and reliability) is critical. A diagnostic system designer typically does not have much difficulty to obtain a decent number of features by applying different feature extraction methods on PD measurements. However, the designer often faces challenges in finding a set of features that give optimal classification performance for the given PD diagnosis problem. The primary reasons for that are: a) features cannot be evaluated individually since feature interaction affects classification performance more significantly than features themselves; and b) optimal features cannot be obtained by simply combining all features from different feature extraction methods since there exist redundant and irrelevant features. This paper attempts to address the challenge by introducing feature selection to PD diagnosis. Through an example this paper demonstrates that feature selection can be an effective and efficient approach for systematically finding a small set of features that correspond to an optimal classification performance of PD diagnostic systems.
This paper explores classifier fusion problems where the task is selecting a subset of classifiers from a larger set with the goal to achieve optimal performance. To aid in the selection process we propose the use of several correlationbased diversity measures. We define measures that capture the correlation for n classifiers as opposed to pairs of classifiers only. We then suggest a sequence of steps in selecting classifiers. This method avoids the exhaustive evaluation of all classifier combinations which can become very large for larger sets of classifiers. We then report on observations made after applying that method to a data set from a real-world application. The classifier set chosen achieves close to optimal performance with a drastically reduced set of evaluation steps.
Classification requirements for real-world classification problems are often constrained by a given true positive or false positive rate to ensure that the classification error for the most important class is within a desired limit. For a sufficiently high true positive rate, this may result in the set-point being located somewhere in the flat portion of the ROC curve where the associated false positive rate is high. Any further classifier design will then attempt to reduce the false positive rate while maintaining the desired true positive rate is. We call this type of performance requirements for classifier design the constrained performance requirement. This type of performance requirements is different from the accuracy maximization requirement and thus requires different strategies for classifier design. This paper is concerned with designing classifier ensembles under such constrained performance requirements. Classifier ensembles are one of the most significant advances in pattern recognition/classification in recent years and have been actively studied by many researchers. However, not much attention has been given to designing ensembles to satisfy constrained performance requirements. This paper attempts to identify and address some of design related issues associated with the constrained performance requirement. Specifically, we present a design strategy for designing neural network ensembles to satisfy constrained performance requirements, which is illustrated by designing a real-world classification problem. The results are compared to those from conventional design method.
KEYWORDS: Sensors, Information fusion, Feature extraction, Sensor fusion, Data fusion, Environmental sensing, Data transmission, Reliability, Gas sensors, Sensor performance
Vibration monitoring is an important practice throughout regular operation of gas turbine power systems and, even more so, during characterization tests. Vibration monitoring relies on accurate and reliable sensor readings. To obtain accurate readings, sensors are placed such that the signal is maximized. In the case of characterization tests, strain gauges are placed at the location of vibration modes on blades inside the gas turbine. Due to the prevailing harsh environment, these sensors have a limited life and decaying accuracy, both of which impair vibration assessment. At the same time bandwidth limitations may restrict data transmission, which in turn limits the number of sensors that can be used for assessment. Knowing the sensor status (normal or faulty), and more importantly, knowing the true vibration level of the system all the time is essential for successful gas turbine vibration monitoring. This paper investigates a dynamic sensor validation and system health reasoning scheme that addresses the issues outlined above by considering only the information required to reliably assess system health status. In particular, if abnormal system health is suspected or if the primary sensor is determined to be faulted, information from available “sibling” sensors is dynamically integrated. A confidence expresses the complex interactions of sensor health and system health, their reliabilities, conflicting information, and what the health assessment is. Effectiveness of the scheme in achieving accurate and reliable vibration evaluation is then demonstrated using a combination of simulated data and a small sample of a real-world application data where the vibration of compressor blades during a real time characterization test of a new gas turbine power system is monitored.
Classifier performance evaluation is an important step in designing diagnostic systems. The purposes of performing classifier performance evaluation include: 1) to select the best classifiers from the several candidate classifiers, 2) to verify that the classifier designed meets the design requirement, and 3) to identify the need for improvements in the classifier components. In order to effectively evaluate classifier performance, a classifier performance measure needs to be defined that can be used to measure the goodness of the classifiers considered. This paper first argues that in fault diagnostic system design, commonly used performance measures, such as accuracy and ROC analysis are not always appropriate for performance evaluation. The paper then proposes using misclassification cost as a general performance measure that is suitable for binary as well as multi-class classifiers, and -most importantly- for classifiers with unequal cost consequence of the classes. The paper also provides strategies for estimating the cost matrix by taking advantage of fault criticality information obtained from FMECA. By evaluating the performance of different classifiers considered during the design process of an engine fault diagnostic system, this paper demonstrates that misclassification cost is an effective performance measure for evaluating the performance of multi-class classifiers with unequal cost consequence for different classes.
UAVs demand more accurate fault accommodation for their mission manager and vehicle control system in order to achieve a reliability level that is comparable to that of a pilot aircraft. This paper attempts to apply multi-classifier fusion techniques to achieve the necessary performance of the fault detection function for the Lockheed Martin Skunk Works (LMSW) UAV Mission Manager. Three different classifiers that meet the design requirements of the fault detection of the UAAV are employed. The binary decision outputs from the classifiers are then aggregated using three different classifier fusion schemes, namely, majority vote, weighted majority vote, and Naieve Bayes combination. All of the three schemes are simple and need no retraining. The three fusion schemes (except the majority vote that gives an average performance of the three classifiers) show the classification performance that is better than or equal to that of the best individual. The unavoidable correlation between the classifiers with binary outputs is observed in this study. We conclude that it is the correlation between the classifiers that limits the fusion schemes to achieve an even better performance.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.