The MIT Lincoln Laboratory IDS evaluation methodology is a practical solution in terms of evaluating the
performance of Intrusion Detection Systems, which has contributed tremendously to the research progress in
that field. The DARPA IDS evaluation dataset has been criticized and considered by many as a very outdated
dataset, unable to accommodate the latest trend in attacks. Then naturally the question arises as to whether
the detection systems have improved beyond detecting these old level of attacks. If not, is it worth thinking of
this dataset as obsolete? The paper presented here tries to provide supporting facts for the use of the DARPA
IDS evaluation dataset. The two commonly used signature-based IDSs, Snort and Cisco IDS, and two anomaly
detectors, the PHAD and the ALAD, are made use of for this evaluation purpose and the results support the
usefulness of DARPA dataset for IDS evaluation.
KEYWORDS: Computer intrusion detection, Data fusion, Neural networks, Sensors, Sensor fusion, Systems modeling, Detection and tracking algorithms, Data modeling, Analytics, Internet
The acceptability and usability of Intrusion Detection Systems get seriously affected with the data skewness in
network traffic. A large number of false alarms mean a lot in terms of the acceptability of Intrusion Detection
Systems. The reason for the increase in false alerts is that the normal traffic abound. Even with highly accurate
Intrusion Detection Systems, the effective detection rate of the minority attack types will be unacceptably low
and those attack types are often the most serious ones. Thus high accuracy is not necessarily an indicator
of high model quality, and therein lies the accuracy paradox of predictive analytics. The cost of missing an
attack is higher than the cost of false alarms. The data-dependent sensor fusion architecture presented in this
paper learns from the data and then appropriately gives weighting to the decisions of various Intrusion Detection
Systems. The fusion enriches these weighted decisions to provide a single decision, which is better than those of
the existing Intrusion Detection Systems. This method reduces the false positive rate and improves the overall
detection rate and also the detection rate of minority class types in particular.
The motivation behind the fusion of Intrusion Detection Systems was the realization that with the increasing traffic and increasing complexity of attacks, none of the present day stand-alone Intrusion Detection Systems can meet the high demand for a very high detection rate and an extremely low false positive rate. Multi-sensor fusion can be used to meet these requirements by a refinement of the combined response of different Intrusion Detection Systems. In this paper, we show the design technique of sensor fusion to best utilize the useful response from multiple sensors by an appropriate adjustment of the fusion threshold. The threshold is generally chosen according to the past experiences or by an expert system. In this paper, we show that the choice of threshold bounds according to the Chebyshev inequality priciple performs better. This approach also helps to solve the problem of scalability and has the advantage of failsafe capability. This paper theoretically models the fusion of Intrusion Detection Systems for the purpose of proving the improvement in performance, supplemented with the empirical evaluation. The combination of complementary sensors is shown to detect more attacks than the individual components. Since the individual sensors chosen detect sufficiently different attacks, their result can be merged for improved performance. The combination is done in different ways like (i) taking all the alarms from each system and avoiding duplications, (ii) taking alarms from each system by fixing threshold bounds, and (iii) rule-based fusion with a priori knowledge of the individual sensor performance. A number of evaluation metrics are used, and the results indicate that there is an overall enhancement in the performance of the combined detector using sensor fusion incorporating the threshold bounds and significantly better performance using simple rule-based fusion.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.