Abstract
Tracking changes in feature distributions is very important in the domain of network anomaly detection. Unfortunately, these distributions consist of thousands or even millions of data points. This makes tracking, storing and visualizing changes over time a difficult task. A standard technique for capturing and describing distributions in a compact form is the Shannon entropy analysis. Its use for detecting network anomalies has been studied in-depth and several anomaly detection approaches have applied it with considerable success. However, reducing the information about a distribution to a single number deletes important information such as the nature of the change or it might lead to overlooking a large amount of anomalies entirely. In this paper, we show that a generalized form of entropy is better suited to capture changes in traffic features, by exploring different moments. We introduce the Traffic Entropy Spectrum (TES) to analyze changes in traffic feature distributions and demonstrate its ability to characterize the structure of anomalies using traffic traces from a large ISP.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Barford, P., Kline, J., Plonka, D., Ron, A.: A signal analysis of network traffic anomalies. In: IMW 2002: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, pp. 71–82. ACM, New York (2002)
Scherrer, A., Larrieu, N., Owezarski, P., Borgnat, P., Abry, P.: Non-gaussian and long memory statistical characterizations for internet traffic with anomalies. IEEE Transactions on Dependable and Secure Computing 4(1), 56–70 (2007)
Dubendorfer, T., Plattner, B.: Host behaviour based early detection of worm outbreaks in internet backbones. In: 14th IEEE WET ICE, pp. 166–171 (2005)
Cisco Systems Inc.: Netflow services solutions guide, http://www.cisco.com
Quittek, J., Zseby, T., Claise, B., Zander, S.: Rfc 3917: Requirements for ip flow information export (ipfix) (October 2004)
Wagner, A., Plattner, B.: Entropy based worm and anomaly detection in fast ip networks. In: 14th IEEE WET ICE, Linköping, Sweden (June 2005)
Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: ACM SIGCOMM, Portland (August 2004)
Li, X., Bian, F., Crovella, M., Diot, C., Govindan, R., Iannaccone, G., Lakhina, A.: Detection and identification of network anomalies using sketch subspaces. In: Internet Measurement Conference (IMC), Rio de Janeriro, Brazil, pp. 147–152. ACM, New York (2006)
Ziviani, A., Monsores, M.L., Rodrigues, P.S.S., Gomes, A.T.A.: Network anomaly detection using nonextensive entropy. IEEE Communications Letters 11(12) (2007)
Shannon, C.: Prediction and entropy of printed english. Bell System Tech. Jour. (January 1951)
Tsallis, C.: Possible generalization of boltzmann-gibbs statistics. J. Stat. Phys. 52 (1988)
Tsallis, C.: Nonextensive statistics: theoretical, experimental and computational evidences and connections. Brazilian Journal of Physics (January 1999)
Tsallis, C.: Entropic nonextensivity: a possible measure of complexity. Chaos (January 2002)
Dauxois, T.: Non-gaussian distributions under scrutiny. J. Stat. Mech. (January 2007)
Wilk, G., Wlodarczyk, Z.: Example of a possible interpretation of tsallis entropy. arXiv cond-mat.stat-mech (November 2007)
Willinger, W., Paxson, V., Taqqu, M.S.: Self-similarity and heavy tails: Structural modeling of network traffic. In: Statistical Techniques and Applications (1998)
Kohler, E., Li, J., Paxson, V., Shenker, S.: Observed structure of addresses in ip traffic. In: Proceedings of the SIGCOMM Internet Measurement Workshop, pp. 253–266. ACM, New York (2002)
SWITCH: The swiss education and research network, http://www.switch.ch
Gu, Y., McCallum, A., Towsley, D.: Detecting anomalies in network traffic using maximum entropy estimation. In: IMC 2005, pp. 1–6. ACM, New York (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tellenbach, B., Burkhart, M., Sornette, D., Maillart, T. (2009). Beyond Shannon: Characterizing Internet Traffic with Generalized Entropy Metrics. In: Moon, S.B., Teixeira, R., Uhlig, S. (eds) Passive and Active Network Measurement. PAM 2009. Lecture Notes in Computer Science, vol 5448. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00975-4_24
Download citation
DOI: https://doi.org/10.1007/978-3-642-00975-4_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00974-7
Online ISBN: 978-3-642-00975-4
eBook Packages: Computer ScienceComputer Science (R0)