Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme

David Landsberg; Hana Chockler; Daniel Kroening; Matt  Lewis

doi:10.1007/978-3-662-46675-9_8

Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme

David Landsberg, Hana Chockler, Daniel Kroening, Matt Lewis

University of Oxford

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

20 Citations (Scopus)

252 Downloads (Pure)

Abstract

Statistical Fault Localisation (SFL) is a widely used method for localizing faults in software. SFL gathers coverage details of passed and failed executions over a faulty program and then uses a measure to assign a degree of suspiciousness to each of a chosen set of program entities (statements, predicates, etc.) in that program. The program entities are then inspected by the engineer in descending order of suspiciousness until the bug is found. The effectiveness of this process relies on the quality of the suspiciousness measure. In this paper, we compare 157 measures, 95 of which are new to SFL and borrowed from other branches of science and philosophy. We also present a new measure optimiser Lex g , which optimises a given measure g according to a criterion of single bug optimality. An experimental comparison on benchmarks from the Software-artifact Infrastructure Repository (SIR) indicates that many of the new measures perform competitively with the established ones. Furthermore, the large-scale comparison reveals that the new measures Lex Ochiai and Pattern-Similarity perform best overall.

Original language	English
Title of host publication	Fundamental Approaches to Software Engineering
Subtitle of host publication	18th International Conference, FASE 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings
Editors	Alexander Egyed, Ina Schaefer
Place of Publication	London, UK
Publisher	Springer International Publishing
Pages	115-129
Number of pages	15
Volume	9033
ISBN (Print)	9783662466742
DOIs	https://doi.org/10.1007/978-3-662-46675-9_8
Publication status	Published - 2015

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	9033
ISSN (Print)	0302-9743

Access to Document

10.1007/978-3-662-46675-9_8

fase2015_submission_13Accepted author manuscript, 633 KB

Cite this

Landsberg, D., Chockler, H., Kroening, D., & Lewis, M. (2015). Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme. In A. Egyed, & I. Schaefer (Eds.), Fundamental Approaches to Software Engineering: 18th International Conference, FASE 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings (Vol. 9033, pp. 115-129). (Lecture Notes in Computer Science; Vol. 9033). Springer International Publishing. https://doi.org/10.1007/978-3-662-46675-9_8

Landsberg, David ; Chockler, Hana ; Kroening, Daniel et al. / Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme. Fundamental Approaches to Software Engineering: 18th International Conference, FASE 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings. editor / Alexander Egyed ; Ina Schaefer. Vol. 9033 London, UK : Springer International Publishing, 2015. pp. 115-129 (Lecture Notes in Computer Science).

@inbook{07fba49168d0498a9f44c808555378e0,

title = "Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme",

abstract = "Statistical Fault Localisation (SFL) is a widely used method for localizing faults in software. SFL gathers coverage details of passed and failed executions over a faulty program and then uses a measure to assign a degree of suspiciousness to each of a chosen set of program entities (statements, predicates, etc.) in that program. The program entities are then inspected by the engineer in descending order of suspiciousness until the bug is found. The effectiveness of this process relies on the quality of the suspiciousness measure. In this paper, we compare 157 measures, 95 of which are new to SFL and borrowed from other branches of science and philosophy. We also present a new measure optimiser Lex g , which optimises a given measure g according to a criterion of single bug optimality. An experimental comparison on benchmarks from the Software-artifact Infrastructure Repository (SIR) indicates that many of the new measures perform competitively with the established ones. Furthermore, the large-scale comparison reveals that the new measures Lex Ochiai and Pattern-Similarity perform best overall.",

author = "David Landsberg and Hana Chockler and Daniel Kroening and Matt Lewis",

year = "2015",

doi = "10.1007/978-3-662-46675-9_8",

language = "English",

isbn = "9783662466742",

volume = "9033",

series = "Lecture Notes in Computer Science",

publisher = "Springer International Publishing",

pages = "115--129",

editor = "Alexander Egyed and Ina Schaefer",

booktitle = "Fundamental Approaches to Software Engineering",

}

Landsberg, D, Chockler, H, Kroening, D & Lewis, M 2015, Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme. in A Egyed & I Schaefer (eds), Fundamental Approaches to Software Engineering: 18th International Conference, FASE 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings. vol. 9033, Lecture Notes in Computer Science, vol. 9033, Springer International Publishing, London, UK, pp. 115-129. https://doi.org/10.1007/978-3-662-46675-9_8

Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme. / Landsberg, David; Chockler, Hana; Kroening, Daniel et al.
Fundamental Approaches to Software Engineering: 18th International Conference, FASE 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings. ed. / Alexander Egyed; Ina Schaefer. Vol. 9033 London, UK: Springer International Publishing, 2015. p. 115-129 (Lecture Notes in Computer Science; Vol. 9033).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme

AU - Landsberg, David

AU - Chockler, Hana

AU - Kroening, Daniel

AU - Lewis, Matt

PY - 2015

Y1 - 2015

N2 - Statistical Fault Localisation (SFL) is a widely used method for localizing faults in software. SFL gathers coverage details of passed and failed executions over a faulty program and then uses a measure to assign a degree of suspiciousness to each of a chosen set of program entities (statements, predicates, etc.) in that program. The program entities are then inspected by the engineer in descending order of suspiciousness until the bug is found. The effectiveness of this process relies on the quality of the suspiciousness measure. In this paper, we compare 157 measures, 95 of which are new to SFL and borrowed from other branches of science and philosophy. We also present a new measure optimiser Lex g , which optimises a given measure g according to a criterion of single bug optimality. An experimental comparison on benchmarks from the Software-artifact Infrastructure Repository (SIR) indicates that many of the new measures perform competitively with the established ones. Furthermore, the large-scale comparison reveals that the new measures Lex Ochiai and Pattern-Similarity perform best overall.

AB - Statistical Fault Localisation (SFL) is a widely used method for localizing faults in software. SFL gathers coverage details of passed and failed executions over a faulty program and then uses a measure to assign a degree of suspiciousness to each of a chosen set of program entities (statements, predicates, etc.) in that program. The program entities are then inspected by the engineer in descending order of suspiciousness until the bug is found. The effectiveness of this process relies on the quality of the suspiciousness measure. In this paper, we compare 157 measures, 95 of which are new to SFL and borrowed from other branches of science and philosophy. We also present a new measure optimiser Lex g , which optimises a given measure g according to a criterion of single bug optimality. An experimental comparison on benchmarks from the Software-artifact Infrastructure Repository (SIR) indicates that many of the new measures perform competitively with the established ones. Furthermore, the large-scale comparison reveals that the new measures Lex Ochiai and Pattern-Similarity perform best overall.

U2 - 10.1007/978-3-662-46675-9_8

DO - 10.1007/978-3-662-46675-9_8

M3 - Conference paper

SN - 9783662466742

VL - 9033

T3 - Lecture Notes in Computer Science

SP - 115

EP - 129

BT - Fundamental Approaches to Software Engineering

A2 - Egyed, Alexander

A2 - Schaefer, Ina

PB - Springer International Publishing

CY - London, UK

ER -

Landsberg D, Chockler H, Kroening D, Lewis M. Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme. In Egyed A, Schaefer I, editors, Fundamental Approaches to Software Engineering: 18th International Conference, FASE 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings. Vol. 9033. London, UK: Springer International Publishing. 2015. p. 115-129. (Lecture Notes in Computer Science). doi: 10.1007/978-3-662-46675-9_8

Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme

Abstract

Publication series

Access to Document

Fingerprint

Cite this