"More Debugging in Parallel" Resource Page

Projects used for evaluation

The following projects have been used for our evaluations with the EzUnit framework.

Archive format

For each project, an archive can be downloaded which contains the TCM for the initial run without any faults, and several TCMs for program versions with injected faults (use 7zip for unpacking). The structure of each archive is as follows:

|-- reference.txt
|-- 1-fault
    |-- [filename1].txt
    |-- [filename1].txt
    ...
|-- 2-fault
    |-- [filename1].txt
    ...
...

The file reference.txt contains the TCM of the unaltered program. Each subfolder contains TCMs for program versions with as many injected faults as denoted by the folder name. The file names contain the name of the projected and a timestamp when the run has been executed.

File format

Each TCM file consists of three sections. Each section begins with a '#' and a short description of the line format of that section (in EBNF), followed by the data sets, one set per line. Sections are divided by an empty line. The first section contains all tests executed by the test run with information about their result. The second section contains all executed units (e.g. methods); if a fault has been injected into the unit, the corresponding line will contain information about that fault. The third section contains the coverage matrix in column index format, where each line marks a test (in order of the tests in the first section) and contains the indices of the called units with the number of calls by that test. All indices are 0-indexed and point to the units in the second section.

An example file looks as follows:

  #tests: test_name, ' ', status(PASSED | FAILED | ERROR), ' ', [exception, { trace_mut_index | '(', { alternative_trace_mut_index, ',' }, ')' }]
  jester.tests.IgnoreListTest:testReadingEmptyIgnoreValues()V PASSED
  jester.tests.IgnoreListTest:testReadingIgnoreValues()V PASSED
  jester.tests.IgnoreListTest:testReadingIgnoreValuesEndOfLineSpecialCase()V PASSED
  jester.tests.IgnoreListTest:testReadingMultipleIgnoreValues()V PASSED
  jester.tests.IgnoreListTest:testReadingMultipleIgnoreBlankLines()V PASSED
  [...]

  #uuts: uut_name, [ '|', mutator_class, '|', change_position_in_file_buffer, '|', change ]
  jester.SourceChangeException:SourceChangeException(Ljava.lang.String;)V
  jester.RealMutationsList:RealMutationsList(Ljava.lang.String;Ljava.io.PrintStream;)V
  jester.RealClassTestTester:RealClassTestTester(Ljester.TestRunner;Ljester.MutationsList;)V
  jester.RealReport:redBar()V
  jester.ReportItem:ReportItem(Ljava.lang.String;Ljester.IgnoreListDocument;ILjava.lang.String;Ljava.lang.String;)V
  [...]

  #matrix (1 line per test, in order of tests): { mut_index, ' ', mut_call_count, ' ' }
  73 1 123 1 
  60 1 85 1 73 1 123 1 11 2 48 1 
  60 1 85 1 73 1 123 1 11 2 48 1 
  60 1 85 1 73 1 123 1 11 2 48 1 
  60 1 85 1 73 1 123 1 11 2 48 1 
  0 1 73 1 123 1 88 1 
  [...]

Readers for the matrix format have already been implemented for C++ and Java. The following sample programs can be used to read a given TCM for further evaluation.

Used Algorithms

The used algorithms can be used together with the matrix readers to evaluate a given matrix.

Download

Diagrams

	avg. w.e.	avg. w.e. (Tarantula)	lost TCs	lost UUTs	avg. bug races	avg. speed up	avg. speed up (Tarantula)	avg. num blocks	avg. w.e. per fault	avg. w.e. per fault (Tarantula)
barbecue
commons.codec
commons.lang
daikon
draw2d
eventbus
htmlparser
jaxen
jdepend
jester
jexel
jmeter
jparsec
mime
timeandmoney
xmlsec