CodeCover

Measurement under Java

The following sections describe some special features of the instrumentation and coverage measurement process, which are provided by the Java instrumenter shipped together with CodeCover.

Coverage criteria

CodeCover can measure a number of so called coverage criteria. The basics are explained on the feature page Code Coverage and in the Specification. There are some specifics for Java, that are necessary to mention, which is done in the next few lines.

Statement coverage

Statement coverage is only measured for the following statement types:

In consequence, all conditional statements (e.g. if) and looping statements (e.g. while) are not considered for statement coverage. If a statement is executed and it causes an exception – e.g. a NullPointerException – the statement will nevertheless be covered. But the next statement will be uncovered, which can be a hint for an exception.

Branch coverage

Branches are created by:

The branch coverage, that is measured for Java, does not consider the body of a looping statement as a branch. Moreover the branches, created by possible exceptions – e.g. a NullPointerException – are ignored too.

Condition coverage

Boolean terms within the following statements are considered:

Boolean expressions in assignments, method parameters and ternary operators (a ? b : c) are ignored.

Our approach for conditional coverage needs a special instrumentation of Java boolean terms. Unfortunately this approach makes a boolean expression more complex. If assignments are used in boolean terms of if, while and so on, the compilers might get problems, to decide about the definite assignment when analyzing the control flow. Take a look at the following example:

public class B {
    public Object bar() { return null; }
    public Object foo(Object b) throws Exception {
        Object a;
        if (b == null || ((a = bar()) == null)) { throw new Exception();}
        return a.toString();
    }
}

Getting to know, whether a is definitely assigned before its usage is very complex after instrumentation. For this reason, CodeCover does not consider boolean expression, when there occurs at least one assignment (a = b) operator.

Loop coverage

Loop coverage is measured for the following statements:

Test case selection

All coverage data collected during a test run is called a test session. You can subdivide this test session into test cases. There is a number of methods to do this.

One test case

If you just instrument your classes and run it on your own, only one test case containing all coverage data will be produced. This test case has the name UNNAMED TESTCASE.

Comments

You can use specific comments in source files, that will be instrumented. These comments will be translated into method calls during the instrumentation process. For this reason, it is important, that you place these comments at such positions in your source files, where a method call is allowed. Otherwise you will receive compiler errors.

The possible comments are: 

// startTestCase("NAME");
// startTestCase("NAME", "COMMENT");
// endTestCase();
// endTestCase("NAME");
// endTestCase("NAME", "COMMENT");
// finishTestSession();

You have to ensure, that a comment looks exactly like one of these patterns. Otherwise a comment will be ignored.

The start and end test case comments explicitly start or end a test case. The start of a test case implicitly ends a prior test case, if it has not ended yet. So two test cases can not overlap.

The finish test session comment forces the coverage measurement to stop, flush all remaining coverage counters and close the log. No more coverage results will be collected after this call. You can use this method for example to finish the coverage measurement of a Java dynamic web project. Here the ShutdownHook approach of CodeCover might not catch the shutdown of the server or webapp.

Method calls

If you like the method of the comments, but want to use dynamic Strings as parameters, you can use specific method calls in your own test suite. For this approach, you need to add a special jar file to your class path:

%CodeCoverRelease%/lib/java-protocol-dummy.jar

And you have to call one of the methods of the class:

org.codecover.instrumentation.java.measurement.Protocol

Here is an example for this approach:

try {
    Person person = null;
    for (int i = 0; i < MAX; i++ {
        person = persons[i];
        Protocol.startTestCase(person.getName());
        person.prepare();
        person.callTestMethods();

        // throws Exception
        String result = person.checkResults();
        Protocol.endTestCase(person.getName(), result);
    }
} catch (Exception e) {
    Protocol.endTestCase(person.getName(),
        "An Exception occurred " + e.getMessage()) ;
}

JUnit

CodeCover supports JUnit for the test case selection too. For this approach you don't need to instrument your JUnit TestCases. But you have to use so called TestRunners provided by CodeCover in the jar:

JUnit-TestRunner.jar

There are a number of TestRunners available:

These test runners can be called by:

java -cp junit.jar:JUnit-TestRunner.jar:bin
     org.codecover.junit.swing.TestRunner
     [-methodsAsTestCases]
     de.myprogram.AllTests

Where AllTests is a JUnit test suite. There is an optional argument: -methodsAsTestCases.
This tells CodeCover whether to use JUnit test cases or the methods of JUnit test cases as test cases in the understanding of the software. CodeCover will explicitly start end end test cases synchronously to JUnit. If there are own calls of start test case, using one of the methods described above, they might be ignored, because JUnit test cases have a higher priority.

Coverage result

The coverage results of a test run is stored in a so called coverage log file. Per default its file name has the following format:

coverage-log-yyyy-MM-dd-HH-mm-ss-SSS.clf

This file is stored in the current directory. To change the target of this coverage log file, you can use one of the following methods:

  1. System property:
    You can set a so called system property when starting your instrumented program. The name of the property is:
    org.codecover.coverage-log-file.
    An example call: 
    java -Dorg.codecover.coverage-log-file=archiv/coverage1.clf
     -cp bin:library.jar
     Main
  2. Environment variable:
    You can set a system variable with the name:
    org.codecover.coverage-log-file
  3. CoverageLogPath:
    After having instrumented your source files, additionally classes have been added. One of them is: org.codecover.instrumentation.java.measurement.CoverageLogPath
    You just can change the return value of the method getDefaultPath() to another file name or another path. Afterwards you have to recompile this class.

If a coverage log file with the same name already exists, the name of the new coverage log file will be extended by (1). If you want to enable overwriting, set the following system variable to true or on:

org.codecover.coverage-log-overwrite

Example:

java -Dorg.codecover.coverage-log-file=archiv/coverage2.clf
     -Dorg.codecover.coverage-log-overwrite=true
     -cp bin:library.jar
     Main

Note: Only if there was at least one coverage data—e.g. one statement executed—the coverage log file is produced. Otherwise this file is not created, indicating that no coverage was made.

Special characteristics

When instrumenting your Java source files, some additional files will be added. They are in the following package:

org.codecover

You have to compile them and add them to your class path when executing your program, because these files are needed for the coverage measurement.

The coverage measurement under Java is not threadsafe. For this reason, the number of executions for a statement might not be correct, when your program uses multithreading. Nevertheless, CodeCover gets to know when a statement is covered correctly.

The measurement approach under Java uses a so called ShutdownHook, to guarantee, that all measurement results are captured. If you use an own ShutdownHook its effected coverage data might not be captured.

If you use an own ClassLoader you have to ensure, that this classloader is not instrumented because the instrumentation causes problems in this specific case.

SourceForge.net Logo Valid XHTML 1.0 Strict Viewable With Any Browser