CoverageMeter
Code Coverage Measurement for C/C++

Part V
Frequently Asked Questions

Code Coverage Instrumentation

Does CoverageMeter support line coverage?

CoverageMeter does not support line coverage because this kind of measurement and statistic is not accurate.

This metric depends on how you format the code. For example, take the following function:

   1 int main()
   2 {
   3   if (true) return 1;
   4     foo();
   5   return 0;
   6 }

Execute it and the line code coverage will produce:

   1 int main()
   2 {
   3 HIT   if (true) return 1;
   4 MIS   foo();
   5 MIS   return 0;
   6 }

Line Coverage: 33%
Reformat the code as follows:

   1 int main()
   2 {
   3   if (true)
   4     return 1;
   5   foo();
   6   return 0;
   7 }

Execute it and the line code coverage will produce:

   1 int main()
   2 {
   3 HIT   if (true)
   4 HIT     return 1;
   5 MIS   foo();
   6 MIS   return 0;
   7 }

Line Coverage: 50%
Reformat the code as follows:

   1 int main()
   2 {
   3   if (true)
   4     return 1;
   5   foo(); return 0;
   6 }

Execute it and the line code coverage will produce:

   1 int main()
   2 {
   3 HIT   if (true)
   4 HIT     return 1;
   5 MIS   foo(); return 0;
   6 }

Line Coverage: 66%
Reformat the code as follows:

   1 int main()
   2 {
   3   if (true) return 1; foo(); return 0;
   4 }

Execute it and the line code coverage will produce:

   1 int main()
   2 {
   3 HIT   if (true) return 1; foo(); return 0;
   4 }

Line Coverage: 100%
This small example shows that line coverage produces very different result depending on how the source code is formated. The decision coverage provided by CoverageMeter is independent of the coding style.

The code coverage analysis of CoverageMeter version 3 is different to version 4, why?

CoverageMeter version 3 records all conditions but not the branches. CoverageMeter version 4 records all branches and it will record a condition only if this is not covered by a branch. This is more precise, because the instrumentation occurs after a complete branch resulted from a condition and not when the condition is checked.

Example: in the following example, CoverageMeter v3 records if the boolean variable a is true or false. This doesn’t give any information if do_some_code() or do_something_else() was fully executed. CoverageMeter v4 places the instrumentations just before the end of the then and else block. The boolean variable is a no more instrumented.

CoverageMeter v3
CoverageMeter v4
   1 if (a<< Condition instrumented (true and false))
   2 {
   3   do_some_code();
   4 }
   5 else
   6 {
   7   do_something_else();
   8 }
   1 if (a)
   2 {
   3   do_some_code();
   4 } << Instrumented
   5 else
   6 {
   7   do_something_else();
   8 } << Instrumented

Example: if the else block is suppressed, CoverageMeter v4 instruments the variable a becomes false. But due to the fact that the then block is instrumented, it does not instrument if a becomes true.

CoverageMeter v3
CoverageMeter v4
   1 if (a<< Condition instrumented (a==true and a==false))
   2 {
   3   do_some_code();
   4 }
   1 if (a<< Only a==false instrumented)
   2 {
   3   do_some_code();
   4 } << Instrumented

Compiling

Microsoft®Visual Studio® rebuilds always my project even if it is not modified.

This problem occurs if a project is compiled with code coverage and precompiled headers support enabled. Simply deactivate the precompiled headers in the concerned build mode.

Instrumentation

How to exclude a source file from the code coverage analysis?

Excluding some source files from the code coverage analysis needs to be performed during the compilation. Two methods are possible:

  1. Compile using CoverageScanner with the command line option --cs-off.
  2. Use CoverageScanner pragmas to deactivate the instrumentation, by adding the following lines after all #include commands.
       1 #ifdef __COVERAGESCANNER__
       2 #pragma CoverageScanner (cov-off)
       3 #endif
    Adding this portion of code at the top of the source file (and so before all #include commands), will also deactivate the coverage analysis of the headers.

My source code contains inline functions, are they instrumented?

Inline functions are instrumented like other functions. But if its header file is not in the current directory, it is necessary to tell CoverageScanner to instrument it too. The command line options --cs-include-file-regex, --cs-include-file-wildcard and --cs-include-path permits to force the instrumentation of additional files.

Is it normal that the .csmes file contains a copy of the source code?

Yes.
The .csmes file contains all information necessary for CoverageBrowser. That’s why the source code and the preprocessed source file is also included.

CoverageBrowser

Performance

Is there a way to minimize the CPU usage?

CoverageBrowser needs a lot of CPU resources and disk accesses to calculate the statistics for each execution. This calculation is performed in background, but can also be deactivated. Just proceed as follows: hide the column Coverage of the docking window "Executions" using the context menu "Show/Hide Columns->Coverage".

Statistics

How to get the code coverage statistic using a script?

CoverageBrowser does not provide a direct way to get the global code coverage statistic per command line. But, using CoverageBrowser API, it is possible to write a small C application which extracts this information. It had to perform the following operations:

  1. Start CoverageBrowser.
  2. Open a .csmes file.
  3. Read the list of all executions.
  4. Get the number of executed and unexecuted instrumentations for these executions.
  5. Stop CoverageBrowser.

Example: 

To download: http://www.coveragemeter.com/cmstat.c
To build an application using CoverageBrowser API see chapter D

When I’m exporting the statistics per methods I can only see the 65536 first entries

This is a limitation of Microsoft Excel or OpenOffice. Excel 97 can only display 16384 rows. Excel 2000 and Excel 2003 can display up to 65536 rows and newer version of Excel can display up to 1048576 rows. OpenOffice Calc is limited to using 32000 rows.

Reporting issues

How to report an issue concerning CoverageScanner?

It is possible to generate a log file which permits to investigate on issues concerning CoverageScanner. Proceed as follows:

  1. In a console Window: 
    cd %COVERAGEMETER%
coveragemeter_debug.bat
    This will install a debug version of CoverageScanner which will produce a set of log files.
  2. Reproduce your issue.
  3. In the console Window, press enter to install the normal version of CoverageScanner.
  4. The log files are located in %COVERAGEMETER%\logfiles. Zip it together and send it to CoverageMeter support (email.gif support@coveragemeter.com).
The log file may contains part of your source code. If this is an issue, simply remove the concerned lines with your favorite editor.