CoverageMeter

Code Coverage Measurement for C/C++ and C#

Part V
Appendix

Appendix A  Additional Tools

A.1  cmreport - Code Coverage Report Generation

Syntax: cmreport -m <csmes_file> -s <selection> ...


Where:

-m <argument> | --csmes=<argument>
CSMes file name
-l <argument> | --level=<argument>
code coverage level
-s <argument> | --select=<argument>
select executions using regular expression
-d <argument> | --deselect=<argument>
deselect executions using regular expression
-h <argument> | --html=<argument>
HTML report output file name
-h <argument> | --xml=<argument>
XML report output file name
--csv-method=<argument>
Generate a CSV report file for each method
--csv-function=<argument>
Generate a CSV report file for each file

Specific options for HTML/XML output:

--execution-level=<argument>
code coverage level for executions
--method-level=<argument>
code coverage level for methods
--css=<argument>
CSS style sheet
--icon=<argument>
Icon
--title=<argument>
Title
--global=<argument>
Global statistics, values=all, selected or selected_and_first
--method=<argument>
Method statistics, values=all, selected or selected_and_first
--execution=<argument>
Execution statistics, values=all, selected or selected_and_first
--source=<argument>
Source file statistics, values=all, selected or selected_and_first
--method-sort=<argument>
Method sorting, values=name or coverage
--execution-sort=<argument>
Execution sorting, values=name or coverage
--source-sort=<argument>
Source file sorting, values=name or coverage
--global-watermark-low-medium=<argument>
Global watermark setting
--global-watermark-medium-high=<argument>
Global watermark setting
--source-watermark-low-medium=<argument>
Source watermark setting
--source-watermark-medium-high=<argument>
Source watermark setting
--method-watermark-low-medium=<argument>
Method watermark setting
--method-watermark-medium-high=<argument>
Method watermark setting
--execution-watermark-low-medium=<argument>
Execution watermark setting
--execution-watermark-medium-high=<argument>
Execution watermark setting
--source-sublevels=<argument>
Maximum number of intermediate levels for sources
--execution-sublevels=<argument>
Maximum number of intermediate levels for executions
--global-sublevels=<argument>
Maximum number of intermediate levels for global statistics
--method-sublevels=<argument>
Maximum number of intermediate levels for methods
--source-filter-min=<argument>
Filtering out source files according the statistic
--source-filter-max=<argument>
Filtering out source files according the statistic
--method-filter-min=<argument>
Filtering out methods according the statistic
--method-filter-max=<argument>
Filtering out methods according the statistic
--execution-filter-min=<argument>
Filtering out executions according the statistic
--execution-filter-max=<argument>
Filtering out executions according the statistic
-b | --coverage-branch
code coverage on branch level only
-t | --test-coverage
test count mode
-D | --debug
debug flag
--toc
Table of contents
--manually-validated
Manually validated code fragments
--unexecuted
Unexecuted code fragments
--executed
Executed code fragments
--bargraph
Coverage data displayed in a bargraph

Specific options for CSV output:

--csv-field-separator=<argument>
Field separator for a CSV file
--csv-coma=<argument>
Coma (, or .) used for floats in a CSV file

cmreport is a utility which permits to generate HTML, XML or CSV reports from an instrumentation data base (.csmes file). It generates exactly the same reports as these generated by CoverageBrowser (see chap. 17) .

A.2  cmcsexeimport - Command Line Import Utility

Syntax: cmcsexeimport -m <csmes_file> -e <csexe_file> -t <title> [-p <policy>]


Where:

-m <argument>|--csmes=<argument>
CSMes file name
-e <argument>|--csexe=<argument>
CSExe file name
-t <argument>|--title=<argument>
Execution title
-p <argument>|--policy=<argument>
Import policy (ignore_duplicates, import_duplicates_and_empty, import_duplicates or merge)
-P|--passed
Execution marked as passed.
-F|--failed
Execution marked as failed.
-C|--check-manually
Execution marked as to be checked manually.

cmcsexeimport is a utility which permits to import an execution report (.csexe file) into an instrumentation data base (.csmes file). It behaves exactly like CoverageBrowser load of execution report function (see chap. 5.2) .

A.3  cmmerge - Merging Utility

Syntax: cmmerge <output_file> <input_file> ... <input_file>


cmmerge is a small utility which permits to merge several instrumentation databases (.csmes file) together. It behaves exactly like CoverageBrowser merge function (see chap. 16.1) .

Appendix B  Code Coverage Benchmarks

The sorting algorithm used for the tests is quicksort.
Source code:

CompilerNormal Execution
(time)
Execution with Instrumentation
(time)
Difference
(%)
GCC without optimization 100 140.3 40.3
GCC with optimization -Os 52.1 61.9 18.8
GCC with optimization -O1 51.6 61.5 19.2
GCC with optimization -O2 55.4 58 4.6
GCC with optimization -O3 55.5 58.1 4.5
Benchmark (sorting algorithm) on VIA Eden 600Mhz
CompilerNormal Execution
(time)
Execution with Instrumentation
(time)
Difference
(%)
GCC without optimization 100 102.2 2.2
GCC with optimization -Os 49.7 54.7 9.9
GCC with optimization -O1 48.9 55.7 13.8
GCC with optimization -O2 48.2 53 9.9
GCC with optimization -O3 48.3 53 9.6
Benchmark (sorting algorithm) on AMD Athlon 1330Mhz
CompilerNormal Execution
(time)
Execution with Instrumentation
(time)
Difference
(%)
GCC without optimization 100 119.6 19.6
GCC with optimization -Os 51.4 76.2 48.1
GCC with optimization -O1 49.4 76.4 54.6
GCC with optimization -O2 48 74 54
GCC with optimization -O3 48 74.3 54.8
Benchmark (sorting algorithm) on Intel Pentium-M 1300Mhz
Compiler Without Instrumentation
(bytes)
With Instrumentation
(bytes)
Difference
(bytes)
GCC without optimization data20256 236
  code12002304 1104
GCC with optimization -Os data20256 236
  code8001840 1040
GCC with optimization -O1 data20256 236
  code8961936 1040
GCC with optimization -O2 data20256 236
  code9761968 992
GCC with optimization -O3 data20256 236
  code9761968 992
Sorting algorithm code size

Appendix C  License

C.1  Purchasing a License

The CoverageMeter license is restricted to a single user. A permanent and valid email address is necessary to purchase a license. This email address cannot be modified after purchase.
To purchase the license proceed as follows:

  1. Open the registration window "Help->Registration".
  2. Enter your name, your email address and if necessary a comment.
  3. Click on "Generate License". An inactive license file is now generated.
  4. Click on "Send Email Verification". An email, which contains the activation key, will be sent to your email address.
    An other way to receive such key is to send an email to /pictures/email.gif key@coveragemeter.com. A reply email which contains the activation key will be sent in few minutes.
  5. Enter the activation key and click on "Email Verification".
  6. Open the URL http://www.coveragemeter.com/shop.html in your Internet browser, and purchase a license using the same email address.
  7. The license key will be sent via email. Enter it into the "Activation Key" field and click on "Activate License". CoverageMeter is now fully activated.
Watch demonstration…License Activation
Do not use a temporary or dummy email address for purchasing a license: this address is used for sending you the activation key if the license becomes inactivated or if you want to install it on an other computer.

C.2  Restrictions of the Evaluation Version

The evaluation version of CoverageMeter runs exactly like the registered version: the code coverage analysis of CoverageScanner produces exactly the same result and CoverageBrowser functionalities are exactly the same. The only difference is that a portion of the code coverage status is hidden (displayed in Magenta by CoverageBrowser).

C.3  License Check

If CoverageMeter detects any hardware modifications or if CoverageMeter is installed on another computer, the license becomes inactivated. In this case, CoverageBrowser displays a message that indicates that the evaluation mode is reactivated.
To reactivate the license proceed as follows:

  1. Open the registration window "Help->Registration".
  2. Click on "Send Email Verification". An email, which contains the activation key, will be sent to your registration email address.
  3. Enter the activation key and click on "Email Verification". Your license is now reactivated.

C.4  Techincal information

Location of the license file:

PlatformLocation
Microsoft® Windows%HOMEDRIVE%"%HOMEPATH%"\coveragemeter.lic
Linux$HOME/.coveragemeter.lic
Apple® Mac OS X$HOME/.coveragemeter.lic

Appendix D  Customizing I/O of CoverageScanner library

D.1  Custom I/O using C file access

The following example shows how to generate the execution report using the standard C file API.


To compile the example on Microsoft® Windows:

cscl customiofile.c

To compile the example on Linux or Apple® Mac OS X:

csgcc customiofile.c -o customiofile

Source code:

D.2  Custom I/O using SFTP protocol

The following example shows how to generate the execution report directly on a SFTP server. The SFTP server is part of SSH v2 and is available on most of the Unix platforms. On Microsoft® Windows, a free SSH server can be downloaded from http://www.freesshd.com.


To compile the example on Microsoft® Windows:

  1. Download libSSH2 from http://www.libssh2.org. Generate the library and set the environment variable LIBSSH2 to the location of the libSSH2 source code.
  2. To compile the example:
    cscl %LIBSSH2%\win32\debug_dll\libssh2.lib -DWIN32 --cs-libgen=/MTd 
         /MTd -I %LIBSSH2%\include ws2_32.lib customiosftp.c 
    
  3. Execute custom_io_sftp.exe.

To compile the example on Linux:

  1. Install the development package of libssh2.
  2. To compile the example:
    csgcc -lssh2 customiosftp.c -o customiosftp
    
  3. Execute custom_io_sftp.

Source code: