• Overview
• Documentation
  • Code Coverage
  • CoverageBrowser
  • CoverageScanner
  • FAQ
  • Appendix
• Download
• User Forum
• License

Part III
CoverageScanner Reference Manual

• Introduction
• Compiler Support
• Supported Compilers
• C# compilers
• Microsoft^® Visual C++
• Intel^® C++ Compiler
• GNU gcc
• IDE Support
• GNU Makefile
• Scratchbox
• Qt
• qmake
• moc
• Visual Studio
• Visual Studio .NET C# Compiler
• Visual Studio .NET C++ Compiler
• Visual Studio Add-In
• Visual C++ Express
• Visual Studio 6.0
• eMbedded Visual C++
• Eclipse
• Xcode
• Command Line
• Synopsis
• Instrumenting using preprocessor defines
• C/C++ Library
• __coveragescanner_install
• __coveragescanner_testname
• __coveragescanner_teststate
• __coveragescanner_save
• __coveragescanner_clear
• __coveragescanner_filename
• __coveragescanner_set_custom_io
• C# Library
• CoverageScanner.__coveragescanner_testname
• CoverageScanner.__coveragescanner_teststate
• CoverageScanner.__coveragescanner_save
• CoverageScanner.__coveragescanner_clear
• CoverageScanner.__coveragescanner_filename
• CoverageScanner.__coveragescanner_set_custom_io
• Controlling the instrumentation during the compilation
• C/C++ Pragma
• C# regions
• Regular Expressions
• Code Coverage of Libraries
• Code Coverage of Static/Shared Libraries and DLL
• Code Coverage of Plugins/Manually Loaded Shared Libraries
• Code Coverage of Plugins Generated with Microsoft^® Visual Studio^®
• Code Coverage of Plugins Generated with GNU gcc
• Test Suite and TestCocoon
• Execution Name And Status
• Unit Testing
• Test Framework Support
• CppUnit
• QTestLib
• CxxTest
• CoverageScanner Adaptation to a Tool Suite

Chapter 19  Introduction

CoverageScanner is the application which inserts the instrumentation into the source code. It performs the following operations:

1. Runs the C preprocessor.
2. Instruments the preprocessed source code.
3. Compiles the source code using the native C/C++ compiler.

Mainly, during normal usage, it replaces the native compiler.

Chapter 20  C/C++ Compiler Support

Compiler support is carried out through a profile. A profile contains a set of declarations which makes it possible to adapt CoverageScanner to C/C++ preprocessors, compilers and linkers.

20.1  Supported Compilers

CoverageScanner executable for a specific compiler is called to cs+’native compiler name’ or alternatively ’native compiler name’+-cs. (example: cl.exe is the Microsoft^® compiler and the corresponding CoverageScanner executable is cscl.exe or cl-cs.exe).
It is also possible to call ’coveragescanner.exe’ directly when the option ’--cs-compiler=<compiler name>’ is specified.

The following compilers are currently supported by CoverageScanner:

20.1.1  C# compilers

The command line C# compiler of Microsoft^® Visual Studio^® .NET and Mono C# compiler are supported.

20.1.2  Microsoft^® Visual C++

The command line compiler and linker of Microsoft^® Visual C++ and Microsoft^® Visual C++ Toolkit 2003 are supported.

20.1.3  Intel^® C++ Compiler

The C/C++ command line compiler from Intel^® is supported.

20.1.4  GNU gcc

Only g++ and gcc command line compilers are directly supported.

To implement support for GNU cross-compilers proceed as follows:

1. Copy ’coveragescanner’ to the file ’cs+<compiler name>’.
   (ex: copy coveragescanner.exe csarm-linux-gcc.exe)
2. Copy the profile ’gcc.cspro’ or g++.cspro’ to ’<compiler name>+.cspro’.
   (ex: copy gcc.cspro arm-linux-gcc.cspro)
3. Edit the new profile and set the parameter TOOL to ’<compiler name>’.
   (ex: TOOL=arm-linux-gcc)
4. The new GNU cross-compiler ’cs+<compiler name>’ can now be used. It inserts the instrumentations and calls the ’<compiler name>’ for compilation.

20.2  Integrated Development Environment Support

20.2.1  GNU Makefile

Mostly, in makefiles, the C/C++ compiler and the linker are defined using the environment variable CC, CXX and LINK. This can be substituted by CoverageScanner by setting CC, CXX and LINK in the command arguments of make.


Example: make LINK=csg++ CXX=csg++ CC=csgcc

20.2.2  Scratchbox

If TestCocoon is installed on the root file system¹, a compiler wrapper is created for each compiler supported by Scratchbox. To invoke CoverageScanner, prepend cs to the name of the cross-compiler.

20.2.3  Qt Library from Nokia

Nokia qmake

The variables QMAKE_CC, QMAKE_CXX and QMAKE_LINK contain the name of the C compiler, the C++ compiler and the linker command line used by qmake to generate the makefiles. This variable can be set as a command line argument of qmake.


Example: qmake QMAKE_LINK=cslink QMAKE_CXX=cscl QMAKE_CC=cscl

An other alternative is to create a configuration called TestCocoon in the .pro file:

TestCocoon {
  QMAKE_CXX=cs$$QMAKE_CXX
  QMAKE_CC=cs$$QMAKE_CC
  QMAKE_LINK=cs$$QMAKE_LINK
}

Calling qmake CONFIG+=TestCocoon will generate the makefiles with code coverage activated.

Nokia Meta-Object Compiler (MOC)

Nokia Meta-Object Compiler add automatically in each classes derived from QObject new methods. For example, the translation function tr, the source code for all signals, the cast operator qt_cast, …In order to instrument the code using the Qt Library and not the library itself, CoverageScanner provides the command line options --cs-qt4 for Qt4 (resp. --cs-qt3 for Qt3).
In this case:

• Q_OBJECT macro is no more instrumented.
• All signal are instrumented in order to track their emission.
• The glue code necessary for the signal/slot mechanism is not instrumented.

20.2.4  Microsoft^® Visual Studio^®

TestCocoon provides a wrapper for link.exe and cl.exe located on the %TESTCOCOON%\visualstudio directory. It behaves exactly like the corresponding Microsoft^® wrapper except that the code coverage analysis becomes activated when the option --cs-on is added to the command arguments. These wrappers call the Microsoft^® tools for compilation or for linkage.

Microsoft^® Visual Studio^® .NET C# Compiler

To activate the instrumentation of C# source code, it is only necessary to add the define COVERAGESCANNER_COVERAGE_ON in the properties of the Microsoft^® Visual Studio^® .NET project. Other defines can be appended to select additional instrumentation options. The full list can be found on the chapter preprocessor-defines.

Microsoft^® Visual Studio^® .NET C/C++ Compiler

To use TestCocoon with Microsoft^® Visual Studio^® .NET proceed as follows:

1. Add the location of the CoverageScanner wrappers to the first position in the VC++ Directories.
   
   1. Start Microsoft^® Visual Studio^® .NET.
   2. Open the option dialog: click on "Tools->Preferences…".
   3. Select the item "Projects->VC++ Directories".
   4. Add the entry $(TESTCOCOON)\visualstudio to the first position in the list of directories. (see figure fig:visual_option_net_fig)

      Installation of CoverageScanner on Visual Studio® .NET: Setting the path of CoverageScanner. Installation of CoverageScanner on Visual Studio® .NET: Setting the path of CoverageScanner.

2. To activate code coverage analysis:
   1. Open a Visual C/C++ project.
   2. Edit the project settings (click on "Project->Properties").
   3. Add to the option --cs-on to the additional command line arguments of the C/C++ compiler and linker. (see figure fig:visual_compiler_net_fig)
   4. In the additional arguments of the linker, add the --cs-libgen which permits to specify which library should be used for the generation of the CoverageScanner library. The table tab:librarysettings contains the list of recommended settings.
   5. For Microsoft^® Windows CE applications, append to the linker arguments the command line option --cs-architecture which permits to specify the target platform. The table tab:architectures contains the list of available architectures.

Activation of the instrumentation under Visual Studio® .NET. Activation of the instrumentation under Visual Studio® .NET.

Activation of the instrumentation under Visual Studio® .NET. Activation of the instrumentation under Visual Studio® .NET.

TestCocoon Add-In for Microsoft^® Visual Studio^® 2005, 2008 and 2010

TestCocoon provides for Microsoft^® Visual Studio^® 2005, 2008 and 2010 an Add-In which does automatically the steps described in the section visualstudiodotnet. To use it:

1. Open a C++ project.
2. Click on Tools->Code Coverage Build Mode….
3. Select the base configuration and the project which should be compiled with code coverage support.
4. Click on Enable code coverage to generate a new build mode with code coverage support. Disable code coverage removes all CoverageScanner command line options.

Microsoft® Visual Studio® 2005 & 2008 Add-In Microsoft® Visual Studio® 2005 & 2008 Add-In

Microsoft^® Visual C++ Express

To use TestCocoon with Microsoft^® Visual C++ Express proceed as follows:

1. Add the location of the CoverageScanner wrappers to the first position in the VC++ Directories.
   
   1. Start Microsoft^® Visual C++ Express.
   2. Open the option dialog: click on "Tools->Preferences…".
   3. Select the item "Projects->VC++ Directories".
   4. Add the entry $(TESTCOCOON)\visualstudio to the first position in the list of directories. (see figure fig:visual_option_express_2005_fig)

      Installation of CoverageScanner on Visual C++ Express: Setting the path of CoverageScanner. Installation of CoverageScanner on Visual C++ Express: Setting the path of CoverageScanner.

2. The activation of the code coverage analysis is similar to Microsoft^® Visual Studio^® .NET. (see chap. visualstudiodotnet)

Microsoft^® Visual Studio^® 6.0

To use TestCocoon with Microsoft^® Visual Studio^® 6.0 proceed as follows:

1. Add the location of the CoverageScanner wrappers to the first position in the executable directories.
   
   1. Start Microsoft^® Visual Studio^® 6.0.
   2. Open the option dialog: click on "Tools->Preferences…".
   3. Select the item "Directories".
   4. Select "Executable files" in the combobox "Show directories for:".
   5. Add the path of the directory visualstudio of the TestCocoon installation² to the first position in the list of directories. (example: if TestCocoon is installed on c:\programme\TestCocoon, add the path c:\programme\TestCocoon\visualstudio, (see figure fig:visual_option_6_fig))

      Installation of CoverageScanner on Visual Studio® 6.0: Setting the path of CoverageScanner. Installation of CoverageScanner on Visual Studio® 6.0: Setting the path of CoverageScanner.

2. To activate the code coverage analysis:
   1. Open a Visual C/C++ project.
   2. Edit the project settings (click on "Project->Properties").
   3. Add the option --cs-on to the additional command line arguments of the C/C++ compiler and linker. (see figure fig:visual_compiler_6_fig)
   4. In the additional arguments of the linker, add the --cs-libgen which permits to specify which library should be used for the generation of the CoverageScanner library. The table tab:librarysettings contains the list of recommanded settings.

Activation of the instrumentation under Visual Studio® 6.0. Activation of the instrumentation under Visual Studio® 6.0.

Activation of the instrumentation under Visual Studio® 6.0. Activation of the instrumentation under Visual Studio® 6.0.

20.2.5  Microsoft^® eMbedded Visual C++^®

To use TestCocoon with Microsoft^® eMbedded Visual C++^®proceed as follows:

1. Add the location of the CoverageScanner wrappers to the first position in the executable directories.
   
   1. Start Microsoft^® eMbedded Visual C++^®.
   2. Open the option dialog: click on "Tools->Preferences…".
   3. Select the item "Directories".
   4. Select "Executable files" in the combobox "Show directories for:".
   5. Select "Platform" and the targeted "CPUs".
   6. Add the path of the directory WinCE of the TestCocoon installation³ to the first position in the list of directories. (example: if TestCocoon is installed on c:\programme\TestCocoon, add the path c:\programme\TestCocoon\WinCE, (see figure fig:visual_option_embedded_fig))

      Installation of CoverageScanner on eMbedded Visual C++®: Setting the path of CoverageScanner. Installation of CoverageScanner on eMbedded Visual C++®: Setting the path of CoverageScanner.

2. To activate the code coverage analysis:
   1. Open a Visual C/C++ project.
   2. Edit the project settings (click on "Project->Properties").
   3. Add the option --cs-on to the additional command line arguments of the C/C++ compiler and linker. (see figure fig:visual_compiler_embedded_fig)

Activation of the instrumentation under eMbedded Visual C++®. Activation of the instrumentation under eMbedded Visual C++®.

Activation of the instrumentation under eMbedded Visual C++®. Activation of the instrumentation under eMbedded Visual C++®.

20.2.6  Eclipse^™ IDE for C/C++

Eclipse^™ IDE for C/C++ rely on a GNU-GCC command line compiler. Before trying to compile a project with coverage support, it is necessary to create a compiler wrapper ((see chap. gcc-compiler)).


To compile a project with code coverage support, proceed as follows:

1. Start Eclipse^™.
2. Load a C/C++ project.
3. Open the property window (Project-> Properties)
4. Click on "C/C++ Build/Settings"
5. Create a new configuration by clicking on "Manage Configurations..." and select it.
6. Click on "Tools Settings" tab.
7. Click on "GCC C++ Compiler" and prepend cs to the name of the compiler.
8. Click on "GCC C Compiler" and prepend cs to the name of the compiler.
9. Click on "C++ Linker" and prepend cs to the name of the linker.

Eclipse™ settings Eclipse™ settings

20.2.7  Apple^® Xcode

To use TestCocoon with Apple^® Xcode proceed as follows: To activate the code coverage analysis:

1. Open a terminal window and set the CPLUSPLUS, LDPLUSPLUS, LD and CC to CoverageScanner compiler wrapper. Start Xcode using the open command. Example: 

   TESTCOCOON=/Applications/TestCocoon/wrapper
   export CC=$TESTCOCOON/gcc
   export LD=$TESTCOCOON/gcc
   export CPLUSPLUS=$TESTCOCOON/g++
   export LDPLUSPLUS=$TESTCOCOON/g++
   
   open /Developer/Applications/Xcode.app
   

2. Open a Xcode C/C++ project.
3. Edit the project settings (click on "Project->Edit Project Settings").
4. Add the option --cs-on to the additional command line arguments of the C/C++ compiler (fields Other C Flags and Other C++ Flags) and linker (field Other Linker Flags). (see figure fig:xcode_configuration_fig)
5. Disable the usage of the precomiled header: Open the settings of the active target ("Project->Edit Active Target") and remove the contains of Prefix Header.

Activation of the instrumentation under Apple® Xcode. Activation of the instrumentation under Apple® Xcode.

Chapter 21  Command Line Reference

21.1  Synopsis

Where:

--cs-include-file-wildcard=<STRING>

This command line option enables the user to include a file in the coverage analysis. The file name is specified using a wildcard expression.
Example: --cs-include-file-wildcard=^c:\\include\\*.h includes all header file names with the extension .h located in c:\include.

--cs-exclude-file-wildcard=<STRING>

This command line option enables the user to exclude a file from the coverage analysis. The file name is specified using a wildcard expression.
Example: --cs-exclude-file-wildcard=*.h excludes all header file names with the extension .h.

--cs-include-file-regex=<STRING>

This command line option enables the user to include a file in the coverage analysis. The file name is specified using a regular expression. (see chapter sec:regular_expression for the syntax)
Example: --cs-include-file-regex=^c:\\include\\[^\\]*\.h$ includes all header file names with the extension .h located in c:\include.

--cs-exclude-file-regex=<STRING>

This command line option enables the user to exclude a file from the coverage analysis. The file name is specified using a regular expression. (see chapter sec:regular_expression for the syntax)
Example: --cs-exclude-file-regex=^.*\.h$ excludes all header file names with the extension .h.

--cs-exclude-path=<STRING>

This command line option enables the user to exclude the files located in a specific path from the code coverage analysis.
Example: --cs-exclude-path=c:\include excludes all files located in c:\include.

--cs-include-path=<STRING>

This command line option enables the user to include files located in a specific path in the coverage analysis.
Example: --cs-include-path=c:\include includes all files located in c:\include.

--cs-include-function-wildcard=<STRING>

This command line option enables the user to include a function in the coverage analysis. The function name is specified using a wildcard expression and must include for C++ code the class name and namespace.
Example: --cs-include-function-wildcard=C::* includes all members of the class C.

--cs-exclude-function-wildcard=<STRING>

This command line option enables the user to exclude a function from the coverage analysis. The function name is specified using a wildcard expression and must include for C++ code the class name and namespace.
Example: --cs-exclude-function-wildcard=*::Get* excludes all merbers of any classes which starts with Get.

--cs-include-function-regex=<STRING>

This command line option enables the user to include a function in the coverage analysis. The function name is specified using a regular expression and must include for C++ code the class name and namespace. (see chapter sec:regular_expression for the syntax)

--cs-exclude-function-regex=<STRING>

This command line option enables the user to exclude a function from the coverage analysis. The function name is specified using a regular expression and must include for C++ code the class name and namespace. (see chapter sec:regular_expression for the syntax)

--cs-compiler=<STRING>

This command option enables the user to select a profile. This option does not normally need to be specified, due to the fact that the profile name is usually extracted from the command line name. (cscl.exe implicitly selects the profile cl.cspro)
Example: --cs-compiler=cl selects the profile ’cl.cspro’.

--cs-architecture=<STRING>

Permits to specify the target architecture.

--cs-libgen=<STRING>

Adds an additional command line option to the compiler which generates the code coverage library. This command can be used more than once.

--cs-off

Disables the code coverage analysis of CoverageScanner. This is equivalent to calling the native compiler or linker.

--cs-on

Activates the code coverage analysis of CoverageScanner. This option is set by default.

--cs-hit

Generation of code coverage hit instrumentation. This option is set by default.

--cs-count

Generation of code coverage count instrumentation.

--cs-branch

Generation of code coverage instrumentation of branches.

--cs-decision

Generation of code coverage instrumentation of branches and decisions.

--cs-condition

Generation of code coverage instrumentation of branches, decisions and conditions. (default)

--cs-partial-instrumentation

Suppression of redundant conditions. (default)

--cs-full-instrumentation

No suppression of redundant conditions.

--cs-no-exceptions

Do not instrument the catch block of a try…catch statement.

--cs-qt4

Function filter which suppresses the instrumentation of class members (Q_OBJECT, qt_metacall, qt_metacast, …) generated by Nokia Meta-Object Compiler (MOC) for Qt4 library.

--cs-qt3

Function filter which suppresses the instrumentation of class members (Q_OBJECT, className, tr, …) generated by Nokia Meta-Object Compiler (MOC) for Qt3 library.

--cs-output

Set the default output file name of the execution report. Per default, CoverageScanner uses coverage.csexe. The extension .csexe is added automatically.

--cs-lock-csexe

Create a lock file which lock the access of the .csexe file during its generation.

--cs-nolock-csexe

Inverse option to --cs-lock-csexe.

--cs-pipe

The preprocessor output is transmitted to CoverageScannerthrough a pipe. This option is set by default.

--cs-nopipe

The preprocessor output is transmitted to CoverageScannerthrough a temporary file.

--cs-keep-instrumentation-files

Do not delete temporary source files.

21.2  Instrumenting using preprocessor defines

CoverageScanner activates also the instrumentation if some defines are present in the command line option. This permits to instrument code with an IDE which does not permits to add custom command line options to the compiler or linker.
Example: Defining COVERAGESCANNER_COVERAGE_ON is equivalent of adding --cs-on to the command line.

21.3  C/C++ Library

CoverageScanner library is generated during the link process and enables the user to:

1. Initialize measurements.
2. Generate the execution report.
3. Give a name to the executed tests.
4. Install a signal handler which saved the execution report on exit.

Additionally CoverageScanner sets the preprocessor define __COVERAGESCANNER__ which excludes this function from a normal compilation.

21.3.1  __coveragescanner_install

Syntax: void __coveragescanner_install(const char *appname)


__coveragescanner_install performs complete initialization of the CoverageScanner library. This function is the first function called in the main() function. The parameter appname contains the executable name. The application name is used to define the measurement file name (by appending ’.csexe’ to appname).


__coveragescanner_install registers an exit handler via atexit() which stores execution traces using the __coveragescanner_save procedure. This handler is also called when SIGABRT, SIGTERM, SIGFPE, SIGILL, SIGINT, SIGSEGV and SIGTERM signal are received. This makes it possible to save the execution report upon every application termination.


Suggested usage:

21.3.2  __coveragescanner_testname

Syntax: void __coveragescanner_testname(const char *name)


__coveragescanner_testname sets the name of the test which is currently being executed. It will be saved to the execution report and is displayed in the "Executions" window (see chap. execution-management) when the loaded in CoverageBrowser.

21.3.3  __coveragescanner_teststate

Syntax: void __coveragescanner_teststate(const char *state)


__coveragescanner_teststate sets the state of the test which is currently being executed. The string parameter state can have the following values:

"PASSED"

indicates that the test was successfully executed.

"FAILED"

indicates that the test was not successfully passed.

"CHECK_MANUALLY"

indicates that it was not possible to determinate if the test was successfully executed.

It will be saved to the execution report and is displayed in the "Executions" window (see chap. execution-management) when the loaded in CoverageBrowser.

21.3.4  __coveragescanner_save

Syntax: void __coveragescanner_save()


__coveragescanner_save saves the execution report and resets the status of all instrumentations.

21.3.5  __coveragescanner_clear

Syntax: void __coveragescanner_clear()


__coveragescanner_clear resets the status of all instrumentations.

21.3.6  __coveragescanner_filename

Syntax: void __coveragescanner_filename(const char *name)


this function sets the file name of the execution report. The extension ’.csexe’ is added automatically. The name must be less than 80 characters.

21.3.7  __coveragescanner_set_custom_io

Syntax: void __coveragescanner_set_custom_io(
           char *(*csfgets)(char *s, int size, void *stream),
           int (*csfputs)(const char *s, void *stream),
           void *(*csfopenappend)(const char *path),
           void *(*csfopenread)(const char *path),
           void *(*csfopenwrite)(const char *path),
           int (*csfclose)(void *fp), 
           int (*csremove)(const char *filename) ) 



Parameters:

csfgets

csfgets reads in at most one less than size characters from stream and stores them into the buffer pointed to by s. Reading stops after an EOF or a newline. If a newline is read, it is stored into the buffer. A ’\0’ is stored after the last character in the buffer.

csfputs

csfputs writes the string s to stream, without its trailing ’\0’.

csfopenappend

csfopenappend open the file path for writing at the end.

csfopenread

csfopenread open the file path for reading.

csfopenwrite

csfopenwrite open the file path for writing.

csfclose

The csfclose function will flush and close the stream pointed to by fp.

csremove

csremove removes filename from the file system.

CoverageScanner writes the execution report (.csexe file) into file using the common C library (fopen, fputs, …). In some situations, for embedded systems for example, it is necessary to use an other way for recording it. For this reason, CoverageScanner provides __coveragescanner_set_custom_io which permits to replace the IO functions used by __coveragescanner_save.

Default implementation:

Some examples are available in the annex customioexample.

21.4  C# Library

CoverageScanner library is generated during the link process and enables the user to:

1. Generate the execution report.
2. Give a name to the executed tests.
3. Install a signal handler which saved the execution report on exit.

Additionally CoverageScanner sets the preprocessor define __COVERAGESCANNER__ which excludes this function from a normal compilation.

21.4.1  CoverageScanner.__coveragescanner_testname

Syntax: void CoverageScanner.__coveragescanner_testname(string name)


CoverageScanner.__coveragescanner_testname sets the name of the test which is currently being executed. It will be saved to the execution report and is displayed in the "Executions" window (see chap. execution-management) when the loaded in CoverageBrowser.

21.4.2  CoverageScanner.__coveragescanner_teststate

Syntax: void CoverageScanner.__coveragescanner_teststate(string state)


CoverageScanner.__coveragescanner_teststate sets the state of the test which is currently being executed. The string parameter state can have the following values:

"PASSED"

indicates that the test was successfully executed.

"FAILED"

indicates that the test was not successfully passed.

"CHECK_MANUALLY"

indicates that it was not possible to determinate if the test was successfully executed.

It will be saved to the execution report and is displayed in the "Executions" window (see chap. execution-management) when the loaded in CoverageBrowser.

21.4.3  CoverageScanner.__coveragescanner_save

Syntax: void CoverageScanner.__coveragescanner_save()


CoverageScanner.__coveragescanner_save saves the execution report and resets the status of all instrumentations.

21.4.4  CoverageScanner.__coveragescanner_clear

Syntax: void CoverageScanner.__coveragescanner_clear()


CoverageScanner.__coveragescanner_clear resets the status of all instrumentations.

21.4.5  CoverageScanner.__coveragescanner_filename

Syntax: void CoverageScanner.__coveragescanner_filename(string name)


this function sets the file name of the execution report. The extension ’.csexe’ is added automatically. The name must be less than 80 characters.

21.4.6  CoverageScanner.__coveragescanner_set_custom_io

Syntax: void CoverageScanner.__coveragescanner_set_custom_io(
           __cs_fgets_delegate cs_fgets,
           __cs_fputs_delegate cs_fputs,
           __cs_fopenappend_delegate cs_fopenappend,
           __cs_fopenread_delegate cs_fopenread,
           __cs_fopenwrite_delegate cs_fopenwrite,
           __cs_fclose_delegate cs_fclose, 
           __cs_remove_delegate cs_remove) 



Parameters:

csfgets

Type:public delegate string __cs_fgets_delegate(System.IO.Stream stream)
csfgets reads in at most one less than size characters from stream and stores them into the buffer pointed to by s. Reading stops after an EOF or a newline. If a newline is read, it is stored into the buffer. A ’\0’ is stored after the last character in the buffer.

csfputs

Type:public delegate void __cs_fputs_delegate(string s, System.IO.Stream stream)
csfputs writes the string s to stream, without its trailing ’\0’.

csfopenappend

Type:public delegate System.IO.Stream __cs_fopenappend_delegate(string path)
csfopenappend open the file path for writing at the end.

csfopenread

Type:public delegate System.IO.Stream __cs_fopenread_delegate(string path)
csfopenread open the file path for reading.

csfopenwrite

Type:public delegate System.IO.Stream __cs_fopenwrite_delegate(string path)
csfopenwrite open the file path for writing.

csfclose

Type:public delegate void __cs_fclose_delegate(System.IO.Stream fp)
The csfclose function will flush and close the stream pointed to by fp.

csremove

Type:public delegate void __cs_remove_delegate(string n)
csremove removes filename from the file system.

CoverageScanner writes the execution report (.csexe file) into file using the common C# file IO functions. In some situations, for embedded systems for example, it is necessary to use an other way for recording it. For this reason, CoverageScanner provides CoverageScanner.__coveragescanner_set_custom_io which permits to replace the IO functions used by CoverageScanner.__coveragescanner_save.

21.5  Controlling the instrumentation during the compilation

CoverageScanner offers the possibility to modify the coverage settings of some source code parts during the compilation. This permits, for example, to exclude some source code from the coverage analysis.

21.5.1  C/C++ Pragma

CoverageScanner defines C/C++ pragmas which permits to control during the compilation the generation of the instrumentation. All control commands are handled in a stack. The pragmas have the following syntax:
#pragma CoverageScanner ( string )

List of supported pragmas:

#pragma CoverageScanner(cov-on)

Enables code coverage instrumentation.

#pragma CoverageScanner(cov-off)

Disables code coverage instrumentation.

#pragma CoverageScanner(cov-hit)

Instruments using code coverage hit.

#pragma CoverageScanner(cov-count)

Instruments using code coverage count.

#pragma CoverageScanner(cov-branch)

Select the code coverage branch as instrumentation method.

#pragma CoverageScanner(cov-decision)

Select the code coverage decision as instrumentation method.

#pragma CoverageScanner(cov-condition)

Select the code coverage condition as instrumentation method.

#pragma CoverageScanner(cov-partial-instrumentation)

Partial instrumentation of conditions and decisions.

#pragma CoverageScanner(cov-full-instrumentation)

Full instrumentation of conditions and decisions.

#pragma CoverageScanner(pop)

Restore the instrumentation option before the call of the last CoverageScanner pragma.

21.5.2  C# regions

CoverageScanner defines C# extensions of the #region keyword which permits to control during the compilation the generation of the instrumentation. The #region extension have the following syntax:
#region CoverageScanner ( string )

The keyword #endregion restores the generation options.

List of supported regions:

#region CoverageScanner(cov-on)

Enables code coverage instrumentation.

#region CoverageScanner(cov-off)

Disables code coverage instrumentation.

#region CoverageScanner(cov-hit)

Instruments using code coverage hit.

#region CoverageScanner(cov-count)

Instruments using code coverage count.

#region CoverageScanner(cov-branch)

Select the code coverage branch as instrumentation method.

#region CoverageScanner(cov-decision)

Select the code coverage decision as instrumentation method.

#region CoverageScanner(cov-condition)

Select the code coverage condition as instrumentation method.

#region CoverageScanner(cov-partial-instrumentation)

Partial instrumentation of conditions and decisions.

#region CoverageScanner(cov-full-instrumentation)

Full instrumentation of conditions and decisions.

21.6  Regular Expressions

Regular expressions are composed of items (see table tab:regexp_item for the full list) and combining operators. These can be:

• […]: The brackets are used to define a set of characters. For example, [123] corresponds to the characters 1, 2 or 3. It is possible to define a range using the minus sign: [a-z] corresponds to all characters between a and z.
• [^…]: The caret defines a set of characters which should not match. For example, [^0-9] corresponds to any character except a digit.
• ?: The question mark after a regular expression item makes it optional.
  Example: ab? matches the string a or ab.
• +: The plus operator makes it possible to repeat the regular expression item.
  Example: ab+ matches every strings ab…b.
• *: The star operator after a regular expression item signifies that the item is optional and can be repeated.
  Example: ab* matches every string ab…b and a.
• |: The pipe operator specifies an alternative.
  Example: a|b matches the character a or b.
• (…): The parenthesis groups expressions into sub-expressions.
  Example: (true)|(false) matches the string true or false.
• Regular Expression Items: The regular expression items are ASCII or magic characters (See table tab:regexp_item for the complete list). If a magic character needs to be used as an ASCII character, it has to be escaped with a backslash (\).

Chapter 22  Code Coverage of Libraries

22.1  Code Coverage of Static/Shared Libraries and DLL

During the linking operation, CoverageScanner includes all instrumentations of the shared libraries (if these are compiled with CoverageScanner). CoverageBrowser displays the code coverage of the complete application (executable and its libraries) in one view.

22.2  Code Coverage of Plugins/Manually Loaded Shared Libraries

CoverageScanner cannot handle the instrumentation of plugins (i.e. shared libraries loaded manually) during the linking phase. In this case, the library must initialize and store itself the executions.
Therefore, the shared library needs to call __coveragescanner_filename to set the name of the execution file during its initialization and __coveragescanner_save to save the instrumentations when its becomes unloaded.

22.2.1  Code Coverage of Plugins Generated with Microsoft^® Visual Studio^®

The function DllMain is called on the initialization and the termination of the DLL generated using Microsoft^® Visual Studio^®. When the ’reason’ field is equal to DLL_PROCESS_ATTACH the function __coveragescanner_filename should be called. To save the measures on exit the function __coveragescanner_save shall be called when ’reason’ is DLL_PROCESS_DETACH.

Example:

22.2.2  Code Coverage of Plugins Generated with GNU gcc

The GNU compiler offers 2 attributes which permits to execute a function when a library becomes loaded or unloaded:

__attribute__ ((constructor)) my_init(void);

This attribute permits to call a function when the library is loaded. Call the function __coveragescanner_filename in the custom initialization function of the library.

__attribute__ ((destructor)) my_fini(void);

This attribute permits to call a function when the library is unloaded. Call the function __coveragescanner_save on the termination of the library.

Example:

Chapter 23  Test Suite and TestCocoon

23.1  Execution Name And Status

Test suites are able to provide to CoverageBrowser the name of the test and also the status of their execution (passed or failed). Two methods are possible:

• Using the functions __coveragescanner_testname and __coveragescanner_teststate can be used to specify the test name and its status. This is particularly useful for unit tests in which the test framework is compiled into the application to test.(see chap. executionreportname)
• Writing directly into the .csexe file this information. (see below)

The .csexe file is a simple text file on which is it possible to append additional lines to extend the information provided to CoverageBrowser.

To set the name of a test simply add before executing it the following line: §name of the test\n
or
*name of the test\n
The character * or § must be the first character of the line. The name of the test is placed directly after and a is terminated with a carriage return.

To set the status of a test simply add after executing it the following line: !status\n
The character ! must be the first character of the line. The status of the test is placed directly after and a is terminated with a carriage return. The status can be one of the following strings:

"PASSED"

indicates that the test was successfully executed.

"FAILED"

indicates that the test was not successfully passed.

"CHECK_MANUALLY"

indicates that it was not possible to determinate if the test was successfully executed.

Example: the following batch file execute the test First Test and set the status of the execution to CHECK_MANUALLY.

echo §First Test >> myapp.csexe
myapp
echo !CHECK_MANUALLY >> myapp.csexe

23.2  Unit Testing

CoverageBrowser import execution results for each object which are part of an application. This mean that, if an unit test uses the same objects as these for generating the application, CoverageBrowser is able to import the execution result of the unit test and merge it into the code coverage of the compiled application.

Example: 
An application is composed to 3 files:

app.cpp

This file contains the main() function of the application.

library.cpp

This file contains a set of functions called in app.cpp.

testlibrary.cpp

This contains the test code of the functions of the file library.cpp. testlibrary.cpp have its own main() function.

To generate the application:

cscl app.cpp /Foapp.obj
cscl library.cpp /Folibrary.obj
cscl library.obj app.obj /Feapp.exe

To generate the unit test:

cscl testlibrary.cpp /Fotestlibrary.obj
cscl library.obj testlibrary.obj /Fetestlibrary.exe

The execution of testlibrary.exe generates the execution report testlibrary.exe.csexe. This report can be loaded in the measurement database app.exe.csmes. In this case, only the code coverage analysis of the file library.cpp is loaded. (the coverage analysis of the file app.cpp indicates that no source line are executed).

23.3  Test Framework Support

23.3.1  CppUnit

CppUnit¹ is a unit test framework for C++. This environment can easily be adapted to get the code coverage from each unit test.
Simply proceed as follows:

1. Call __coveragebrowser_install() in the main() function.
2. Write a CppUnit TestListener class which records the code coverage report upon every unit test completion. The listener should set the name (using __coveragescanner_testname()) and clear the instrumentation (using __coveragescanner_clear()) before executing a test item (class member startTest()) to ensure to get only the coverage data of the concerned test. When an test item is executed, the instrumentation and the execution status should be saved (using __coveragescanner_teststate() and __coveragescanner_save()) in the class member endTest(). The class CoverageScannerListener give an implementation example.
3. Add this listener in the test manager of CppUnit (class CPPUNIT_NS::TestResult).
   In the example below

   CoverageScannerListener coveragescannerlistener;
   controller.addListener( &coveragescannerlistener );        
   

4. Compile the unit test using CoverageScanner.

To download: http://www.testcocoon.org/CppUnitListener.cpp

23.3.2  QTestLib

QTestLib² is a unit test framework for the Qt Library. This environment can easily be adapted to get the code coverage from each unit test.
Simply proceed as follows:

1. Call __coveragebrowser_install() in the main() function.
2. Write a QObject derived class called TestCoverageObject which records the code coverage report upon every unit test completion.
3. Instead of inheriting from QObject, make all your test cases inheriting from TestCoverageObject.
4. TestCoverageObject class provide its own init() and cleanup() slot, which use the CoverageScanner API to save the code coverage report. If these slots are also declared in the test case class, it is necessary to rename it in initTest() and cleanupTest().
5. Compile your project with code code coverage enabled.

TestCoverageObject header:

To download: http://www.testcocoon.org/testcoverageobject.h

TestCoverageObject source:

To download: http://www.testcocoon.org/testcoverageobject.cpp

23.3.3  CxxTest

CxxTest³ is a unit test framework for C++. This environment can easily be adapted to get the code coverage from each unit test.
Simply proceed as follows:

1. Call __coveragebrowser_install() in the main() function.
2. Create a CxxTest TestListener class called CoverageScannerListener by subclassing an existing listener (in the example bellow: ErrorPrinter). It will record the code coverage report upon every unit test completion. The listener should set the name (using __coveragescanner_testname()) and clear the instrumentation (using __coveragescanner_clear()) before executing a test item (class member enterTest()) to ensure to get only the coverage data of the concerned test. When an test item is executed, the instrumentation and the execution status should be saved (using __coveragescanner_teststate() and __coveragescanner_save()) in the class member leaveTest(). Finally, all test failure members of this class need to be reimplemented to record the test failure.
3. In the main() function call the run() function of CoverageScannerListener instead of CxxTest::ErrorPrinter().run().
4. Compile the unit test using CoverageScanner.

To download: http://www.testcocoon.org/CxxTestListener.cpp

Chapter 24  CoverageScanner Adaptation to a Tool Suite

CoverageScanner can be adapted to any compiler using a profile. This file is called ’<tool name>.cspro’ and must be placed in the TestCocoon installation directory. For each profile the CoverageScanner executable should be renamed to ’cs<tool name>’: in this case CoverageScanner directly loads the profile for ’<tool name>’. The command line option --cs-compiler makes it possible to explicitly to select a profile.


It is possible to rename the CoverageScanner executable to the same name as the native compiler¹. In this case CoverageScanner will work exactly like the native tool, and the code coverage analysis will only be activated when command line option --cs-on is used.

The profile permits CoverageScanner:

• to interpret the command line of the compiler (see table tab:profile_compiler).
• to interpret the command line of the linker (see table tab:profile_linker).
• to customize the output of the coverage analysis ² (see table tab:profile_io).

A profile file can contain:

Comments

comments must start with a # character at the beginning of a line and ends at the end of the line.

Global parameters

parameters are defined with the following syntax:
<item>=<value>
Only one definition per line is allowed.

Parameters for a specific architecture

parameters for a specific architecture are defined with the following syntax:
<item>[<architecture>]=<value>
Only one definition per line is allowed and the definition must be placed after the global parameter definition.

1

Example: rename coveragescanner.exe to cl.exe instead of cscl.exe.

2

This can be necessary on embedded systems which do not have access to a file system.