Tag: NUnit

There are 2 ways to discover and execute unit tests using Microsoft developed test harnesses:

• Vstest.console.exe = This is the command-line used to execute tests within/embedded in Visual Studio IDE
• Dotnet.exe = This is the command line interface (CLI) specific to .Net Core Projects

Documentation for Vstest.console.exe is documented here: https://msdn.microsoft.com/en-us/library/jj155796.aspx

For .Net Core Projects: https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test?tabs=netcore2x

The primary difference between both is that vstest.console.exe can execute tests developed in .Net Framework and .Net Core while dotnet.exe is specifically for .Net Core

An example of executing tests for the same assembly domain (test project) would be:

VSTest.Console.exe:

vstest.console.exe <testassembly>.dll (Pointer to the compiled Assembly)

Dotnet.exe:

dotnet test <testassemblyproject>.csproj (Pointer to the actual .Net Core Test Project)

The issue with dotnet.exe (CLI) is that Code Coverage doesn’t work. In order for code coverage to work on .Net Core projects, you need to:

1. Edit the .Net Core projects you want to instrument for code coverage
2. Use vstest.console.exe and supply /EnableCodeCoverage switch

Edit the .Net Core project/s for code coverage instrumentation

When you run unit tests in visual studio and select the option to “Analyze Code Coverage for Selected Tests” (as seen below), by default, code coverage results will not be captured.

As of writing of this post, the fix is to modify the project file and enable DebugType to Full on the propertygroup section of the project file.

Save the project file and run the unit tests again by selecting the option: to “Analyze Code Coverage for Selected Tests” and you’ll see similar results as shown below.

Use vstest.console.exe and supply /EnableCodeCoverage switch

As you saw within Visual Studio, running tests with code coverage can be trigged via a simple click on the context menu. If you want to execute your unit test with code coverage in a command line, you invoke /EnableCodeCoverage switch.

vstest.console.exe <testassembly>.dll /EnableCodeCoverage

The result would be an export of the code coverage results to a .coverage file. You can then open the file within Visual Studio to inspect the results. See screenshot below:

Setting up your .Net Core projects appropriately using the preceding steps should give you the proper code coverage numbers. More importantly, this allows you to seamlessly integrate with various build systems. Additionally, here are some tips and practices around code coverage:

Use a test .runsettings

Use a test .runsettings file to exclude assemblies you don’t want to instrument. The .runsettings file can be used on how tests are executed from vstest.console.exe. For more information, see the following: Configure unit tests by using a .runsettings file

Here an example on how you would want to exclude piece of code not to be measured for code coverage:

<DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Include>  
                <!-- Include all loaded .dll assemblies -->  
              </Include> 
              <Exclude>
                <!-- Exclude all loaded .dll assemblies with the words moq, essentially regex -->
                <ModulePath>.*\\[^\\]*moq[^\\]*\.dll</ModulePath>
                <ModulePath>.*\\[^\\]*Moq[^\\]*\.dll</ModulePath>
              </Exclude>
            </ModulePaths>
            <!-- We recommend you do not change the following values: -->
            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>s
            <Attributes>
              <Exclude>
                <Attribute>^System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>
              </Exclude>
            </Attributes>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>

To use the .runsettings file, in Visual Studio, click on Test, Test Settings, Select Test Settings File (see below image)

[ExcludeFromCodeCoverage] attribute

Use [ExcludeFromCodeCoverage] attribute wherever appropriate. When a section of code is decorated with this attribute, that section of the code will be skipped for code coverage. Why? In certain cases, you don’t want code to be measured with code coverage. An example would be entity objects that have default property setters (get / set) that has no functionality. If there is “NO” logic developed on either the get and/or set property why measure it?

This ends the first part of this series, on the next part (VSTS Build Definition Setup – .Net Core and NUnit), we will hook up the test tasks in VSTS to include code coverage reporting.