Category Archives: Unit Testing

Running .Net 4.6 Unit tests using vstest.console or TFS builds

Published January 20, 2016 9:29 pm

Unit tests can be authored in VS 2015 and can be targeted to various framework versions e.g. 4.0, 4.5 using the project Target Framework settings. When the framework is set to 4.6 or 4.6.1, you may face a known issue with the vstest.console.exe command line test runner or within TFS (Team foundation server) Builds.

When the tests built using fx >= 4.6 and executed along with another unit test dll that is built with framework version < 4.0 – the runner (vstest.console.exe) is unable to find any tests in the dll. This is because it uses incorrect version of framework in the test host process to discover the tests and hence does not find any tests.

The error may look like this and no tests are discovered.

D:\git\myproj\NetFx46Issue\NetFx46Issue\bin\Debug>c:vstest.console NetFx46Issue.dll Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine 
as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true

In addition, there may be another error if .net frameworks 2.0/3.5 is not installed on the machine and dialogs may popup to install .Net Frameworks 2.0/3.5 windows feature until the command line is aborted with CTRL+C. Capture

Error: The active Test Run was aborted because the execution process exited unexpectedly. To investigate further, enable local crash dumps either at the machine level or for process te.processhost.managed.clr20.exe. Go to more details: http://go.microsoft.com/fwlink/?linkid=232477

The error in XamlBuild or Build vNext environment in a TFS 2013/2015 can be on these lines depending on whether frameworks 2.0/3.5 is installed on the build agent machine or not.

The workaround for this is to use /framework:framework40 switch with vstest.console.exe command line.

D:\git\myproj\NetFx46Issue\NetFx46Issue\bin\Debug>c:vstest.console.exe NetFx46Issue.dll Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll /framework:framework40
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Passed   TestMethod1

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 13.1705 Seconds

For Xaml build definition, TargetFrameworkVersion=Framework40 setting needs to be passed using a .runsettings file.

  1. Download TargetFrameworkVersion40.runsettings here.
  2. This setting file (TargetFrameworkVersion40.runsettings) needs to be committed to source control along side the test code
  3. Edit the build definition to specify runsettings for the automated test run. Refer msdn for details.
  4. “Source Settings” for the build definition must ensure that the TargetFrameworkVersion40.runsettings file is synced up on the build agent.

For build vnext definition also the steps will be same except the step (3). Runsettings can be specified through the web interface to edit “Visual Studio Test” task’s settings.

The issue will be addressed in VS 2015 Update 2.

vstest.console.exe /logger

Published December 25, 2015 11:00 am

If you use vstest.console.exe, you are user of unit testing in Visual Studio. If you use it with /logger:TfsPublisher – you qualify as advanced user of vstest.console.exe. /logger switch lets you publish the results to the tfs server against a build & test runs show up on the build page.

// publishes results to the build 
vstest.console.exe mytests.dll /logger:TfsPublisher;Collection=http://mytfsserver:8080/tfs/DefaultCollection/;BuildName=MyBuild_20151222.1;TeamProject=myproj

It can publish results to either XAML Builds or new builds (tfs 2015). To publish test results from within TFS build 2015 – build definition enables to add “Publish test results” task out of the box. It can be used to publish the TRX file produced by other tasks during the build. It is not recommended to use /logger:tfspublisher from within a build. While it can be made to work, it may have pitfalls.

  1. The build runs in the context of a service or user based on how the build agent is configured. vstest.console.exe  may not have access to the tfs server.
  2. The build number in the TFS 2015 – is by default of YYYYMMDD.N format – in the new build definition. It must be prefixed e.g. MyBuild_YYYYMMDD.N when editing the build definition. Otherwise – two build definitions may have the same build number & confuse the publisher. publishing may fail with error (Build 20151222.1 cannot be found under team project myproj).