Running .Net 4.6 Unit tests using vstest.console or TFS builds

Published January 20, 2016 9:29 pm

Unit tests can be authored in VS 2015 and can be targeted to various framework versions e.g. 4.0, 4.5 using the project Target Framework settings. When the framework is set to 4.6 or 4.6.1, you may face a known issue with the vstest.console.exe command line test runner or within TFS (Team foundation server) Builds.

When the tests built using fx >= 4.6 and executed along with another unit test dll that is built with framework version < 4.0 – the runner (vstest.console.exe) is unable to find any tests in the dll. This is because it uses incorrect version of framework in the test host process to discover the tests and hence does not find any tests.

The error may look like this and no tests are discovered.

D:\git\myproj\NetFx46Issue\NetFx46Issue\bin\Debug>c:vstest.console NetFx46Issue.dll Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine 
as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true

In addition, there may be another error if .net frameworks 2.0/3.5 is not installed on the machine and dialogs may popup to install .Net Frameworks 2.0/3.5 windows feature until the command line is aborted with CTRL+C. Capture

Error: The active Test Run was aborted because the execution process exited unexpectedly. To investigate further, enable local crash dumps either at the machine level or for process te.processhost.managed.clr20.exe. Go to more details: http://go.microsoft.com/fwlink/?linkid=232477

The error in XamlBuild or Build vNext environment in a TFS 2013/2015 can be on these lines depending on whether frameworks 2.0/3.5 is installed on the build agent machine or not.

The workaround for this is to use /framework:framework40 switch with vstest.console.exe command line.

D:\git\myproj\NetFx46Issue\NetFx46Issue\bin\Debug>c:vstest.console.exe NetFx46Issue.dll Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll /framework:framework40
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Passed   TestMethod1

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 13.1705 Seconds

For Xaml build definition, TargetFrameworkVersion=Framework40 setting needs to be passed using a .runsettings file.

  1. Download TargetFrameworkVersion40.runsettings here.
  2. This setting file (TargetFrameworkVersion40.runsettings) needs to be committed to source control along side the test code
  3. Edit the build definition to specify runsettings for the automated test run. Refer msdn for details.
  4. “Source Settings” for the build definition must ensure that the TargetFrameworkVersion40.runsettings file is synced up on the build agent.

For build vnext definition also the steps will be same except the step (3). Runsettings can be specified through the web interface to edit “Visual Studio Test” task’s settings.

The issue will be addressed in VS 2015 Update 2.