TFS on premise or Visual Studio Team Services (VSTS) - I use them interchangeably from now on - provide a build task to run VSTest. This task optionally allows one to enable code coverage. However, this feature requires one to have a Visual Studio Enterprise license / installation on the build agent.
Luckily, as so many times, there are open source alternatives available. I’m sure their are more, but I’m going with OpenCover. Mostly, because I have prior experience with it.
As OpenCover works, it starts the actual test runner (vstest.console.exe
for VSTest) and by doing that it can sort of hook into the actual test code being called and thus figure out coverage information - now, that is a pretty non technical explanation here, but you could find out more looking at OpenCover’s source or docs. The important point here is that, you have to run your test runner from OpenCover.
Why is that an issue? Well, because a potential build task for OpenCover would have to no only provide the necessary options to configure and run OpenCover itself, but would have to sort of replicate all the options and features that the VSTest build task already has.
If you look at the code there is a lot that is done there.
This is the main reason why I refrained from create such a build task for some time now. However, circumstances caused me to change my mind and a give it try.
At work we wanted to integrate OpenCover into our build process, but preferably not run tests twice (once with the stock test task and then once again with the OpenCover task just to gather coverage information). That meant, that OpenCover task would have to emulate as much as possible from the VSTest task. I finally settled on the features of the version task that was shipped with TFS 2017 Update 1. The code is a lot easier (as in smaller) as for the 2.x version of the task. As said, the OpenCover task emulates most of the VSTest 1.x settings, currently excluding the “Run in parallel” feature - but work is not done yet and I might add it in the time coming. UPDATE: 7th Sept: Parallel execution of test cases is now supported.
You can find the results (i.e. code, etc.) right here, along with instructions on how to upload it into your environment. (Many thanks to Miłosz Piechocki and his informative article, that gave a good starting point, integrating OpenCover using a TFS PowerShell task with a script.)
The configuration settings look somewhat like this (depending on the exact TFS version or iteration of Visual Studio Team Services):
The result in the build summary looks like this:
Additionally, you can download a ZIP file containing a report generated by ReportGenerator during the run.