As of writing of this post, I prefer NUnit when using .Net for developing application deployment tests. Why? Simply put, better support and test execution. More importantly, the ability to simply reference test runners without installing the actual test execution files (Like VSTest or MSTest). This guide will walk you through on developing simple tests via NUnit, build the artifacts then executing those tests on the server where the application is being deployed to.
NOTE: NUnit and MSTest are completely separate testing frameworks. You can mix/reference both NUnit and MSTest Test frameworks in the same VS Test Project and Tests (Test Methods) without duplicating the same tests. This however is a separate post.
Backing a bit, we use VSTS as our primary adoption platform for Continuous Integration and Deployment. In short, CI through build definitions and CD through release management. We have standards within our processes and one of those standards is to ensure that any application being deployed has the appropriate application deployment tests. Nope, I’m not talking about smoke tests where an application is being verified “after” its been deployed through a UI tests or simply some high level scenarios. I refer to “application deployments tests” as tests developed to be run specifically on the server where the application is being deployed to. This mostly maps to any application being deployed and hosted to individual servers (IIS hosted machines if you’re a Windows shop)
Let’s begin.
1) Develop NUnit Tests to validate application deployment.
Before proceeding, you need to ensure that you at least have one application deployment tests. The number of tests vary on the application that you want to verify. In my scenario, I’m deploying a Web API so one of the tests to validate is whether a web method works after deployment.
Here, we’re going to start off with an existing Visual Studio Test Project. However, If this is your first time (as a devops person), feel free to go through this guide, “Create a unit test project”
Reference NUnit assemblies in your test project.
Expand your Test Project, right click on “References” and click on “Manage NuGet Packages”. The nuget package explorer window opens and type in “Nunit” under the search box within the “Browse” tab.
NOTE: If you want to see your NUnit Tests within Visual Studio, you’ll need to install the NUnit Test Adapter within Visual Studio, “Tools”, “Extensions and Updates”
Decorate your code with the NUnit Test Attributes
The goal for running tests is to ensure that the app has been deployed properly. In these situations, tests would then normally target or construct the URL based on the IP Address or the HostName of the sever then the application URL. This would be something that you’ll need to plan on how to build your application URLs correctly. There are many ways to do this and I’ll creating a separate blog post for this.
Once you’ve validated your tests targeting the right server, we then package the tests as artifacts to be executed on the server. For this, see the next step.
2) Package your tests files as build artifacts to be consumed in VSTS Release Management
In normal CI processes, your tests files should be coupled together with your application deployment files. In an event that your tests files are not packages together with your application deployments files, follow these steps:
Start off by creating a build definition in VSTS. For this, follow this guide. “Build and deploy your app”.
Build your Test Solution with the Test Project
Publish your Test Artifacts/Files
NOTE: Very Important! Make sure your Test Files includes the NUnit Console Runner assemblies and executable. There are many ways to reference the Nuget Test Runner files but for the purpose of this demo, ensure that your Test artifacts include all test runner files. Here’s a snipped of such files.
3) Modify your release tasks to include running application deployment tests via NUnit
In your existing release definition:
Add the build artifacts (test artifacts) as part of your Release Definition
Add a Command Line Task to Run NUnit Tests
And the result…
DTan's Blog 