This post walks you through on how to implement continuous testing using the following technologies:
- Using Config Transform to configure your tests to run multiple browsers during runtime
- Selenium WebDriver – Automation UX framework to drive UX Testing
- VSO Build – ALM (Application Lifecycle Management) tool suite for storing code (via GIT), creating the build definition (Build) and configuring On-Premise machines as Test Agents
What is the Config transform? This will allow you to change the configuration settings for your app or web configuration files. When you use Selenium WebDriver, you have the option to run your tests using either Chrome, Internet Explorer, FireFox even PhantomJS (Headless testing). See the sample C# code below to generate the correct web driver instance through configuration settings. The method takes in a string value with a default browser of Internet Explorer. Note that sample code below uses try catches to log exception using Log4Net. You can ignore this but just re-use the code for creating the proper Selenium WebDriver
NOTE: If you’re not familiar on using Selenium to run automated UX testing, see these samples below:http://docs.seleniumhq.org/docs/03_webdriver.jsp
public static T OpenBrowser<T>(string browser = "iexplore", bool useExistingBrowser = false, bool usebrowserstack=false) { string error = string.Empty; try { if (!string.IsNullOrEmpty(browser)) { browser = browser.ToLower(); switch (browser) { case "firefox": IsFirefox = true; return (T)Convert.ChangeType(ReturnDriver<FirefoxDriver, FirefoxProfile>(useExistingBrowser, ref _firFoxDriver, Firefoxprofile), typeof(FirefoxDriver)); case "chrome": IsChrome = true; return (T)Convert.ChangeType(ReturnDriver<ChromeDriver, ChromeOptions>(useExistingBrowser, ref _chromeDriver, Chromeprofile), typeof(ChromeDriver)); case "headless": return (T)Convert.ChangeType(ReturnDriver<PhantomJSDriver, PhantomJSOptions>(useExistingBrowser, ref _phantomjsdriver, phantomjsoptions), typeof(PhantomJSDriver)); } } // ie (internet explorer) IEprofile.IgnoreZoomLevel = true; IEprofile.EnsureCleanSession = true; IsIE = true; return (T)Convert.ChangeType(ReturnDriver<InternetExplorerDriver, InternetExplorerOptions>(useExistingBrowser, ref _internetDriver, IEprofile), typeof(InternetExplorerDriver)); } catch (Exception ex) { if (string.IsNullOrWhiteSpace(ex.Message)) _Testlog.Info(MethodBase.GetCurrentMethod().Name + " Results = Success"); else _Testlog.Info(MethodBase.GetCurrentMethod().Name + " Results = " + ex.Message); throw new ArgumentException(ex.Message, ex.InnerException); } return default(T); }
The Generic Type to return the driver is:
NOTE: I’ve hardcoded the value of T as IWebDriver instance to maximize the browser. You don’t have to do this but since we’re using Selenium WebDriver, I’ll just embed it on this method. You can also change the T type to any UX automation.
private static T ReturnDriver<T, TT>(bool existing, ref T driver, TT profile) { string error = string.Empty; try { if (driver == null || existing == false) { driver = (T)Activator.CreateInstance(typeof(T), profile); } ((IWebDriver)driver).Manage().Window.Maximize(); } catch (Exception ex) { if (string.IsNullOrWhiteSpace(ex.Message)) _Testlog.Info(MethodBase.GetCurrentMethod().Name + " Results = Success"); else _Testlog.Info(MethodBase.GetCurrentMethod().Name + " Results = " + ex.Message); throw new ArgumentException(ex.Message, ex.InnerException); } return driver; }
This method takes in a string param. You can pass the value from the config file and if you use config transforms, you can change the run behavior of the browser just by passing in the correct app config value.
Almost forgot! Download the config transform here: Configuration Transform
https://visualstudiogallery.msdn.microsoft.com/579d3a78-3bdd-497c-bc21-aa6e6abbc859
Here’s a look at what the config transform would look like in your project:
The particular key that I used and configured would look very similar to this (this is from, App.Chrome.config):
<!-- valid options for Browsers are: chrome , firefox, iexplore, headless --> <add key="Browser" value="chrome" xdt:Transform="Replace" xdt:Locator="Match(key)" />
Source Control is GIT in VSO
Given that we’re using VSO as our ALM suite, we’ve opted to use GIT as our backend source control system. This make it’s easier for configuring your build definition since all source code is stored in VSO
Configuring On-Premise VSO Test Agents
The next step is to ensure that you have On-Premise Test Agents that VSO can talk to and execute the tests. For this follow the steps on this article to configure your VSO Test Agents. Note that in VSO, build and test agents are now in the same machine. Also, note that this article talk about On-Premise TFS HOWEVER, the same applies in VSO. You have to go to your VSO instance (https://xxxx.visualstudio.com) and configure your agent pools. The rest is shown belowhttps://msdn.microsoft.com/Library/vs/alm/Build/agents/windows
Let’s do a quick check!
- Automated UX Tests have been developed in and uses configuration settings to drive the browser driver – Check!
- Installed Configuration Transform and configured my test project with all appropriate config settings – Check!
- Test Code stored in GIT (VSO) – Check!
- On-Premise VSO Test Agents Configured – Check!
Once all of these have been verified then the final step would be stitching and putting all of these together via VSO Build.
Configuring VSO Build for Continuous Testing:
Step 1: Configure Build Variables:
Step 2: Create Visual Studio Build step:
First step is to build the test solution/project. This is pretty straightforward. On the Solution textbox, browse through the .sln file that you’ve checked in source control (in this case GIT)
Step 3: Deploy the Test Agent on the On-Premise Test Machines:
NOTE: Before completing this step, ensure that you’ve properly configured your test machines. Follow the article below. This articles walks you through creating machines groups for your team project:
The key here is ensuring that:
· You have 1 test machine group that has all the test agents configured correctly
· The $(TestUserName) must have admin privileges on the test agents
Step 4: Copy The Compiled Test Assemblies To The On-Premise Test Agents:
The key to this step is ensure that you copy the compiled test assemblies to the right location. $(Build.Repository.LocalPath) is the directory from the build server where “Destination Folder” will copy the test assemblies to the target test agent machine.
Step 5: Execute the Tests:
Nothing special here. Just make sure that you’re reference the correct Test Drop Location. Just copy the Destination Folder from the previous step:
If you configured it correctly, you should get a successful build! Now the result of the build depends on the Pass/Fail results of the tests. In this case, I’ve intentionally Failed 1 automated test to trigger a build failure that coincides to a failing test. Passing the fail test in the future will result to a complete build
In your VSO Home or Team (Dashboard) Pages, you can pin the build trending charts to see PASS/FAIL Patterns for your UX Automation
DTan's Blog 