Automated UI Testing with Azure DevOps and Selenium

If you’ve never done automated UI testing of your web apps, you’re in for a real treat!  You can visit your pages, click buttons, verify results, take  image snapshots, all automatically through code.  Your code actually controls a web browser that interacts with your site.  I’ve recently rediscovered its benefits and am vowing to up my UI testing game.  Why?  Better code through testing.  I’m not necessarily all in on Test Driven Development, but I’m after a certain set of results.  I want my tests to eventually do my thinking for me.   I can’t possibly remember all the little things I want to validate in every web app I write (many of them).  All the business logic, widgets, rules,  and customized niche stuff.  Best of all, its super easy in your ASP.NET MVC apps using MS Test and Selenium Webdriver.   This post will walk you through setting up your test code in your web solution, then executing that test as part of a build definition in Azure DevOps (formerly Visual Studio Online/ Visual Studio Team Services).  This last part, executing the tests in the cloud, is the best part.   Its hands free, automatic, all web based.  The results are easy to track for someone who can be a little disorganized at times.  I have a complete history of each test execution stored online all in the same place I live anyway, in Azure DevOps.

Get Started Writing Your UI Tests

UI testing is a vast universe of many wonders of which I’m not going to cover for this post.  You will see the most basic of unit tests here.   First, in Visual Studio, add a new project to your web app solution.  In the left menu, select .Net Core if you’re building a .Net Core web app, select Test if you’re building for .Net Framework.  In the middle pane, select the Unit Test Project option.  You’re unit test project will be added.

In your unit test project, add the Selenium Web Driver, and Selenium Web Driver Chrome Driver (or Firefox) Nuget packages.  The Selenium main package is responsible for core UI testing while the browser driver packages interface with the browser of your choice.

Your first test class is added automatically to the project.   The test class itself is decorated with the TestClass attribute.  You also get an automatically created test method decorated with the TestMethod attribute.  Both are required to trigger the MSTest framework.  Some other handy attributes you can use will help you set up and tear down your tests.  These are the TestInitialize and TestCleanup attributes respectively.  Methods decorated with the TestInitialize attribute run before any tests are executed, whereas methods decorated with the TestCleanup attribute run after all tests are executed.

Initialize Selenium Webdriver with your app URL in your TestInitialize method, and dispose of it in your TestCleanup method, as shown below. You’ll need a class level variable of type IWebDriver named driver.  Super important: when running these UI tests in Azure DevOps you have to init the driver with Environment.GetEnvironmentVariable(“ChromeWebDriver”) so the hosted agent can load it.

The Login method instructs the web driver how to log into my web app. I simply reference the html elements by id and tell it what values to enter.

After your init method, all the test methods labeled with the TestMethod attribute will be run and tracked individually in Azure DevOps.  My test method simply shows how to navigate to a URL and take a snapshot.   This one of the coolest features.  I can look back in my test history to see when a page broke.   The snapshots are saved to the cloud under each test result.

I recommend running the tests locally first (remove the environment variable parameter passed to the Chrome Driver constructor). Which brings us to our next part.

Run Your Tests in the Cloud

To get started, you’ll need a build pipeline or a build and deployment pipeline, depending on how you want to configure things.  Most times I just have a build pipeline that also releases to at least staging.  Creating one is beyond the scope of this article.  Add a test assemblies step to your pipeline after whatever step deploys your site.  Its under the tests tab in the add step pane.

When added, the step looks like this.

The great thing about it is that the step is configured correctly with the defaults.  Make sure the upload test attachments option is checked, and that the select tests using step has test assemblies selected.

View Your Results

Run your pipeline.  Your tests will run.  The hosted Crome webdrdiver will go out and test your URL (if its publicly accessible).  There are two places to view your test results.  The first place is the test summary result, under the main left hand side menu :  Pipelines > Builds > click a build result > click the tests tab.  You’ll get a summary view like below.

The next is under the left hand side menu > Test Plans >  the sub menu Test Runs > click a test run.  You’ll get a run summary and a test results tab.  Under the test results tab is where you can view your screenshots.  Under the test results tab, you’ll have to double click a test result to view the details.

Hopefully now you’re off and running building your own UI tests.  Enjoy!

Calling the Visual Studio App Center API from Postman

Visual Studio App Center has a great API exposing most if not all of its functionality.  This is useful for a ton of reasons, whether you want to integrate its app management functionality into your workflow in a custom way, or as in my case, use it to access analytics and display custom dashboards.  You can of course export App Center Analytics data to blob storage or App Insights.  For my purposes, I only needed access to a few aspects of data and also a more fine grained control, so chose to just call the API.  The API is documented using Open API here.   This post will take you through getting all of the API endpoints imported into Postman and even generate code to start calling them. The tooling has certainly come a long way since building API clients by hand!

Get the API Swagger

In order to get the App Center raw Swagger, head on over to https://openapi.appcenter.ms.  Here you can view the interactive, nicely formatted UI where you can easily navigate and even issue test calls.  To get the actual raw swagger from this page, I had to view the page source, upon which I discovered a JavaScript function called on page load to go fetch it.  The URL looks to be https://api.appcenter.ms/preview/swagger.json.

Import the Swagger into Postman

You can import Swagger into Postman by clicking the Import button at the top of the app. From here you choose the source of the Swagger via file, folder, link or raw text. This is how I discovered that simply pasting the App Center Swagger URL did not work, so I posted the raw json from the link above and it created a new collection containing all of the endpoints. Most of the endpoints do not have a general description so they end up having no title in Postman, making it difficult to tell one from another. The good thing is you can add your own title to your favorite endpoints and save them in the collection.

Make your first call

To authenticate your call to the API from Postman, you need to obtain a token and (I had to dig around online for this one) add a key named X-API-Token to the header of the call with a value of the token you obtained from App Center.  Postman has a neat feature where you can store preset headers and add them to your calls, so I stored the token header as a preset so I can easily add them to new calls.

Generate client code

Now that you’re authenticated you can have Postman generate sample code for each endpoint. In my case I used the RestSharp code gen feature. To do that, click the code link (shown in the image above) while you’re in edit mode of a call and select your language. That’s it, you’re off and running!