Friday, April 13, 2012

Assigning Build to Test Plan in MTM

Introduction
So at this point you have your automation running using Microsoft Unit Testing Framework, you have set up your test environments, and you have test cases tied to your automation. So last how do we run the automated tests using MTM? This last step is you need to create a build, using the Microsoft Build Server. Then assign the test plan to this build.

Prerequisites
  • TFS Installed
  • MTM Installed
  • Familiar with Source Control and have a solution running
  • Have Microsoft Build Server installed and running.
Step 1 Add your solution to source control
1a. Go To Team Explorer, then click on Source Control. Click on the icon on the top menu.
1b. Select the your solution folder, and the location that you want to store it in source control. Then click Map.


1c. Check in all of your solution into source control. Right click on the solution in Solution Explorer, and
click on Check in.

Step 2 Create a new build
2a Go to Team Explorer, and left click on builds and click on New Build Definition
2b Go through and fill out each page of the build definition, making sure to associate the build to your project in source control under the Workspace section.
2c Save Build Definition

Step 3 Queue new Build
3a. Right click on the build definition under Builds in the Team Explorer and click Queue new build.

Step 4 Associate new build to Test Plan
4a. In MTM go to Testing Center - Plan and click Properties, Under the properties there is a builds section. Select the build you just added, and save the properties.


Step 5 Run automated test cases
With your test cases associated to the automation, and a build associated to the test plan, you are now ready to run the tests. Go to Testing Center - Test, select the automated test case and click on run. Within a minute it should start running one of your set up test agents.




Thursday, April 12, 2012

Download MormonOrgTest Source Code

Attached is my project source code for Webdriver/Microsoft Unit Testing Software

Using MTM as a load balancer to run test automation



Introduction

So now that you have created your 1,000 automated test cases, now what? Running a 1,000 test cases on one machine, is going to take a day or two. One of the great benefits of Microsoft Test Mamangement System (MTM) is that you can create a lab environment to run your tests on.

Prerequisites

I am assuming that you have a TFS server up and running, and are using MTM for running and tracking test cases.

Overview of TFS Lab Environment Set Up

Microsoft Test Manager (MTM) communicates to the Team Foundation Server (TFS), to get test cases, automation builds, and a list of Test Controllers. When running automated tests, MTM sends the tests to TFS. The Test Controller is on a timer to check the TFS Server for tests needed to be run, it then picks up these tests and waits for each of the test agents to check in, once each of the test agents check in, it delivers a portion of the tests to the test agent, then each test agent reports back the results of the tests it ran, including any files relating to the testing. After the test controller has collected the results of all the tests, it then delivers them to the TFS Server, where the results can be viewed using MTM.



Step 1 Install Test Controller

You are going to want to install a Test Controller on one of your lab machines. Note that the Test Controller can exist on the same machine as a Test Agent.

You download the controller and agent installs here:

When installing the controller you will need a domain user that has access to the TFS server, and has admin rights of the machine that the controller is installed on.

1a. Login as the Domain User that you have created to run the controller. (must be admin)
1b. Download the controller, you may need something to convert ISO to a physical file, so you can launch the setup.
1c. Install the Test Controller
1d. Configure Test Controller should launch after install.
1e. Use specific domain account to login to TFS and other Test Agents. Put in the user credentials of the domain user created in step 1a.
1e. Check "Register with Team Project Collection" and click on Browser add your TFS server location, and select your project.
1f. Click Apply Settings.



Step 2 Install Test Agents

For each lab machine, install the Test Agent on the machine. You will want to add the new domain user to each machine, login as that user on each machine, and install and configure the agent on each machine. To save time, you can copy the files that you expanded from the ISO image in step 1 to each machine, then run the set up from each machine.

1a. Login as the Domain User that you created. (must be admin rights)
1b. Download the Test Agent, copy files from machine in step 1.
1c. Install the Test Agent
1d. Configure Test Agent should launch after install.
1e. Use specific domain account you used to login to machine as.
1e. Check "Register with Test Controller" and put the name of the machine that is on the same domain, that contains the Test Controller
1f. Click Apply Settings.

Step 3 Add New Physical Environment

3a. Open Microsoft Test Management
3b. Navigate to Lab Center -Lab - Environments, Click on New, then Physical Environment.




3c. Fill out the name and description of the new environment, and set the location of the new controller you just added in step 1.


3d. Next select Machines (on the left menu), This page is set up into two planes, on the right is the available machines from the selected controller, and on the left is the machines added to the environment. Select each of the machines that you want to add to the environment. (that will be used for your automation), Set them all to type of Desktop Client. Then click finish.

Step 4 Associate Environment to Test Plan

4a. Navigate to the Testing Center - Plan - Properties, set the Test Environment to the environment you just created. Now any automated test ran in this test suite will run on the Test lab you just created.




Microsoft Unit Testing Framework - Using Test Context to determine test result

When creating your test automation, and you need to run a procedure when the test did not pass. Use the TestContext.CurrentTestOutcome property. This property will determine the result of the test in the Microsoft Unit Testing Framework.

In this following example I wanted to capture a screen shot only if the result had failed.



[TestCleanup]
public void CleanUpTest()
{
if(TestContext.CurrentTestOutcome!=UnitTestOutcome.Passed)
{
Tool.GetScreenShot();
Tool.SaveScreenShot();

}
 
}


The TestContext.CurrentTestOutcome property will reference a UnitTestOutcome enumerator. In the above example, if the test hadn't passed, then on clean up I will take a screen shot, and then save all screen shots to a result file. .

Saturday, April 7, 2012

Microsoft Unit Testing Framework - Using Test Context as a Static Variable

Using Test Context as a static variable
One of the problems I ran into when trying to capture a screen shot. I wanted to capture a screen shot only when an error happened. I wanted the screen shot to be called as part of Test Clean Up or Class Clean Up, I also wanted to add a screen shot right when I logged in, to make sure the screen looked normal at the beginning of the class.

If you are using your initialize and clean up attributes correctly, you probably created a class file that contains all of the logic for starting and ending a test. In this class file if you add the Test Context Initialize sequence:
private TestContext m_testContext;javascript:void(0)
        public TestContext TestContext
        {
            get { return m_testContext; }
            set { m_testContext = value; }
        }

Then add this to the Assembly Initialize:

Screenshot ss = Driver.GetScreenshot()
            String fileNames = Environment.GetEnvironmentVariable("TEMP") + "\\TempFile" + testName + DateTime.Now.ToString("MMddHHmmss") + ".Png");
            ss.SaveAsFile(fileNames, ImageFormat.Png);
            TestContext.AddResultFile();


The Problem
You will get an error on TestContext.AddResultFile(): "An object reference required for the non-static field, method, or property"

Which poses as a big bummer. So does this mean I can't use Test Context for uploading a file on Assembly initialize? No.

Solution
So here's what I did.
First create a tool class if you haven't already that keeps all of the supporting framework methods for your automation. This class should be a static class. In this static class create a two methods 1. Get Screen Shot, 2. Save Screen Shot. The first method will capture the screen shots, and save the file names in a static collection. The second will then use Text Context to go through and upload each of the files contained in the collection and add to the result file. The first method I will call every time I need a screen shot, the second method I will call only on Clean Up attribute at the end to add the files to the result file.


Step 1: Add Static Test Context to Base File
If you have a file that you use for all of your static variables like Driver, Website address, or db connection. Add these lines to the file.

private static TestContext testContextInstance;

public static TestContext TestContext
        {
            get
            {
                return testContextInstance;
            }
            set
            {
                testContextInstance = value;
            }
        }


This will create a static accessible Test Context.

Step 2: Initialize Test Context
On [AssemblyInitialize] include TestContext testContext in the call, then assign testContext to the Static Test Context in the class.

for example:
[AssemblyInitialize]
        public static void SetUp(TestContext testContext)
        {
            TestContext = testContext;
        }


Step 3: Create Method in Tool

The following code will create a screen shot using the Web Driver Framework, save the file in a TEMP directory, and Store the full file name in a static string collection.

First: Create a static collection for the filename:

private static List Filenames = new List();


Second: Create a method to capture screen shot:

public static void GetScreenShot()
{
string testName = Base.TestContext.TestName;
Screenshot ss = Base.Driver.GetScreenshot();
Filenames.Add( Environment.GetEnvironmentVariable("TEMP")+"\\TempFile"+ testName + DateTime.Now.ToString("MMddHHmmss") + ".Png");
ss.SaveAsFile(Filenames[(Filenames.Count-1)], ImageFormat.Png);
}


(Note you will need references: System.Drawing.Imaging; OpenQA.Selenium; System.Collection.Generic; System.Collection.ObjectModel; )

(Additional Note: Notice that we are accessing the static TestContext in the base class to the get the TestName, and include it as part of the file name. )


Step 4: Add Screen Shots to Result File
Now we have to create one more tool method that will add the file to the result set. This method will go through each of the full file names in the static file name collection, and save each file, then delete the temp file.

public static void SaveScreenShot()
        {
 foreach(string filename in Filenames)
            {
                Base.TestContext.AddResultFile(filename);
                File.Delete(filename);
            }
            Filenames.Clear();
         }


Step 5: Call methods

In the TestCleanup add the following:
[TestCleanup]
        public void CleanUp()
        {
      
            Tool.SaveScreenShot();
           
            
        }


Next Add the following to any place you want to grab a screen shot
For Example here I am getting a screen shot before I sign out of a test.

[TestCleanup]
        public void CleanUp()
        { 
            Tool.GetScreenShot();
            Tool.SaveScreenShot();
            
            _nav.SignOut();
            
        }