Capturing Screen Shot and Uploading to Result File in Microsoft Unit Testing Framework (MUTF)

One of the great benefits to Microsoft Unit Test Framework, is it allows you to attach files to the be delivered with the result file. This could be a log file, a system file, or in this example a screen shot.

To create the screen shot I am using the tool built directly into WebDriver.

The code is:

Screenshot ss = Base.Driver.GetScreenshot();
ss.SaveAsFile("TempFileLocation", ImageFormat.Png);

This is the code I used, notice I added the GetEnvironmentVariable to grab the Temp directory of the computer, and I added the Date and Time to the file name to create a unique instance of the file. You will want to keep the filename in a variable so you can reuse later on. (

Screenshot ss = Base.Driver.GetScreenshot();
String Filename = Environment.GetEnvironmentVariable("TEMP")+"\\TempFile"+ testName + DateTime.Now.ToString("MMddHHmmss") + ".Png");
ss.SaveAsFile(Filenames[(Filenames.Count-1)], ImageFormat.Png);


To include the file in the result set you will need to instantiate TestContext, then call:

TestContext.AddResultFile("locationAndNameOfFile");

In the example above I would use the code

TestContext.AddResultFile(Filename);

This will add the file to the result set.

Unit Testing with Test Cases in MTM

In Microsoft Test Manager, you can store test cases that contain steps, and parameters for testing. One of the great advantages to using Microsoft Unit Testing Framework, is you can have a test look at the parameters of a test case, and use those parameters to run automated tests. This way all of the critical information is stored between the test case and the automation.

First start by creating a Test Case in Microsoft Test Manager. In the test case outline the simple steps that your automation will be following. Example:

Notice in the above image I have identified each of the steps in my test. I have also identified the parameters I am going to use for my testing, and the validating parameters needed to assert if the test passes or fails. The parameters are preceded with an @ symbol in the steps section, and for each parameter identified in the steps section, there is a correlating column in the parameter section of the test case. In this section I have put in all of the different variations that I want to test for. In this example I am testing a search function which has 3 different filters Age, Gender, Ethnicity, and also a search text box. I am testing each of the different fields by running through a series of tests.

Now that the test case has all of the parameters, and scenarios identified we now create the automation needed to run the scenarios. When creating the automation you will need to have the following.

1. Make sure your test class is using the using the reference Microsoft.VisualStudio.TestTools.UnitTesting.

2. Instantiate an instance of TestContext. Example:

private TestContext m_testContext;     
public TestContext TestContext    
{        
get { return m_testContext; }        
set { m_testContext = value; }  
}

2. On or with your [TestMethod] add a DataSource() attribute. Template:

[TestMethod]
[DataSource( "Microsoft.VisualStudio.TestTools.DataSource.TestCase", ";", "", Microsoft.VisualStudio.TestTools.UnitTesting.DataAccessMethod.Sequential)]

Example:

[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.TestCase",
"http://icstfs.blogspot.org:8080/tfs/blogspot;Quality Management","77300",
Microsoft.VisualStudio.TestTools.UnitTesting.DataAccessMethod.Sequential)]

3. Call the parameters from the test case into your method using TestContext.DataRow[].ToString();

For example:

  public void SearchPerson()
{
string gender = TestContext.DataRow["Gender"].ToString();
string age = TestContext.DataRow["Age"].ToString();
string ethnicity = TestContext.DataRow["Ethnicity"].ToString();
string searchText = TestContext.DataRow["SearchText"].ToString();
string validPerson = TestContext.DataRow["ValidPerson"].ToString();
string invalidPerson = TestContext.DataRow["InvalidPerson"].ToString();
string validDescription = TestContext.DataRow["ValidDescription"].ToString();
 if(!_meetMormonFilter.IsMeetMormonsVisible())
   {              
     _menu.GoToHome(); //If Meet Mormon wigit isn't visible go to home page                }
   }      

 _meetMormonFilter.FindPeople(gender, age, ethnicity, searchText); //Find a person using the wigit
_peopleSearchResults.ViewByListClick(); //View the search to list to see names and descriptions

Assert.IsTrue(_peopleSearchResults.ListFinderHasPerson(validPerson),"Person: "+validPerson+" Not found in grid"); //Check person name is in grid
Assert.IsFalse(_peopleSearchResults.ListFinderHasPerson(invalidPerson),"Person: "+invalidPerson+" Is found in grid and should not be"); //Check a invalid person is not in grid

string description = _peopleSearchResults.GetProfilDescription(validPerson); //Get Description associated with Valid User
Assert.IsTrue(description.Contains(validDescription), "Description for Person: " + validPerson + " did not contain phrase: "+validDescription);//Check that description contains the text stored in Valid Description
_peopleSearchResults.ListFinderSelectProfile(validPerson); //Go to the User Profile     
}

Finally with the automation linked to the test case, visual studio will run the test once for each row of parameters that you have. After the test is done running you will see one test that has failed or passed in the Test Result. However if you double click on the test result, you will be able to see all of the tests ran.

Unit Testing with Test Cases in MTM

In Microsoft Test Manager, you can store test cases that contain steps, and parameters for testing. One of the great advantages to using Microsoft Unit Testing Framework, is you can have a test look at the parameters of a test case, and use those parameters to run automated tests. This way all of the critical information is stored between the test case and the automation.

First start by creating a Test Case in Microsoft Test Manager. In the test case outline the simple steps that your automation will be following. Example:

Notice in the above image I have identified each of the steps in my test. I have also identified the parameters I am going to use for my testing, and the validating parameters needed to assert if the test passes or fails. The parameters are preceded with an @ symbol in the steps section, and for each parameter identified in the steps section, there is a correlating column in the parameter section of the test case. In this section I have put in all of the different variations that I want to test for. In this example I am testing a search function which has 3 different filters Age, Gender, Ethnicity, and also a search text box. I am testing each of the different fields by running through a series of tests.

Now that the test case has all of the parameters, and scenarios identified we now create the automation needed to run the scenarios. When creating the automation you will need to have the following.

1. Make sure your test class is using the using the reference Microsoft.VisualStudio.TestTools.UnitTesting.

2. Instantiate an instance of TestContext. Example:

private TestContext m_testContext;

public TestContext TestContext

{

get { return m_testContext; }

set { m_testContext = value; }

}

2. On or with your [TestMethod] add a DataSource() attribute. Template:

Example:

[TestMethod]

[DataSource(

"Microsoft.VisualStudio.TestTools.DataSource.TestCase",

"http://icstfs.blogspot.org:8080/tfs/blogspot;Quality Management", //TFS Server

"77300", //Test Case ID

Microsoft.VisualStudio.TestTools.UnitTesting.DataAccessMethod.Sequential)]

3. Call the parameters from the test case into your method using TestContext.DataRow[].ToString(); For example:

public void SearchPerson()

{

string gender = TestContext.DataRow["Gender"].ToString();

string age = TestContext.DataRow["Age"].ToString();

string ethnicity = TestContext.DataRow["Ethnicity"].ToString();

string searchText = TestContext.DataRow["SearchText"].ToString();

string validPerson = TestContext.DataRow["ValidPerson"].ToString();

string invalidPerson = TestContext.DataRow["InvalidPerson"].ToString();

string validDescription = TestContext.DataRow["ValidDescription"].ToString();

if(!_meetMormonFilter.IsMeetMormonsVisible())

{

_menu.GoToHome(); //If Meet Mormon wigit isn't visible go to home page

}

_meetMormonFilter.FindPeople(gender, age, ethnicity, searchText); //Find a person using the wigit

_peopleSearchResults.ViewByListClick(); //View the search to list to see names and descriptions

Assert.IsTrue(_peopleSearchResults.ListFinderHasPerson(validPerson),"Person: "+validPerson+" Not found in grid"); //Check person name is in grid

Assert.IsFalse(_peopleSearchResults.ListFinderHasPerson(invalidPerson),"Person: "+invalidPerson+" Is found in grid and should not be"); //Check a invalid person is not in grid

string description = _peopleSearchResults.GetProfilDescription(validPerson); //Get Description associated with Valid User

Assert.IsTrue(description.Contains(validDescription), "Description for Person: " + validPerson + " did not contain phrase: "+validDescription);//Check that description contains the text stored in Valid Description

_peopleSearchResults.ListFinderSelectProfile(validPerson); //Go to the User Profile

}

Finally with the automation linked to the test case, visual studio will run the test once for each row of parameters that you have. After the test is done running you will see one test that has failed or passed in the Test Result. However if you double click on the test result, you will be able to see all of the tests ran.

Web Driver with TFS MTM

In open source software testing, you typically have two components in your supporting frameworks, you have your assertion framework, and your testing framework.

Your assertion framework is the framework responsible for identifying, executing, and reporting the results of the executed tests with in a class project. (ie. nunit, junit, MUTF, MBunit)

Your testing framework, is the framework that is used to perform the actual automation, the interaction with computer, or server. (ie. WaTiN, Web Driver, Selenium, UI Test)

Microsoft MTM has a very good and robust solution for testing software, however the testing framework: UI Test lacks in simplicity and maintainability when automating web applications. For some of us this lack of functionality causes us to abandon the MTM solution.

There is another solution however, ditch the UI Test, but keep the Microsoft Unit Testing Framework (MUTF). MUTF has some great features and functionality that can work with just about any open source framework and will still allow us to take advantage of all the great functionality in MTM.

Web Driver is slowly becoming the most popular framework in the QA industry. Being partially supported by the brains at Google, Web Driver has become more versatile, stable, and faster than any opensource framework out there, even beating out many commercial solutions.

The trick to creating a Web Driver/MTM solution is understanding all of the benefits of the Unit Testing Framework and using those benefits to work with MTM.

Benefits of Microsoft Test Management Solution:

• Supports Load Balancing, allowing you to run your suite of 1000 tests, across multiple machines cutting down on run time.
• Supports video capture of tests, aiding in the troubleshooting of failed tests.
• Tight Integration with Test Cases, you can associate your automation directly to a test case, allowing you to run the automation for that test case any time, and also allowing you to store your parameters for running the automation in the Test Case.
• Upload additional files with your results (you can upload a variety of files that can be stored with your test results, I often use this feature to capture screen shots on failures
• History of automation results. (I can look at a test and see the history of pass/fail for every time that test has been run)