After more than three months it is time for a new article part of the most successful Automate The Planet's series- Design Patterns in Automated Testing. In the next few publications, I will try to solve a major problem in automated testing in my belief- the tests' failure analysis. Usually, the errors displayed when a test fails a far from the real reason why it fails. In this first part, I am going to explain to you my first approach to the problem using the Chain of Responsibility Design Pattern.

Abstract UML Class Diagram

Participants The classes and objects participating in the original Chain of Responsibility Design Pattern are: Handler -defines an interface for handling requests. Contains the HandleRequest method.

-defines an interface for handling requests. Contains the method. ConcreteHandler - Holds the actual logic for handling a request. It has access to its successor. If it cannot manage the request itself, passes the execution to its successor.

- Holds the actual logic for handling a request. It has access to its successor. If it cannot manage the request itself, passes the execution to its successor. Client - initialize the concrete handlers' chain and initiates the request. What Are the Problems That We Try to Solve? As I already mentioned at the beginning of the article, the most time-consuming activity in automated testing after the code writing is the tests' failure analysis. It is possible in the long run this to be even the most time consuming depending on the complexity of the tests' framework and the code quality. When the tests' failure is more accurate and understandable for people who didn't write the tests, I believe the required time for maintenance will be decreased. Chain of Responsibility Design Pattern for Failed Tests Analysis My main idea here is that almost in 99% of the cases the UI tests fail because some element is not found. So we can plug-in the analysis engine at this exact point when we catch these exceptions. I decided that the Chain of Responsibility Design Pattern is a good fit for the job because we need an algorithm similar to exception handling in C#. We start the analysis from the most common reason causing failures and if the check doesn't succeed we continue with more uncommon tests. Such check can be for example 'yellow exception screen', file not found, page not found and so on. These checks can be shared for multiple applications or can be strictly specific for your current case. I developed the idea even further with the introduction of test case specific handlers where you can plug-in unique handlers only for a particular action on the page. For example, if you click a button, and a validation message is displayed instead of redirecting to another page. Usually, the test will fail because some element on the second page cannot be found. Instead, you can add custom test case handler which can check for the validation message if the element location logic fails. Handler

The main class in this implementation is the abstract Handler. As explained in the abstract UML class diagram it defines an interface for the concrete handlers. Moreover, it contains the implementation of the HandleRequest method. It is modified to fit our needs for the Exception Analysis engine. It requires its successors to implement the IsApplicable method where the actual exception analysation logic is present. If the check is successful, a new AnalyzedTestException is thrown containing the DetailedIssueExplation. The last holds the exact reason why the test failed which will guide us if there was a bug. Through SetSuccessor method we chain the next handler if the initial check is unsuccessful, we call the successor if one is present.

AnalyzedTestException

This is the custom exception type that all handlers will throw if their validation is successful. Below you can find a sample analysed exception message.

Concrete UML Class Diagram

Participants The classes and objects participating in the modified Chain of Responsibility Design Pattern are: Handler -defines an interface for handling requests. Contains the HandleRequest method. Through SetSuccessor and ClearSuccessor a new handler can be added or removed from the chain. The IsApplicable method should be implemented in the concrete handlers.

-defines an interface for handling requests. Contains the method. Through and a new handler can be added or removed from the chain. The method should be implemented in the concrete handlers. HtmlSourceExceptionHandler - checks in the HTML source code of a page if specific text is present if so a beatified exception is thrown.

- checks in the HTML source code of a page if specific text is present if so a beatified exception is thrown. IExceptionAnalyser - defines an interface for easier global work with handlers.

- defines an interface for easier global work with handlers. ExceptionAnalyzer - implements the IExceptionAnalyzer interface. It holds the main handler for the application and the whole chain that will be executed in case of an exception.

- implements the interface. It holds the main handler for the application and the whole chain that will be executed in case of an exception. ElementFinderService - is the class responsible for localisation of elements on the page and it uses the ExceptionAnalyzer in case some element is not found. HtmlSourceExceptionHandler

Above you can find the code of the concrete handler for finding particular text in the HTML of a page. It uses the IBrowser interface part of the hybrid test automation framework. You can read more about it in my Design & Architecture Series. The another member of the class is the property TextToSearchInSource which holds the text that we will search for. ServiceUnavailableExceptionHandler

The above code prints a beatified message if there are problems with the service and it is unavailable. FileNotFoundExceptionHandler

Similarly, the FileNotFoundExceptionHandler notify us if the file or directory we are trying to open is not found. CustomHtmlExceptionHandler

As I mentioned earlier, the idea about the exception analysis can be developed even further. It can be applied just for specific test cases. For example, if you click a button and the page is not redirected to the next one because of a validation message. Through the above CustomHtmlExceptionHandler, we can create such use-only-once Handlers for particular tests. IExceptionAnalyzer

The IExceptionAnalyzer interface defines methods for easier global work with Handlers. You can add and remove them and access the primary Handler of the app. ExceptionAnalyzer

There is nothing special about the actual implementation of the ExceptionAnalyzer. It uses the methods of the Handler class to setup the chain.

While ago when we were working on the first version of the BELLATRIX test automation framework, I did this research while I was working on a similar feature for our solution.

Chain of Responsibility Design Pattern in Tests ExceptionAnalizedElementFinderService

The ExceptionAnalyzedElementFinder is almost identical to the original one discussed in my Design & Architecture Series. The only difference is that if an element is not found, the IExceptionAnalyzer is called to analyse the page. The primary Handler's HandlerRequest method is used. Extend ExecutionEngineBehaviorObserver

A small addition to the ExecutionEngineBehaviorObserver should be added to register the instance of the actual implementation of the ExceptionAnalyzer. EmptyEmailValidationExceptionHandler

This is a case-specific custom exception Handler which checks whether a particular validation message is displayed on a page or not. LoginProductSiteTests