To build or to buy a Test Automation Tool

Should I build or should I buy?

As continuous integration and development shift the momentum of software delivery to a higher gear, testing teams need to upscale their test strategies. We talk about faster and frequent deliveries with higher quality. We stress on tools that lubricate efficient and effective software delivery. Therefore, quality of these releases play a major role in meeting these delivery expectations. It is for this reason that testing has evolved into an automatic execution process.

The history of software automated testing has been trailed by the evolution of software development. The introduction of the GUI (Graphical User Interface) applications paved way for automation tools that have the recording and playing back functions. Over the years, we as quality assurance engineers have been investing our time and resources on learning sophisticated enterprise test automation tools that were built by well-known solutions providers such as HP, Microsoft, Atlassian and many more.

However, there are many instances where these tools have not provided a blanket solution for our automated testing needs. Also, integrating and adjusting an off the shelf automated testing tool could be even more complicated and time consuming. This is when testers should be driven to build their own custom automation tool. This article will help you decide whether to write your own tool for automation and if you choose to do so, what are some of the key parameters you should consider during your implementation.

Analyze the pros and cons

As an initial step to decide whether to build your own tool, it is advised to measure the pros and
cons. Below are some of the advantages of writing your own test tool:

Can acquire more control over the test design and architecture.
Independence in development and maintenance.
Flexibility of adding or removing features as per requirement.
Easy to apply coding, testing process and delivery standards in the framework.
Creates a platform to market the framework as an enterprise tool.
Improve and utilize the knowledge and capacity of the QA Engineers within a team.

Although investing on a framework on your own seems appealing given the above advantages,
there are negative impacts of this approach as well. We can list down the disadvantages as
follows:

Consumes time, development resources and other costs.
Requires a thorough study and evaluation to assure if such a tool is not already in the market. No need to reinvent the wheel!
Requires highly skilled automation engineers.
Uncertainty of the return on investment.
Requires a strong proof of concept to impress investors.

Incorporate standards and best practices

After measuring the pros and cons of building your own tool for test automation, if you finally decide to go ahead with writing your own, then there are few concepts and parameters to consider. It is important to note here that a test automation framework can be explained as a collection of standards, processes that facilitate component interactions and integrations on top of which test scripts can be executed. In a pro-agile development background, writing a framework that incorporates best practices is a challenge. Following are key concepts and standards that facilitate building a powerful automation test framework:

Separate the tests from the framework
To promote reusability of tests and easier maintainability, it is advised to separate the tests from the framework. Which means that your test scripts need to be included in a separate package while your framework related code exists inside a separate package.
Separate the tests from test data
This separation will make sure that no modifications will be required to the tests during each test
run when data needs to be changed for input parameters.
Look for other mechanisms other than UI to verify tests
This means that the assertions you use in your tests should not only depend on the UI. For
example, you can verify if a test is passed by a web service call. This will largely cut
down the UI control waiting times and will eventually speed up test execution.
Use libraries
For maintainability purposes, it is advised to separate out reusable classes, external connections,
common components and generic functions into a library. Test script writers can then invoke these libraries in their code.
Following after coding standards
The automation framework should follow after coding standards to maintain consistency, enforce security measures and also to help coordinate with the development team. This will largely improve the knowledge of the QA engineers who are involved with the test framework development as well.
Maintain versioning
This will largely help monitor the modifications done to the framework over time. Versioning
will also play a major role during licensing and deployments, if the tool is eventually
marketed as a commercial product.

If you have very specific test automation requirements and you cannot find a suitable automated testing tool in the market to facilitate your needs, the answer is Yes! Consider the pros and cons and build your own tool. The value you are adding to your organization and the benefits you are earning for your personal development will be immeasurable!

Menaka de Silva

Lead QA Engineer