Importance of AI for test automation

Software Testing Evolution

In software development life cycle, software testing has higher importance. Even though developers develop according to requirement, during testing only we make sure that everything meets the expectation. When looking at past decades, it is quite evident that software testing has gradually evolved. At the first stage, testing started only after the software development was completed. Currently, most of the companies have moved to agile processes and that has enabled software testing to be initiated parallel to the development process. Reason behind selecting agile processes is finding bugs as early as possible and fixing those earlier. For that automation testing came to play a role with the manual testing process. In this era, it can be seen that Artificial Intelligence (AI) is stepping out to software testing. It is worth talking about and application of AI to test automation. In this article, we will be taking a look at AI test automation and it will also provide guidance on how to apply it.


What is AI?

AI is also known as machine intelligence. Simply, machines performing natural human intelligence is what’s known as ‘AI’ or Artificial Intelligence. Image Recognition, Speech Recognition, Chatbots, Natural Language Generation etc. came to the world as applications because of AI support.
When looking at the current AI systems, most of them belong to “Limited memory” (based on the past experiences, system/machine reacts). Above mentioned can be also be used in test automation to maintain tests and here will provide directions to apply AI to your project.


Why should you apply AI on Test Automation?

As the survey conducted by testcraft using 200+ testers, followings are the major concerns identified in testing

  • Test Maintenance
  • Not enough man power
  • Lack of integrations
  • Want to increase coverage
  • Hard to find good test engineers
  • Not able to keep up with an agile schedule

Out of all participants, 50% of testers have mentioned that Test Maintenance is the biggest bottleneck for testing. So AI will be the best solution to overcome this issue. Other than the test maintenance, AI supports time saving, stability of tests, finding bugs fastest way and fixing those much faster etc.


AI enhance the software testing efficiency
Self-Healing

“Self-healing” is the mechanism that is used in AI to overcome the above mentioned drawbacks. What self-healing does is, it identifies damages, errors by itself and has ability repair those damages and fix errors automatically by itself without human involvement.

The problem in test automation is, once we automated things, if those got failed we have to spend some time to validate it to identify root cause. This will be time taken one. So by patterning this self-healing behavior in to our test automation, mainly it supports error handling and also for information flow management. When occurring errors, AI is able to adjust those by viewing system response and accordingly patterned to self-heal those errors. With this mechanism, followings are the advantages can be gained for testing.

  1. Able to maintain test stability
  2. Able to create most reliable automated tests
  3. Able to identify bugs earlier and resolve those so fast way.
  4. Able to save time and reduce the cost of failures
  5. Reduce the maintenance
  6. Able to follow continues learning with data using and make correct decisions to overcome failures


AI test automation Tools

Following are the currently most popular AI test automation tools

  • Applitools
  • SauceLabs
  • Testim
  • Sealights
  • Test.AI
  • Mabl
  • ReTest
  • ReportPortal

Stay informed about future AI test automation trends

Applying AI for test automation will be a great opportunity for software testing in future since it addressing current test maintain issues without any human involvement. As a result of that testers will not have to go into the code manually to identify issues and to validate those issues. To get the greatest benefit, it is good keeping touch with future AI test automation trends. For that you can follow Blogs, research articles related to AI.

AI in test automation is not an obstacle, but an opportunity

Vindya Gunarathna

QA Engineer

Robotic Process Automation

Robotic Process Automation

Today, the world is moving forward with variants of technologies. As a result of that, the future workplace will be blend with human and software bots. This human and software bots relationship will create many exciting opportunities for the world.

Are you ready for that challenge?

If not, now would be an excellent time to consider those exciting technologies.

RPA is one of that technology that you can challenge to world technology.

The process of automating business operations with the help of robots to reduce human intervention is called RPA. Actually, RPA is a challenging technology that is designed to automate the high volume of repeatable tasks that take a large percentage of worker’s time.

This is a technology that allows anyone today to configure computer software to emulate and integrate the actions within the humans and digital systems to execute a business process.

RPA allows organizations to automate at a fraction of the cost and time previously encountered by delivering direct profitability while improving accuracy across organizations and industries.


Getting started with RPA

What is RPA?

The process of automating business operations with the help of robots to reduce human intervention is known as Robotic Process Automation

Why RPA?

Think you want to publish an article on social media every day at a specific time. For that, you have to do it manually or appoint an employee to do that task. Both methods are time and money consuming. Imagine, if you can appoint a software robot to do that repetitive tasks on behalf of you. It’s awesome, That’s why RPA comes into the picture.


What are the things that RPA robots can do?

RPA robots are capable of mimicking human actions and by using this RPA robot we can log into applications, move files and folders, copy and paste data, fill in forms, extract data from documents, and scraping data from the web sites


Difference between test automation and RPA

RPA Implementation methodology

  • Planning – Identify processes which you want to automate
  • Development – Start developing the automation workflow as per agreed plan
  • Testing – Run testing cycles to identify and correct defects
  • Maintenance – Provide continuous support after going production


Best Practices of RPA Implementation

  • Consider business impact before opting for the RPA process
  • Define and focus on the desired ROI
  • Focus on targeting larger groups and processes
  • Don’t forget the impact on people


Advantages of RPA

  • RPA can automate complex process easily
  • RPA takes care of repetitive tasks and saves time and resources
  • Real-time visibility into a defect
  • Software robots do not get tired

Disadvantages of RPA

  • The bot is strict to the speed of the application
  • After changing the automation scripts that need to reconfigure the bot


RPA Tools


We can select the RPA tool by considering below 4 parameters.

  • Data
  • Interoperability
  • Type of tasks mainly performed
  • Artificial Intelligence

There are a lot of tools that are used for robotic process automation and below are some popular tools that are used in today’s industry.

  • UiPath
  • Blue Prism
  • Automation Anywhere

Wrapping Up

Finally, this is a marvelous technology which will make our life much more exciting. Here I have only shared the basics of RPA. It has more to discover and learn. Keep learning more about RPA……

Sajeeka Mayomi

QA Engineer

Woes of a Fleet Support Engineer During a Pandemic

Synopsis

Pandemics such as COVID-19 lead many organizations to request their employees around the world to adapt to work from home (WFH), while attempting to run operations in unaffected regions as efficiently as possible. This is especially true for organizations that have large global transport and logistics operations. Technology such as wearables and handheld smart devices are a crucial part of any such operation today. The upkeep of this technology is heavily dependent on tech support engineers, who are expected to investigate and remediate user queries and potential breakdowns from all over the world. This operation is commonly known as Remote Monitoring and Troubleshooting (RMT).

This article discusses the challenges faced by such support engineers during WFH periods and how the right tool could have solved these problems.

Off-the-Shelf EMM Software (a.k.a MDM) are Failures as RMT Solutions

* EMM – Enterprise Mobility Management * MDM – Mobile Device Management * RMT – Remote Monitoring and Troubleshooting

Due to misinformed decisions of organizations, many support engineers are stuck with off the shelf EMM tools for supporting Remote Monitoring and Troubleshooting (RMT) requests. Unified Endpoint Management(UEM) tools serve the need of tasks such as O.S and application updates, security compliance and lock downs; but miserably fall short when it comes to remote monitoring and troubleshooting.
Why? Because every organization’s uniqueness reflects in their devices, software as well as how they use them. Being able to troubleshoot such devices and applications require bespoke solutions. Such bespoke solutions are not the focus of UEMs, as they don’t focus on offering all the details required for a support engineer to perform RCA (Root Cause Analysis). As a result, they are forced to resort to very cumbersome methods of obtaining the information they need, such as Remote Desktop, or using a mix of their own scripts and dashboards, which are highly insecure and unstable.


Remote Desktop is a Cumbersome Bandwidth Bandit

Due to the lack of information and remote control for troubleshooting, whenever a ticket is opened, support engineers have got used to resorting to remote desktop access to the end device to get the data and control that they need to solve a problem. Remote desktop is heavily dependent on a stable and consistently fast network connection, which is not something that you can expect from someone working from home.


Domestic Data Rates Are Exorbitantly Higher

Organization’s generally opt for unmetered leased lines for their internet connectivity. When support engineers work from home, they are forced to use domestic connections which are metered, and result in sending the operational costs through the roof.


Can’t Look Away for a Second, but You Have To

One of the biggest perks of working from home, is the fact that you get to be around your loved ones. But, support engineers who need to keep their eyes on a terminal or a display are under a lot of tension whenever they are away from it even for a second. At the same time, being away for a short time is not something that can be fully avoided. This is a lose-lose situation for the organization as well the employee. The support engineer faces anxiety of failing to meet SLAs because he/she didn’t see the incident on time and the organization suffers the consequences.

MATRIX24x7 The Solution

1

2

3

4

A Tailor Made Solution Vs. Ready Made Software

  • Free yourself from the fixed functionality and rigidity of off the shelf, ready-made applications.
  • Mold the platform to your requirements vs. adapting to it.

A Partner Vs. A Vendor

  • Empathetically driven towards gaining a full understanding of the business and operation prior to solutioning.
  • Access to our best tech and domain experts vs. business sales intermediaries.

This is exactly what MATRIX24x7 is. An RMT platform that can deliver tailor made solutions, and a partner that operates as the CTO of your support operation.

Heshan Perera

Associate Architect / Manager – IoT Platform

How Many Test Cases Do You Need? How Many Do You Have?

A recent discussion with a top-brass individual in Software QA Domain, brought up this very question, which I really didn’t bother about for the past seven years as a QA professional. After the discussion was over, I managed to find some time to reflect on it. Does it matter how many test cases are there to test a particular application?

First, a bit of theoretical background…

Well, this is something we all know. According to the ISTQB foundation level text (which by the way, can be considered as a comprehensive handbook for a test engineer) and IEEE829 Test Documentation Standard, test cases are stemmed out as part of test design and documentation procedures.

Given a software system to test, the test engineers first analyze the system specification to find test conditions – which in simple terms, are things that could be tested. Once the engineers are satisfied that they have found a good amount of conditions that covers the system’s functional and non-functional aspects (which is named as ‘Test Oracle’), they select which of these to be actually used for testing and prioritize them based on importance and risk levels assigned to each one.

Then come the test cases – they are the detailed specifications on how one or more test conditions are actually tested. A pre-determined set of pre-conditions, inputs, steps to follow and expected outputs are documented as a test case, which resides in a formal test specification.

On top of these, test procedures are also documented to specify how the test cases should be executed. Incase test automation comes in to the picture, the procedures may extend to automation scripts etc.

But what is most important and the thing to remember is that no matter how these standards and ‘conventional ways of doing things’ are set, during the actual execution, it all depends on several factors.

Practically, you can’t always do it by the book…

Unless you’re developing and testing your own product and you are not in a hurry to take it to the market, which is highly unlikely, there is a plethora of constraints to consider.

First and foremost, it is Time. If you are working on a fixed-cost project, you have to utilize the maximum amount of available time for testing. This leaves the documentation with a smaller ration. If the project is budgeted on time-and-material basis, the customer will want to see the latest iteration in working condition after each build or sprint. Again leaving documentation and design a smaller chunk of the time available.

But anyway, you cannot test without doing any of these…

Yes, you have to analyze the system and identify the test conditions – otherwise there is little or no means of knowing what kind of a beast you are dealing with. And yes, you have to prepare test cases – otherwise there is no way to communicate to other stakeholders of the project, what you test and how you do it. And even if you do all the testing by yourself, keeping track of what to do and what have been done is always the best practice.

You may use different means to keep track of these test cases – ranging from spreadsheet based check-lists to well-structured and mapped test cases in a test management tool. The means of managing test cases can be selected to suit the constraints relevant to the project.

Which method should you choose?

The common rule of thumb is, the more details you have, the better the test case is. For an example, ‘Input numbers 2 and 3, calculate total, correct output should be 5’ is more clear cut than just saying ‘test the sum of two numbers’.

But still, it might make no sense to spend three days to compose the test cases, if you only have five days to test, fix bugs and re-test a particular application. The way the test cases should be prepared will have to be decided on a case-by-case basis.

The best way is to be ‘Efficient & Effective’!

As our management gurus put it, ‘to be efficient is to do the thing right, and to be effective is to do the right thing’. Putting this into context, it can be safely considered effective if the test cases you have adequately cover the functional and non-functional aspects of the software under test.

Efficiency, however, is in a league of its own. Depending on the person, one might cover a certain function with just one test case whereas another might require more than one. If all the functional and non-functional aspects can be covered with say, 10 test cases, no purpose is served by having more than 10 test cases or less thereof.

Turns out, it is a numbers game…

There is this specific case I want to cite as an example. For this particular web-based application, my test team had prepared about 90 test cases, excluding certain edge scenarios. We ensured all the test conditions were covered and the test execution time was typically 32 man hours.

Once the application finally passed through my team and sent to the client, he wanted to test it again using some third-party testers. After about two weeks, this test team had reported 80 new defects! Both my test team and development team were not convinced by this result, so we called up a triage meeting to review the defects. After thoroughly analyzing all 80 defects, what the team determined was that only two of the reported 80 incidents were actually valid defects.

When we took it back to that third-party test team, it was found out that they have prepared over 300 test cases and to execute all these, the team (we were told there were two engineers) took about two whole weeks. Amazed by this result, we cordially requested them to share the test cases just to get an idea on what they were up to. Needless to say, even after months and several reminders, it is apparent that they did not want to share the test cases with us. Therefore, the conclusion was that they have exaggerated on the work they did.

Ultimately it comes back to the numbers game I was talking about. Showing your management or the client that you have a large number of test cases and logging a large number of defects is one method some test teams resort to, in order to just ‘show’ that they did good work. Since the management or a client would rarely dig deep to examine the validity in such situations, it was a safe bet to make.

At the end of the day, ‘How many test cases should be there?’

The first thing to do is to identify the test conditions. The time spent here will never go wasted since this also helps you to understand the system better. Then you have to understand the nature of the project including the perception of the management and the client about testing.

From there, use test design techniques you prefer to carefully craft the test cases. Remember, the set of test cases that comes out as a result of this should be both efficient and effective.

The challenge is to prepare not more or less but the right amount of test cases which will provide complete coverage to achieve the greatest effectiveness.

Reviewing this work in collaboration with the development and business teams is always a good practice. It would help to bridge any gaps or misunderstandings. Other than that, as long as the set of test cases you have could cover all functional and non-functional aspects of the software application under test, the number of test cases does not really matter.

Image Courtesy: unsplash.com/@theheartdept

Deshal Weerasinghe

Deshal Weerasinghe was an Associate QA Lead at Zone24x7.

Automation to Drive Development Productivity

The sudden ring of an alarm clock awakens me. It is the dawn of a new day, where I hurry off to get ready for work. How often in our lives do we encounter this? If it wasn’t for the alarm clock, all that was planned for the coming day would have been at stake, wouldn’t it?

Well, this post isn’t about the values of having an alarm clock nor is it something to promote timeliness and due diligence through the usage of alarm clocks. Instead, it is about my opinionated view on the usage of automation, and how disruptive and productive it can be when introduced into all aspects of the software development life cycle.

I guess, we all do agree that automating repetitive and tedious procedures add value for software development projects. Yet, even though we believe in this so obvious concept, how often do we notice that certain parts of our project development process and workflows, have been overlooked in terms of automation?

We often tend to be optimistic, and rely on “simple” written or even verbal instructions, and expect all professionals that we work with to be likely disciplined enough to follow them.

But boy aren’t we wrong!


Written instructions are forgotten or worse, misinterpreted overtime . . .

My experience has always shown that even the simplest of recurrent instructions, we put aside for a team to perform, tend to become overwhelmingly tedious as time passes on.

Let’s take the example of instructions, we write for setting up prerequisites of a development environment.

We do this with the hope that a newcomer would be able to set it up him or herself, so that experienced members would not have to spend time doing so. However, our hopes are often shattered due to misinterpretations of these instructions, which lead to problems that are sometimes found out at a later stage of the project. Problems resulting from such misinterpretations, generally surface later in the software development life cycle and are very time consuming to solve. That’s when we realize that it is always better to write a configuration to automate this task, than to write instructions for a human.


Automation cannot be blamed for not knowing what happens beneath . . .

It is sometimes debated that total automation creates an environment, where on-boarding engineers would lack true understanding of underlying principles. But I do see it in a different way.

Any tool, even a compiler, would hide and abstract out a certain amount of its underlying process. However, an individual could either go with the flow as it is, or can be inquisitive enough to dig deep and understand the overall principles and concepts of what is being automated.

The mere fact of manual intervention, to provide understanding of something automatable doesn’t seem meaningful to me.


Everyone can and should contribute to automation, EVERYONE. It demonstrates true mastery . . .

There is also a belief that automation holds fit, only for certain aspects of the software product life cycle such as QA, Test Automation and Deployment Automation. However, I believe everything that can be automated should be automated. Hence, improving the productivity of the development life cycle through automation and evolving in a smarter way, is part and parcel of everyone contributing to a team.

For example, a development team could identify repetitive and mundane items that consume their time and pitch in to automate such tasks. Code Templating and Generators such as Yeoman are tools that developers can leverage for this purpose.  These are excellent for putting best practices into place, while minimizing mistakes of repetitive tasks that involve minor modifications.

When it comes to setting up development environments, Vagrant is a champion. It enables seamless and error free setup and deployment of development environments among the development team.


Good news, there are lots of tools out there. Bad news, we rarely use them! 

Workflow automation and productivity tools have evolved mostly in the realm of Web Development during the past years. Tools ranging from Grunt and Gulp to Webpack comes to mind immediately.

These tools, though minor, have been proven to speed up and reduce the recurrent work done by a developer. Tasks like Code Minification for file optimization, hashing files to avoid cache updates, and many more recurring tasks, can be automated through a well thought out workflow for modern web development.

If done manually, a developer would need to perform each of these tasks and the degree of error in missing one would be relatively high. Even when it comes to attaching scripts to a web page, the days where manual script tags being referenced individually is long gone. Web tools are now capable of identifying dependencies and logically sequencing the script files that go into a web page.

Functions such as Continuous Integration (CI) and DevOps also gain immensely from automation. Containerization technologies like DockerRocket and LXD provide isolated sandboxed environments to build, deploy and run in a repeatable fashion. Build recipes written through Ansible or Chef,  ease out the struggle in maintaining a continuous pipeline and build flow.


Start early, it makes things easier . . .

Therefore, from the very beginning, starting off with the intuitive developer who is driven towards automation, the source control that automatically triggers the build in a CI server and the point it is deployed and finally to an automated test suite, which is executed to verify the build stability, an automation pipeline should be facilitated.  This brings focus to minimizing defects and error prone situations throughout the entire software development life cycle. Ultimately this will uplift productivity.

Planning for automation at the start of a project, at its infant stages, yields in easier adaptation. If overlooked at the start, as any change, automation too would become an anchor and a rigorous task due to complexities added later on.

Therefore, automation planned out carefully, considering developer productivity to testability during the early stages, results in automation artifacts that are similar to the other product artifacts.  These can be versioned and would evolve with time.

Though the ROI of automation may not be obvious in the short run, automation is a silent servant that proves its worthiness in the long run.

References: DeveloperDotStar

Lasitha Petthawadu

was a Tech Lead at Zone24x7