Gihani Naligama

Robotics is an interdisciplinary branch of engineering and science that includes mechanical engineering, electronic engineering, information engineering, computer science, and much more. Robotics deals with the design, construction, operation, and use of robots, as well as computer systems for their control, sensory feedback, and information processing.

A robot is a machine programmable by a computer capable of carrying out a complex series of actions automatically. It can be autonomous or semi-autonomous. An external control device can guide robots or the control may be embedded within.

When we consider about Robot test engineering nowadays-mobile autonomous robots are progressively entering the mass market. Thus, manufacturers have to perform quality assurance tests on a series of robots. Therefore, tests should be repeatable and as automated as possible. Testing activity applied to mobile robotic systems is a challenge because new features, such as non-determinism of inputs, communication among components and time constraints must be considered. Simulations have been used to support the development and validation of these systems. Coverage testing criteria can contribute to this scenario adding mechanisms for measuring quality during the development of systems.

When we move from software test engineering to robot test engineering we mainly focus on the following five areas.

1. Stress Testing
2. Performance Testing
3. Longevity Testing
4. Security Testing
5. Functional Testing

Let’s take a look at each, one by one:

Stress Testing

Stress testing is defined as a type of software testing that verifies the stability & reliability of the system. We use the stress test in Robot test engineering to determine the robustness and error handling under extremely heavy load conditions of the robot as well as its backend. Under stress test following areas are addressed.

• Highly Accelerated Life Test (HALT)
  
  HALT is a technique designed to discover weak links of a product. It can trigger failure modes faster than traditional testing approaches. Rather than applying low-stress levels over long durations, HALT testing applies high-stress levels for short durations well beyond the expected field environment. Hardware units like electronics, drive systems, sensors used in the robot are tested under the HALT test by driving robot with different speed limits. Software components, for example, the robot’s control system, which is the brain of the robot, communication between the robot to the backend as well as the monitoring and managing tools of the robot are validated under extreme workload or stress.
  
  When we come to the control node stress test, we stress the different states of the robot and test how it behaves under stress. Failures that are triggered under the above-mentioned tests do not have pass/fail results and it requires a root cause analysis and corrective action to achieve optimum value from testing. It creates the ability to learn more about the design and material limitations as well as the bottlenecks and provides opportunities to continually improve the design before bringing the robot to market. As most items are only as good as their weakest link, the testing should be repeated to systematically improve the overall robustness of the product.
  
  
• Environmental conditions
  
  Environmental conditions testing is the measurement of the performance of equipment under specified environmental conditions such as temperature, static charge, obstacles, climate. When we run the robot in different environmental conditions, how environmental factors affect the robot’s smooth functioning and how much stress it creates on hardware as well as software of the robot was tested.
  
  
• Benchmarking values
  
  Considering the Mean Time Between Failures (MTBF) and the performance statistics of each component, the benchmark is derived for the robot.
  
  A couple of examples are;
  
  
  • The battery charge retention time
  • The motor gear wastage
  • RF antenna interference

Performance Testing

Performance testing is the process of determining the speed, responsiveness, and stability of a computer, network, software program or device under a workload.

In Robot test engineering, we do a performance test to ensure the performance of software, network &, hardware.

Under software, QA should ensure the performance of software resources used like MQTT broker, responsiveness of monitoring & troubleshooting software.

To ensure the network performance, QA should validate the teleoperation control, which used to control the robot externally from a remote location. This involves accessing and controlling the robot in real-time (autonomous mode & manual mode) including in the Network performance test.

The QA should ensure that the performance of the sensor system, control System, and drive system of the robot is carried under hardware performance.

Longevity Testing

Longevity test is an operational testing scheme that uses a baseline work efficiency specification to evaluate large enterprise applications and hardware performance. LT is applied for error checking or heavy usage after a live operational period and is contingent on complexity and size. In robot test engineering QA should carry out the Longevity test and ensure;

• Hardware (sensors & electronics)
  
  How the hardware performs in the long run and benchmarks the value for maintenance.
• Drive system (Performance in continuous operation)
  
  How the robot’s control node and navigation system performs in the long run and benchmarks the value for maintenance.
• Environment (Parameters change with runtime)
  
  How these environmental factors (temperature, static charge, obstacles, climate) affected the robot in the long run and benchmark value for maintenance.
• Software (Performance over time)
  
  How backend, managing and monitoring systems behave in the long run and identify the bottlenecks to optimize the software.

Security Testing

Security testing is a process intended to reveal flaws in the security mechanisms of an information system that protects data and maintains functionality as intended. When we come to robot test engineering, security is a very sensible and important component to verify.

First, the reasons for an attack needs to be identified. Then, the methods of attacking needs to be thought of, for each reason. Finding out whether precautions have already been taken for the above methods would be the next step. However, a system cannot be secured 100% due to the existence of unavoidable reasons. Such an instance would be, an attacker trying to remove the nuts and bolts of a robot. This will also be identified as a threat that cannot be avoided or take any precautions to. A scope will be laid out in the initial stages and only these threats will be looked into. Some threats that are out of scope will be kept unattended.

Functional Testing

When we consider about functional testing there are 4 main areas, which we consider.

• Software
• Navigation/Autonomous functions
• Use cases
• Simulation

Under software testing, we test the robot’s control node which is the brain of the robot by testing its every state with all the possible scenarios and also we test the robot’s utilities like RDL (Report Definition Language) which uses to communicate between robots and the backend. In addition, QA should ensure the software backend as well as managing and monitoring units of the robot-like Troubleshooter application.

Robot Navigation stack plays a major role. The ROS (Robot Operating System ) navigation stack is useful for mobile robots to move from place to place reliably. The job of navigation stack is to produce a safe path for the robot to execute, by processing data from odometry, sensors and environment map.

QA validates the Navigation Stack and the autonomous function of the robot to ensure that the robot navigates safely and correctly with avoiding blockers. A robot is designed for specific use cases and in the case of AZIRO, the robot’s primary use case was to scan for RFID inventory whilst navigating the store floor. The QA should be able to thoroughly validate this use case with special emphasis on the algorithm used by the robot for navigation and testing the edge cases.

When we do the robot functional testing QA use a testing environment. In the test field firstly it is required to assure the intractability of hardware and software by running the robot. Upon fine-tuning the robot’s performance, the next step was to create a more advanced testing environment which was very similar to the actual store environment.

Finally, the robot is deployed in the real-world environment, and a final testing round is carried out to ensure the robot works as expected.

Let’s go to the deeper level of robot testing in the next chapter.

To be continued….

References

Robotics

A Methodology for Testing Mobile Autonomous Robots