Despite much progress in the testing framework space, creating automated tests can still require a significant amount of prerequisite coding knowledge. Automated testing solutions, particularly those based in the cloud, offer unique opportunities for teams across an organization — including developers, testers, and non-technical stakeholders. By abstracting large parts of the intricacies involved in automated test creation, each of these roles can align their approach specific to their skillset and needs.
Dive into the fundamentals of cloud-based test automation with this Refcard focused on advantages, considerations, and prominent tools: from code-based solutions to those offering low-code functionality and advanced capabilities like intelligent test automation.
In BDD-Based Integration Testing Framework for Nebula Graph: Part 1, I introduced the evolution of integration testing for Nebula Graph. In this article, I will introduce how to add a test case into the test set and run all the test cases successfully.
Preparing Testing Environment
At the beginning of building the testing framework for Nebula Graph 2.0, we developed some tool classes to help the testing framework quickly start and stop a single-node Nebula Graph cluster, including checking for port conflicts and modifying part configurations. Here is the original execution procedure:
The most effective method of testing an API program involves creating multi-step integration tests that validate common API consumer flows. API endpoints are meant to work together, so it follows that test data coming from one API that feeds another API should not be fixed or pre-built. This is very important because the less you rely on fixed data, the more unpredictable and therefore thorough the testing path will be.
Moreover, API mutation operations may have side effects that cannot be evaluated by simply validating the same endpoint. Side effects propagate throughout the system, and their efficacy can only be validated by querying other endpoints and comparing the results.
Nowadays, more and more companies are building test automation frameworks based on WebDriver and Appium for testing their web and mobile projects. A big part of why there are so many flaky tests is that we don't treat our tests as production code. Moreover, we don't treat our framework as a product. I will present to you all the challenges during the testing of our test automation framework. In the first article from the series, I gave you an overview of our test infrastructure like what tools we use for source control, packages storage, CI and test results visualization. In this second part, we will discuss various types of testing that we did to verify that each aspect of our framework is working as expected — functional, compatibility, integration, usability, installability and many more.
What We Have to Test
Before I share the details of our test infrastructure, I need to explain what we have to test. Our test automation framework is written on .NET Core in C#. This way we achieve the cross-platform requirement. It also has modules for API, desktop, web, Android, iOS and load testing. Generally, there are two ways to use the framework. Most of our clients use it by installing NuGet packages. (NuGet is the package manager for .NET. NuGet client tools provide the ability to produce and consume packages.) To ease the process of configuring the projects we provide a Windows UI installer and CLI installer for Windows and OSX. The UI installer also adds various projects, item templates, and snippets for better user experience in Visual Studio IDE.
This tutorial is a step-by-step guide on how to:
- Install and start using recheck-web and recheck.cli
- Create a first executable test case
- Make it repeatable by ignoring irrelevant changes
Furthermore, this tutorial will show some typical usage scenarios in development and test maintenance, thereby exemplifying the capabilities of recheck-web and recheck.cli.
