Testing an application’s capability usually requires setting up a production-like environment to run tests with realistic insights into the functionality and performance of the application. Tests can be both functional and non-functional, all depending on the requirements on the application and its infrastructure.
Teams usually prefer a testing solution that can be easily integrated into the existing CI/CD infrastructure without much work. But what if your testing tool can not be easily automated with your CI/CD solution or run in your existing Kubernetes infrastructure?
We decided to explore Testkube, which can take the test as code for any testing tool, automate test execution and integrate with CI/CD, and generate artifacts in Kubernetes. This would allow us to perform testing seamlessly and leverage the benefits of Kubernetes. So let's learn in this blog about acceptance testing with the Robot Framework and how to automate it with Testkube.
Acceptance testing is a phase in software development where the software is tested to determine if it meets the requirements specified by the client or stakeholders. It's usually performed after system testing and before deployment. Acceptance tests are typically written from a user's perspective and aim to verify that the system behaves as expected in real-world scenarios.
Acceptance testing also allows you to ensure software complies with legal and regulatory requirements. This reduces the risk of legal consequences or penalties due to non-compliance, especially in certain industries, such as healthcare or finance.
Other testing practices, such as API, load, etc., contribute to different aspects of software quality, such as performance, reliability, and interoperability, and are usually done before acceptance testing.
The Robot Framework is an acceptance testing tool that is easy to write and manage due to its key-driven approach. Let us learn more about the Robot Framework to enable acceptance testing.
Robot Framework is an open-source, keyword-driven test automation framework that supports Python, Java, and .NET and is widely used for acceptance testing, acceptance test-driven development (ATDD), and robotic process automation (RPA). It provides a highly readable, tabular syntax for writing test cases in a plain-text format, making it accessible to both technical and non-technical users.
Robot Framework stands out for the following reasons:
Test cases written in Robot Framework can be executed using the `robot` command-line tool or through various integrated development environments (IDEs) and continuous integration (CI) systems. During execution, Robot Framework generates detailed reports and logs, highlighting the status of each test case and providing insights into test results and any encountered issues.
While running Robot Framework tests against legacy/non-containerized applications is usually straight-forward, there are several challenges associated with running these tests against a containerized application infrastructure built on Kubernetes:
Thus, when using the Robot Framework for acceptance testing of your containerized applications, you will require a test-execution framework that eases the life of a developer and can orchestrate Robot Framework tests with all the above caveats in mind. Fortunately,there is such a framework specifically targeted at running any kind of testing tool, including the Robot Framework, in a containerized application infrastructure: Testkube.
Testkube is a test orchestration platform that supports the execution of any testing tool in a containerized application infrastructure. It leverages Kubernetes to provide a scalable and consistent execution environment for all your tests, and includes a unified dashboard for test definition, reporting, and troubleshooting.
To run tests in Testkube, one defines Test Workflows for managing the execution lifecycle of individual test(s), which will then be executed in your Kubernetes environment using the corresponding testing tool image. Let us understand more about the Test Workflows feature.
Test Workflows provide a declarative approach for defining test execution lifecycles. It leverages Kubernetes-native capabilities to orchestrate and execute tests, collect real-time artifacts, and display them on the Testkube dashboard. A Test Workflow is defined as a single YAML configuration that is stored as a custom resource in your Kubernetes cluster(s), Test Workflows allow integration with existing testing tools and GitOps pipelines, and also provide templates for standardized testing configurations adaptable to various needs.
To run a Robot Framework acceptance test with Testkube, you will need to define a corresponding TestWorkflow that uses a prepackaged Robot container image to run the test cases that you have defined - let’s have a look at how that works.
We will see two demos here using the Robot Framework in Testkube using the sample example, Restful Booker, provided by Robot Framework.
So let us see Testkube in action performing an acceptance test using Robot Framework. Before we get started, let us verify the following prerequisites.
To follow along, you will need to set up the following:
Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube agent configured.
In the basic execution, we will run the example Restful Booker test code in Testkube using Test Workflows. We have pushed the test case to a GitHub repository. These files will by default be mounted by Testkube to the’/data/repo’ directory when the test executes. Let us get started.
Testkube provides an option to start from scratch or use existing examples to set up a workflow
Create a test workflow with your custom details.
kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: basic-acceptance-test
spec:
content:
git:
uri: https://github.com/cerebro1/robot-framework-test.git
container:
image: ppodgorsek/robot-framework:7.1.0
steps:
- name: Run test
shell: robot --outputdir /data/repo/output /data/repo/test-restful.robot
artifacts:
paths:
- /data/repo/output/*
Thus, with the help of Test Workflow, we were able to run a Robot Framework test seamlessly without any setup required on the local machine. The output shows all the tests have passed. Testkube has processed the artifacts and provided them for download.
Now, let us suppose, this Robot Framework test has some parameterized values to be passed with the test. Don’t worry! Testkube has got you covered. In the Test Workflows, you can pass the parameters and provide custom values. Let us understand this in detail in the coming section.
In Robot Framework, you can define the parameters as variables, and the values can be passed at the time of execution. The default values to these parameters can be set in the Robot Framework test configuration as shown below:
*** Variables ***
${my_var} my_default_value
*** Test Cases ***
Test Case 1
Log ${my_var}
While using the `robot` command line utility, these variables can be passed as parameters as shown below:
$ robot --variable my_var:my_value test.robot
Test Workflows allows you to define configuration parameters for your workflows, which can be passed to the Robot Framework as input. Let’s add parameters for the `firstname`, `lastname` and `totalprice` parameters to our workflow:
kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: basic-acceptance-test
namespace: testkube
spec:
config:
firstname:
type: string
default: Jim
lastname:
type: string
default: Henderson
totalprice:
type: string
default: "100"
content:
git:
uri: https://github.com/cerebro1/robot-framework-test.git
container:
image: ppodgorsek/robot-framework:7.1.0
steps:
- name: Run test
shell: robot --variable firstname:{{ config.firstname }} --variable lastname:{{config.lastname
}} --variable totalprice:{{ config.totalprice }} --outputdir /data/repo/output-{{config.firstname}}-{{config.lastname}}
/data/repo/advanced-test-restful.robot
artifacts:
paths:
- /data/repo/output-{{config.firstname}}-{{config.lastname}}/*
As you can see we have defined three config variables which we then pass to the robot command, and use in the naming of the output folder. When we new try to run this test we are first prompted to provide values for our variables:
Just pressing the run button uses the default values provided as you can see in the log output:
Although it is useful to run tests directly from the Testkube Dashboard as shown above, you will most likely want to start these tests from your existing CI/CD pipelines. Fortunately, Testkube makes this really easy:
Using Test Workflows, we were able to easily configure the execution of our Robot Framework test, and also pass custom variables to Robot Framework in a seamless way, and automate the execution of our tests from our CI/CD system. The configuration is easy to understand and manage, which not only reduces the efforts of developers but also increases the maintainability of the configuration.
For a scalable approach to test execution in a Kubernetes environment, Testkube is the solution. It takes any testing tool, provides a template, or allows you to start from scratch and automate your tests. Teams can manage the entire test life cycle in an optimized way from a single dashboard. This not only empowers them with the feasibility of scaling but also standardizes the testing process.
In this tutorial, we have seen how acceptance testing with Robot Framework could be done using Test Workflows in Testkube. Similarly, teams can create multiple test workflows to manage unit, load, API, acceptance, etc., tests from the same platform.
We invite you to try Test Workflows. Visit Testkube website to get started. If you find yourself struggling with anything, feel free to drop a note in our active Slack community, and someone will help you out.
Testkube is a test execution and orchestration framework for Kubernetes that works with any CI/CD system and testing tool you need, empowering teams to deliver on the promise of agile, efficient, and comprehensive testing programs by leveraging all the capabilities of K8s to eliminate CI/CD bottlenecks, perfecting your testing workflow. Get started with Testkube's free trial today!
Related topics:
Join the Testkube Community in one of these channels: