Teams usually prefer a testing solution that can be easily integrated into the existing CI/CD infrastructure without much work. But what if your testing tool can not be easily automated with your CI/CD solution or run in your existing Kubernetes infrastructure?
We decided to explore Testkube, which can take the test as code for any testing tool, automate test execution and integrate with CI/CD, and generate artifacts in Kubernetes. This would allow us to perform testing seamlessly and leverage the benefits of Kubernetes. So let's learn in this blog about acceptance testing with the Robot Framework and how to automate it with Testkube.
Acceptance testing also allows you to ensure software complies with legal and regulatory requirements. This reduces the risk of legal consequences or penalties due to non-compliance, especially in certain industries, such as healthcare or finance.
Other testing practices, such as API, load, etc., contribute to different aspects of software quality, such as performance, reliability, and interoperability, and are usually done before acceptance testing.
The Robot Framework is an acceptance testing tool that is easy to write and manage due to its key-driven approach. Let us learn more about the Robot Framework to enable acceptance testing.
Robot Framework stands out for the following reasons:
Test cases written in Robot Framework can be executed using the robot
command-line tool or through various integrated development environments (IDEs) and continuous integration (CI) systems. During execution, Robot Framework generates detailed reports and logs, highlighting the status of each test case and providing insights into test results and any encountered issues.
While running Robot Framework tests against legacy/non-containerized applications is usually straight-forward, there are several challenges associated with running these tests against a containerized application infrastructure built on Kubernetes:
Thus, when using the Robot Framework for acceptance testing of your containerized applications, you will require a test-execution framework that eases the life of a developer and can orchestrate Robot Framework tests with all the above caveats in mind. Fortunately,there is such a framework specifically targeted at running any kind of testing tool, including the Robot Framework, in a containerized application infrastructure: Testkube.
To run tests in Testkube, one defines Test Workflows for managing the execution lifecycle of individual test(s), which will then be executed in your Kubernetes environment using the corresponding testing tool image. Let us understand more about the Test Workflows feature.
Test Workflows provide a declarative approach for defining test execution lifecycles. It leverages Kubernetes-native capabilities to orchestrate and execute tests, collect real-time artifacts, and display them on the Testkube dashboard. A Test Workflow is defined as a single YAML configuration that is stored as a custom resource in your Kubernetes cluster(s), Test Workflows allow integration with existing testing tools and GitOps pipelines, and also provide templates for standardized testing configurations adaptable to various needs.
We will see two demos here using the Robot Framework in Testkube using the sample example, Restful Booker, provided by Robot Framework.
So let us see Testkube in action performing an acceptance test using Robot Framework. Before we get started, let us verify the following prerequisites.
To follow along, you will need to set up the following:
Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube agent configured.
In the basic execution, we will run the example Restful Booker test code in Testkube using Test Workflows. We have pushed the test case to a GitHub repository. These files will by default be mounted by Testkube to the’/data/repo’ directory when the test executes. Let us get started.
Testkube provides an option to start from scratch or use existing examples to set up a workflow
Create a test workflow with your custom details.
kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: basic-acceptance-test
spec:
content:
git:
uri: https://github.com/cerebro1/robot-framework-test.git
container:
image: ppodgorsek/robot-framework:7.1.0
steps:
- name: Run test
shell: robot --outputdir /data/repo/output /data/repo/test-restful.robot
artifacts:
paths:
- /data/repo/output/*
Thus, with the help of Test Workflow, we were able to run a Robot Framework test seamlessly without any setup required on the local machine. The output shows all the tests have passed. Testkube has processed the artifacts and provided them for download.
Now, let us suppose, this Robot Framework test has some parameterized values to be passed with the test. Don’t worry! Testkube has got you covered. In the Test Workflows, you can pass the parameters and provide custom values. Let us understand this in detail in the coming section.
In Robot Framework, you can define the parameters as variables, and the values can be passed at the time of execution. The default values to these parameters can be set in the Robot Framework test configuration as shown below:
*** Variables ***
${my_var} my_default_value
*** Test Cases ***
Test Case 1
Log ${my_var}
While using the robot
command line utility, these variables can be passed as parameters as shown below:
$ robot --variable my_var:my_value test.robot
Test Workflows allows you to define configuration parameters for your workflows, which can be passed to the Robot Framework as input. Let’s add parameters for the firstname
, lastname
and totalprice
parameters to our workflow:
kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: basic-acceptance-test
namespace: testkube
spec:
config:
firstname:
type: string
default: Jim
lastname:
type: string
default: Henderson
totalprice:
type: string
default: "100"
content:
git:
uri: https://github.com/cerebro1/robot-framework-test.git
container:
image: ppodgorsek/robot-framework:7.1.0
steps:
- name: Run test
shell: robot --variable firstname:{{ config.firstname }} --variable lastname:{{config.lastname
}} --variable totalprice:{{ config.totalprice }} --outputdir /data/repo/output-{{config.firstname}}-{{config.lastname}}
/data/repo/advanced-test-restful.robot
artifacts:
paths:
- /data/repo/output-{{config.firstname}}-{{config.lastname}}/*
As you can see we have defined three config variables which we then pass to the robot command, and use in the naming of the output folder. When we new try to run this test we are first prompted to provide values for our variables:
Just pressing the run button uses the default values provided as you can see in the log output:
For a scalable approach to test execution in a Kubernetes environment, Testkube is the solution. It takes any testing tool, provides a template, or allows you to start from scratch and automate your tests. Teams can manage the entire test life cycle in an optimized way from a single dashboard. This not only empowers them with the feasibility of scaling but also standardizes the testing process.
In this tutorial, we have seen how acceptance testing with Robot Framework could be done using Test Workflows in Testkube. Similarly, teams can create multiple test workflows to manage unit, load, API, acceptance, etc., tests from the same platform.
We invite you to try Test Workflows. Visit Testkube website to get started. If you find yourself struggling with anything, feel free to drop a note in our active Slack community, and someone will help you out.
Testkube is a test execution and orchestration framework for Kubernetes that works with any CI/CD system and testing tool you need, empowering teams to deliver on the promise of agile, efficient, and comprehensive testing programs by leveraging all the capabilities of K8s to eliminate CI/CD bottlenecks, perfecting your testing workflow. Get started with Testkube's free trial today.
Related topics: