Introduction

In this era of cloud native application development, automated test execution is continuing to gain significance. For organizations actively working on improving their applications, testing is now a day-in day-out task. They have to provide their developers with tools to help them with efficient testing and improve the overall testing experience. So, if your testing approach is still in the past, your teams would struggle to identify vulnerabilities, manage test artifacts, compare test result behavior, and most importantly, with time to market.

There are several tools available on the CNCF landscape that can help you automate test execution in your delivery pipelines but are they enhancing the testing experience for your teams? Are they helping you launch features in the stipulated time? These are valid questions if you want to provide your engineers with the right set of tools so that they can ultimately deliver high-quality software faster. 

Let’s have a look at two tools in the CNCF Landscape that except at their corresponding functional areas, and can be combined for a workflow <> test-execution powerhouse; Argo Workflows and Testkube.

What is Argo Workflows?

Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition). Argo Workflows allows you to

• Define workflows where each step is a container.
• Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG).
• Easily run compute-intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.

In many of its use-cases, adding tests to validate intermediate steps of a workflow, or its final outcome, is a common practice to ensure consistent high quality results from your workflows - which brings us to Testkube.

What is Testkube?

Testkube is a cloud native test orchestration and execution platform that supports any testing tool and script you might be using for any kind of test; end-to-end testing, API testing, load testing, and much more. Along with advanced test execution features like parallelization, sharding and parameterization, Testkube also provides centralized management of test results and artifacts for troubleshooting and reporting. 

Integrating Testkube with Argo Workflows

By integrating Testkube to execute tests with Argo Workflows, you can perform tests using any testing framework and gain better insights into your tests. 

How to integrate Testkube with Argo Workflows?

Argo Workflows has the concept of WorkflowTemplates to define reusable templates that you can reuse across your other workflows. We will create a WorkflowTemplate in Argo Workflows for integrating Testkube into our Argo Workflows. This template can be reused by referencing it in `workflow`, making it easier to execute any supported test. 

The below template configures the Testkube CLI to connect to our Testkube environment using the `set context` CLI command. Once configured, we can run any `testkube` command from the Argo Workflow. 

Here is the Argo Workflow Template with Testkube integrated:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: testkube-template
spec:
  templates:
  - name: run-testkube
    container:
      image: "kubeshop/testkube-cli:latest"
      command: ["sh", "-c"]
      args:
        - |
          testkube set context --org-id {{workflow.parameters.org-id}} --env-id {{workflow.parameters.env-id}} -c cloud -k {{workflow.parameters.api-key}}
          testkube {{workflow.parameters.testkube-cli-command}}

In this `WorkflowTemplate`, we have defined:

• templates.container.image: Image that will allow the execution of the Testkube command.
• templates.container.args: Contains the `testkube` command to set context specific to your account and run the CLI command provided by the user. In the `set context` command, we have passed the environment variables `org-id`, `env-id`, and API Token. You can get the required environment variable values for your account from the Testkube Dashboard Settings. The CLI command can be provided as a parameter via `Workflow`.

Test Orchestration with Testkube in Argo Workflows

In a previous blog, we automated acceptance testing with Robot Framework in the Testkube Dashboard using the Test Workflows. We will enhance the same `TestWorkflow` to perform parallel testing in Robot Framework using Testkube and create it in the Testkube Dashboard. 

Using the Argo Workflow Testkube template, we will create a workflow that will execute this TestWorkflow. This automation will help you understand how seamlessly Testkube integrates the execution of testing tools like Robot Framework in the Argo Workflow. Here are the prerequisites.

Prerequisites

• Kubernetes Cluster
• A Testkube Pro account
• Install Argo Workflows on the cluster
• Install Testkube on the cluster

Once you have a cluster running, and you have installed Testkube and Argo Workflows, run the following commands to verify all the resources are deployed successfully:

For Argo Workflows:

$ kubectl -n argo wait deploy --all --for condition=Available --timeout 2m
deployment.apps/argo-server condition met
deployment.apps/workflow-controller condition met

For Testkube:

$ kubectl get pods -n testkube
NAME                                                    READY   STATUS    RESTARTS      AGE
testkube-api-server-744f7595f6-7ptw6                    1/1     Running   3 (19m ago)   20m
testkube-nats-0                                         2/2     Running   0             20m
testkube-operator-controller-manager-74ff57886c-wq75k   2/2     Running   0             20m

Argo Workflows and Testkube are successfully deployed.

Create a Test Workflow in the Testkube Dashboard

Testkube offers a comprehensive Dashboard where you can manage your Test Workflows and their execution. Here is the enhanced Test Workflow for running the parallel test on the Restful Booker using the Robot Framework. In this TestWorkflow, we have used variables in Robot Framework and matrix feature to run parallel tests for the same test case.

We have already created the Test Workflow and executed it in the dashboard:

The Robot Framework test for Restful Booker has been executed successfully.

Apply Testkube Argo Workflow Template on the cluster

Let us go ahead and apply the WorkflowTemplate on the cluster.

$ kubectl apply -f workflowtemplate.yaml -n argo
workflowtemplate.argoproj.io/testkube-template created

The `WorkflowTemplate` has been successfully created and now it can be referenced by any Argo `Workflow` for reuse.

Create a Workflow using the Testkube Workflow Template

In this step, we are creating a `Workflow` for Argo Workflows that references our `WorkflowTemplate`.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: run-testkube-from-template
  generateName: testkube-test-1-
spec:
  entrypoint: get-testworkflow
  arguments:
    parameters:
    - name: org-id
      value: tkcorg_xxx  # Replace with your org-id
    - name: env-id
      value: tkcenv_xxx  # Replace with your env-id
    - name: api-key
      value: tkcapi_xxx  # Replace with your api-key
    - name: testkube-cli-command
      value: run testworkflow advanced-acceptance-test --watch # Replace with command to be run
  templates:
    - name: get-testworkflow
      steps:
        - - name: execute-testkube
            templateRef:
              name: testkube-template
              template: run-testkube

In this configuration, we have passed the values of the variables that are needed in the template. Here are the details on the fields set:

• `templates.steps.templateRef` is set to use the `testkube-template` WorkflowTemplate.
• In `spec.arguments.parameters`, `testkube-cli-command` is set to run any Testkube command. We are going to run the TestWorkflow so pass the command for it.

‍

Apply this configuration on the cluster using the following command:

Name:                run-testkube-from-template2
Namespace:           argo
ServiceAccount:      argo
Status:              Succeeded
Conditions:          
 PodRunning          False
 Completed           True
Created:             Wed Oct 09 14:00:52 +0530 (38 seconds ago)
Started:             Wed Oct 09 14:00:52 +0530 (38 seconds ago)
Finished:            Wed Oct 09 14:01:30 +0530 (now)
Duration:            38 seconds
Progress:            1/1
ResourcesDuration:   51s*(100Mi memory),4s*(1 cpu)
Parameters:          
  org-id:            tkcorg_xxxx
  env-id:            tkcenv_xxxx
  api-key:           tkcapi_xxxx
  testkube-cli-command: run testworkflow advanced-acceptance-test --watch

STEP                            TEMPLATE                        PODNAME                                              DURATION  MESSAGE
 ✔ run-testkube-from-template2  get-testworkflow                                                                                 
 └───✔ execute-testkube         testkube-template/run-testkube  run-testkube-from-template2-run-testkube-1911708985  28s 

The Workflow is successfully created which means the TestWorkflow has been successfully executed.

Verify

View the Argo logs using the following command:

$ argo logs -n argo @latest
run-testkube-from-template2-run-testkube-1911708985: Test Workflow Execution:
run-testkube-from-template2-run-testkube-1911708985: Name:                 advanced-acceptance-test
run-testkube-from-template2-run-testkube-1911708985: Execution ID:         67063f44c3264f754b7b6aa4
run-testkube-from-template2-run-testkube-1911708985: Execution name:       advanced-acceptance-test-3
run-testkube-from-template2-run-testkube-1911708985: Execution namespace:  testkube
run-testkube-from-template2-run-testkube-1911708985: Execution number:     3
run-testkube-from-template2-run-testkube-1911708985: Requested at:         2024-10-09 08:31:00.881848272 +0000 UTC
run-testkube-from-template2-run-testkube-1911708985: Disabled webhooks:    false
run-testkube-from-template2-run-testkube-1911708985: Status:               queued
...
run-testkube-from-template2-run-testkube-1911708985:  (SuccessfulCreate) Created pod: 67063f44c3264f754b7b6aa4-jt2sz
...
run-testkube-from-template2-run-testkube-1911708985: • (2/2) Run test
run-testkube-from-template2-run-testkube-1911708985:  2 instances requested: 2 combinations, all in parallel
...
run-testkube-from-template2-run-testkube-1911708985:  worker/2: created
run-testkube-from-template2-run-testkube-1911708985:  worker/1: assigned to testkube-20241009 node
run-testkube-from-template2-run-testkube-1911708985:  worker/1: running
run-testkube-from-template2-run-testkube-1911708985:  worker/2: assigned to testkube-20241009 node
run-testkube-from-template2-run-testkube-1911708985:  worker/2: running
run-testkube-from-template2-run-testkube-1911708985:  worker/2: passed
run-testkube-from-template2-run-testkube-1911708985:  worker/1: passed
...
run-testkube-from-template2-run-testkube-1911708985:  Successfully finished 2 workers.
run-testkube-from-template2-run-testkube-1911708985: 
run-testkube-from-template2-run-testkube-1911708985: • passed in 12.897s
run-testkube-from-template2-run-testkube-1911708985: 
run-testkube-from-template2-run-testkube-1911708985: test workflow execution completed with success in 15.122338653s 🥇
run-testkube-from-template2-run-testkube-1911708985: Test Workflow URI: https://app.testkube.io/organization/tkcorg_b8ddc820d4919590/environment/tkcenv_94cb6305570f69bd/dashboard/test-workflows/advanced-acceptance-test

From these logs, you can see that the Test Workflow has been queued but the logs could be difficult to compare the test results, view if the artifacts were uploaded, and manage the tests overall. Let us head to the Test Workflow URI given in the logs and check the execution details.

So with Testkube, you were able to perform parallel tests using Robot Framework with Testkube in Argo Workflow. In the above snippet, you can easily view the details related to the two workers that were created for the execution of the test. This level of automation is what the developers need for quality delivery on time. Let us see some of the other benefits Testkube offers to manage tests. 

Why use Testkube with Argo Workflows?

Although Argo Workflows is a formidable engine for generic computational workloads like data and integration pipelines, it lacks functionality specific to the needs of Testers and QA in large-scale organizations. 

Testkube has been designed from the ground-up to align with the specific needs of both DevOps and QA in regard to testing in general and Test Execution specifically, and is being architected in a way that is directly aligned with the most popular testing tools and processes being used in the cloud-native ecosystem and testing community.

To that extent, Teskube provides a number of features that are either lacking or poorly supported in Argo Workflows from a test-execution point-of-view:

• A workflow engine and syntax specifically built for Test Execution, with out-of-the box support for orchestrating and executing any existing testing tool/version/script that your testing teams might be using.
• Direct support for advanced features like test parallelization, sharding, parameterization implemented in line with how testing tools like Playwright, Cypress, JMeter and K6 are built to be used in corresponding use cases.
• Extensive test-triggering mechanics including direct integrations with popular CI/CD tools in line with how tests are commonly triggered in cloud-native build and deploy pipelines.
• An advanced RBAC mechanism giving fine-grained control over who can work with Test Workflows and their results, ensuring that multiple teams can work safely together in complex environments without mistakenly causing misconfigurations or unwanted access of sensitive data.
• A user-friendly interface for troubleshooting complex test executions, including the capability to compare test executions and results over time.

• An advanced Test Insights module for both operational and strategic reporting on Test Results across all testing tools and their executions.

Using Testkube to run your tests as part of your Argo Workflows ensures you are using the right tool for the job; Argo Workflows for advanced workflows and data pipelines, Testkube for advanced test execution.

Conclusion

In this blog, we have performed parallel acceptance tests for the Restful Booker with Robot Framework using Testkube in Argo Workflows. With the right set of tools, we were able to perform parallel tests better, compare results, and manage and view the artifacts. You do not have to worry about that framework’s integration support or test management. Developers can trigger and manage any test, and streamline their testing processes from various other trigger sources.

To learn more about how Testkube works with other Argo tools, check out the following tutorials: 

• GitOps-Powered Testing Machine Using ArgoCD and Testkube
• Automate Canary Deployments with Argo Rollouts and Testkube 
• Automating Blue Green Deployments with Argo Rollouts and Testkube

To experience the full potential of testing on Testkube, we invite you to try Testkube today. Witness firsthand how Testkube simplifies and empowers your testing process with its Kubernetes-native test execution capabilities. Join our active Slack community for guidance and support.