Getting Started with Testkube Open Source

Testkube is the go-to framework for developers, testers, and DevOps engineers to seamlessly integrate testing within their GitOps pipelines so that the applications are tested before deployment. 

Testkube consists of two major components:

1. Testkube Agent is an open source component that manages Testkube resources, listens to Kubernetes Events, and performs the entire test execution along with collecting the results and relevant logs. This is managed via Testkube CLI.
   
   
2. Testkube Control Plane is a commercial solution that comes with a life-savior Dashboard to have better control over multiple testing environments. The control plane can be hosted by Testkube or in your existing infrastructure known as Testkube On-Prem.

In this blog post, we will explore the Testkube Agent as a standalone open source alternative for test execution and show how to get started with its deployment on a Kubernetes cluster, execute tests, and automate test execution using a CI/CD tool like GitHub Actions.

Testkube Agent

The Testkube Agent is 100% Open Source and comprises the core Testkube execution and orchestration engine for running tests in your Kubernetes infrastructure. You can deploy Testkube Agent in two modes:

Standalone Mode

In Standalone mode, the Testkube Agent can be managed directly through the Testkube CLI  or directly with the agent API. This mode is free to use and is best suited for users who prefer to control their testing infrastructure without the need for an external control plane.

Connected Mode

Connected mode is commercial and connects your Agent to the Testkube Control Plane, enabling you to manage any number of Testkube Agents from a centralized dashboard. The mode synchronizes test results to the control plane, enabling centralized test management and better insights from analytics and reporting. The connected mode also enables features like complex test orchestration, parallel execution, parametrization, and spawning dependencies.

Features of the Testkube Open Source Agent

The Testkube Agent provides an extensive set of strong features that are aimed at improving your testing experience, thus making it a strong tool for Kubernetes-native testing. Here are some of Testkube Open Source's features:

• Test Workflows: Manage workflows and templates, execute multiple tests concurrently, and integrate with CI/CD tools such as GitHub Actions, Jenkins, CircleCI, and Azure DevOps.
  
  
• Test Logs: Manage test logs generated during test execution and allow teams to gain insights into their testing processes and optimize their testing strategies.
  
  
• Webhooks: Supports real-time integration with third-party tools, enabling seamless interaction between Testkube and other tools, such as incident response and management systems, communication systems, and monitoring and observability tools.
  
  
• Test Triggers: Automates the running of tests based on the occurrence of particular events in Kubernetes resources. Provides concurrency policies that determine how multiple trigger runs are managed.
  
  
• Testkube CLI: Allows users to manage various Testkube resources. You can initiate test executions directly from the command line; also allows viewing the status and logs of test executions.

• Rest API: Exposed by the Testkube Standalone Agent and offers programmatic access to the majority of the features exposed when the agent is in standalone mode. This API does not authenticate by default for its operations; thereby, it might pose a security concern.

For further details on Testkube Agent features, see the Testkube documentation.

Deployment Architecture of Testkube Open Source

The deployment architecture of Testkube is built for effective test execution and orchestration in Kubernetes environments. A high-level deployment architecture of the Testkube Agent in the standalone mode is shown below:

‍

Components of the standalone architecture:

• Kubernetes cluster: This environment provides a framework for deploying the Testkube Agent, along with all the necessary Kubernetes resources and configurations to execute tests.

• Testkube namespace: A dedicated namespace where all the Testkube-related resources, such as Agent API, MongoDB, and CRDs are deployed.
  
• Agent API (Operator): The lifecycle of Testkube resources is handled by the Agent API or Kubernetes Operator.
  
• Test executions (K8s Jobs): When a test is triggered, the Agent API creates Kubernetes jobs to execute the tests. They are launched within the same Kubernetes cluster, taking advantage of the resources in the cluster to run the tests.
  
• MongoDB: The database employed here is MongoDB for storing metadata about tests, test runs, and other Testkube resources.
  
• MinIO: It is an object storage solution for storing test artifacts like reports, logs, and other data that are created when test runs are executed.
  
• Test/Workflows/etc (CRDs): Testkube relies on Custom Resource Definitions (CRDs) for specifying tests, test workflows, and other setups. The CRDs are monitored by the Agent API (operator) for initiating test runs.

For a more detailed description of the deployment architecture, refer to the Testkube documentation.

Installing the Testkube Agent in Standalone Mode

The Testkube Agent can be deployed on any Kubernetes or Openshift cluster, and there is also a Docker-version available. In this demo, we have set up a Kubernetes cluster on Minikube.

Let us deploy Testkube Agent using the following command and see the relevant outputs:

1. Add the Kubeshop Helm repository:

$ helm repo add kubeshop https://kubeshop.github.io/helm-charts
"kubeshop" already exists with the same configuration, skipping

This shows that Kubeshop repo already exists. You can run `helm repo update` to retrieve the latest versions of the packages.

$ helm repo update
Hang tight while we grab the latest from your chart repositories......
Successfully got an update from the "kubeshop" chart repository...
Successfully got an update from the "testkube" chart repository
Update Complete. ⎈Happy Helming!⎈

2. Install the Helm Chart with defaults that will configure Testkube in `testkube` namespace.

$ helm upgrade --install \
>   --create-namespace \
>   --namespace testkube \
>   testkube kubeshop/testkube
Release "testkube" does not exist. Installing it now.
NAME: testkube
LAST DEPLOYED: Mon Feb 17 18:17:48 2025
NAMESPACE: testkube
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Enjoy testing with Testkube!

This installation created the `testkube` namespace since it does not exist on the cluster.

3. Verify the installation as shown below:

$ kubectl get all -n testkube
NAME                                                      READY   STATUS    RESTARTS   AGE
pod/testkube-api-server-7bb4bc55dc-tbcrj                  1/1     Running   0          16m
pod/testkube-minio-testkube-54f9847486-pvv69              1/1     Running   0          16m
pod/testkube-mongodb-8584d5ff8c-sczhz                     1/1     Running   0          16m
pod/testkube-nats-0                                       2/2     Running   0          16m
pod/testkube-operator-controller-manager-fccfbcdf-lbds4   1/1     Running   0          16m‍
NAME                                                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/testkube-api-server                                    ClusterIP   10.96.164.55     <none>        8088/TCP,8089/TCP            16m
service/testkube-minio-service-testkube                        ClusterIP   10.102.225.84    <none>        9000/TCP,9090/TCP,9443/TCP   16m
service/testkube-mongodb                                       ClusterIP   10.100.106.240   <none>        27017/TCP                    16m
service/testkube-nats                                          ClusterIP   10.111.241.122   <none>        4222/TCP                     16m
service/testkube-nats-headless                                 ClusterIP   None             <none>        4222/TCP,8222/TCP            16ms
ervice/testkube-operator-controller-manager-metrics-service   ClusterIP   10.98.254.127    <none>        8443/TCP                     16m
service/testkube-operator-webhook-service                      ClusterIP   10.101.235.142   <none>        443/TCP                      16m‍

NAME                                                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/testkube-api-server                    1/1     1            1           16m
deployment.apps/testkube-minio-testkube                1/1     1            1           16m
deployment.apps/testkube-mongodb                       1/1     1            1           16m
deployment.apps/testkube-operator-controller-manager   1/1     1            1           16m‍

NAME                                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/testkube-api-server-7bb4bc55dc                  1         1         1       16m
replicaset.apps/testkube-minio-testkube-54f9847486              1         1         1       16m
replicaset.apps/testkube-mongodb-8584d5ff8c                     1         1         1       16m
replicaset.apps/testkube-operator-controller-manager-fccfbcdf   1         1         1       16m‍

NAME                             READY   AGE
statefulset.apps/testkube-nats   1/1     16m

With this installation, the Testkube API server is running on the cluster along with MongoDB for test results, Minio for artifact storage, and other CRDs for managing the TestWorkflows, Triggers, Webhooks, etc.

Running Tests with the Testkube Agent

Testkube provides Test Workflows as a declarative approach to defining how to execute your tests. For understanding purposes, we have taken the example from the quickstart documentation and kept it in a GitHub repository.

Install Testkube CLI

For executing the Test Workflow, you will need Testkube CLI. Install Testkube CLI on your machine and verify using the following command:

$ testkube version‍

Context:  (2.1.88)   Namespace: testkube--------------------------------------                            
/tɛst kjub/ by Kubeshop‍

Client Version 2.1.88

Create Test Workflow

Here is a Test Workflow that will execute a k6 test using Testkube on the cluster.

kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
	name: ephemeral-test
    namespace: testkube
    labels:
    	test: k6
        spec:
    content:
    	git:
        uri: https://github.com/cerebro1/testkube-ephemeral
    revision: main
    paths:
    	- k6/testkube.js
    container:
        workingDir: /data/repo/k6/
    steps:
    - name: Run test
    	container:
        	image: grafana/k6:0.49.0
        shell: |
        	k6 run testkube.js
        artifacts:
        	workingDir: /data
            paths:
            	- '*.html’

Save this configuration in a file `testworkflow.yaml` and create Test Workflow on the cluster using the Testkubde CLI as shown below.

$ testkube create testworkflow -f testworkflow.yaml‍

Context:  (2.1.88)   Namespace: testkube
----------------------------------------

Test workflow created testkube / ephemeral-test 🥇

‍

Since Workflows are regular Kubernetes CRDs, you can also create them by simply using kubectl apply -f testworkflow.yaml

Verify the deployment of the Test Workflow:

$ testkube get testworkflows‍

Context:  (2.1.88)   Namespace: testkube

----------------------------------------‍
NAME           | DESCRIPTION | CREATED                       | LABELS  | STATUS | EXECUTION ID
-----------------+-------------+-------------------------------+---------+--------+---------------
ephemeral-test |             | 2025-02-17 14:51:10 +0000 UTC | test=k6 |        

We have successfully deployed Test Workflow on the cluster. Here the `status` and `execution` are empty because we have not yet executed the Test Workflow. Let us go ahead and execute it.

Execute Test Workflow

Testkube CLI provide `run` command to execute a Test Workflow. Let us go ahead and execute the `ephemeral-test` Test Workflow.

$ testkube run testworkflow ephemeral-test‍

Context:  (2.1.88)   Namespace: testkube
----------------------------------------‍

Test Workflow Execution:
Name:                 ephemeral-test
Execution ID:         67b34e1e29a84abec76e0a14
Execution name:       ephemeral-test-1
Execution namespace:
Execution number:     1
Requested at:         2025-02-17 14:56:30.314865414 +0000 UTC
Disabled webhooks:    false
Status:               queued‍

$ Watch test workflow execution until complete \	kubectl testkube watch twe 67b34e1e29a84abec76e0a14

$ Use following command to get test workflow execution details \
	kubectl testkube get twe 67b34e1e29a84abec76e0a14

Here we have queued the Test Workflow for execution. You can also watch the execution by passing the `--watch` flag.

Verify if the Test Workflow has been executed as shown below:

$ testkube get testworkflows
Context:  (2.1.88)   Namespace: testkube
----------------------------------------‍

NAME           | DESCRIPTION | CREATED                       | LABELS  | STATUS | EXECUTION ID              
-----------------+-------------+-------------------------------+---------+--------+--------------------------- 
ephemeral-test |             | 2025-02-17 14:51:10 +0000 UTC | test=k6 | passed | 67b34e1e29a84abec76e0a14  

The Test Workflow has been successfully executed. Testkube performs the following steps in Test Workflow execution:

1. Initializes the environment.
2. Clones the GitHub repository.
3. Runs the shell command for the k6 test execution.
4. Uploads the artifacts if found.

View Test Workflow execution logs

Testkube manages the logs and lets you access them via CLI. You can have a detailed view of the Test Workflow execution using the following command using the executionId shown in the terminal output above:

‍

$ testkube get twe 67b34e1e29a84abec76e0a14‍

Context:  (2.1.88)   Namespace: testkube
----------------------------------------
Test Workflow Execution:
Name:                 ephemeral-test
Execution ID:         67b34e1e29a84abec76e0a14
Execution name:       ephemeral-test-1
Execution namespace:  
Execution number:     1
Requested at:         2025-02-17 14:56:30.314 +0000 UTC
Disabled webhooks:    false
Status:               passed
Queued at:            2025-02-17 14:56:36 +0000 UTC
Started at:           2025-02-17 14:56:36 +0000 UTC
Finished at:          2025-02-17 14:57:17.176 +0000 UTC
Duration:             46.862s
Getting logs for test workflow execution 67b34e1e29a84abec76e0a14‍

• Initializing
Creating state... done
Initializing state... done
Configuring init process... done
Configuring toolkit... done
Configuring shell... done‍

• (1/3) Clone Git repository📦
Cloning into '/tmp/repo'...

POST git-upload-pack (175 bytes)
POST git-upload-pack (264 bytes)
From https://github.com/cerebro1/testkube-ephemeral

* branch            main       -> FETCH_HEAD...‍

• passed in 2.251s‍

• (2/3) Run shell command

	execution: local
		script: testkube.js
		output:
-‍

	scenarios: (100.00%) 1 scenario, 1 max VUs, 40s max duration (incl. graceful stop):

	* default: 1 looping VUs for 10s (gracefulStop: 30s)‍
    
…‍running (10.0s), 1/1 VUs, 9 complete and 0 interrupted iterations
default ↓ [ 100% ] 1 VUs  10s‍

	✓ Response time is less than 1000 milliseconds...‍
• passed in 23.576s‍

• (3/3) Upload artifacts
Root: /data
Patterns:- /data/*.html‍

No artifacts found.
Took 0s.‍

• passed in 24ms

Head over to the Testkube Examples documentation for examples of how to run popular testing tools with Testkube, including Playwright, Cypress, Postman, and JMeter.

Automate Test Execution with GitHub Actions

Testkube provides "Setup Testkube CLI" action that can be directly used in the workflow to manage Testkube installation on a cluster. In this demo, we will automate the execution of the k6 test in the pipeline when a new Pull Request is raised using the following workflow.

name: Ephemeral Test Using Testkube
on:
	pull_request:
    
jobs:
	testkube-test:
    runs-on: ubuntu-latest
    steps:
    	- name: Checkout Code
        uses: actions/checkout@v4
        
     - name: Kind Cluster
     uses: helm/kind-action@v1.12.0‍
     
     - name: Setup Testkube with Helm
     run: |
     	helm repo add kubeshop https://kubeshop.github.io/helm-charts;
        
 ‍      - name: Configure Testkube Agent
 		run: |
        	helm repo update
            helm upgrade --install --create-namespace --namespace testkube testkube kubeshop/testkube‍
            
       - name: Check Testkube Version and Pods Status
       run: |
       	kubectl get pods -n testkube -o wide
        TESTKUBE_POD_NAME=$(kubectl get pods -n testkube | grep testkube-api-server | awk '{print $1}')
        kubectl wait --for=condition=Ready pod/$TESTKUBE_POD_NAME -n testkube --timeout=150s
        MINIO_POD_NAME=$(kubectl get pods -n testkube | grep testkube-minio | awk '{print $1}')
        kubectl wait --for=condition=Ready pod/$MINIO_POD_NAME -n testkube --timeout=150s
        MONGO_POD_NAME=$(kubectl get pods -n testkube | grep testkube-mongodb | awk '{print $1}')
        kubectl wait --for=condition=Ready pod/$MONGO_POD_NAME -n testkube --timeout=150s‍
        
- name: Setup Testkube
	uses: kubeshop/setup-testkube@v1.0.10‍
    
- name: Create and run Test Workflow
	run: |
    testkube create testworkflow -f testworkflow.yaml
    testkube run tw ephemeral-test --watch

In this configuration, we have defined the following steps:

1. Created a kind cluster.
2. Added the `kubeshop` Helm chart.
3. Installed Testkube on the cluster using Helm chart. For scenarios when you want to run your tests from inside your existing infrastructure it is recommended to have the Testkube Agent installed there (in a dedicated namespace) instead of provisioning it inside each execution of your Github Actions Workflow..
4. Checked the status of the Testkube API server, MongoDB, and MinIO pods.
5. Setup Testkube CLI
6. Created Test Workflow on the cluster using the `testworkflow.yaml` file.
7. Executed the Test Workflow.

Let us check how the above workflow executes and automates test execution.

1. Set up a GitHub repository, add the above manifest to the repository in `.github/workflows/main.yaml` file, the Test Workflow we used in the last section, and the k6 test you want to execute. Here is the sample repository.
2. Create a Pull Request and wait for the workflow to be triggered. In the following image, you can see workflow has been triggered.

3. Once the check has been completed successfully, you should be able to see the following status.

4. Let us check the execution details of the k6 test. Click on the check to view details.

Here you can see all the steps have been completed successfully.

5. Select on “Create and run Test Workflow” step to view details.

In the workflow, you can view the entire execution details related to the k6 test and the Testkube Test Workflow. With this, you were able to automate the execution of a k6 test via GitHub Actions using the Testkube Open Source solution.

Conclusion

The Testkube Agent is Open Source and supports automated testing in Kubernetes environments, making it possible for developers and teams to execute tests at scale inside their clusters. Test execution can be scheduled through Testkubes CLI interface or REST API, making it easy to integrate with  CI/CD tools such as GitHub Actions, where tests can be automatically run on code changes. Following the installation and configuration processes, you can start using the Testkube Agent in Standalone Mode and automate end-to-end testing in your workflows, and later connect it to the Testkube Control Plane if the need for more enterprise-grade functionality arises.

Sign up for Testkube today and explore the full potential of its Control-Plane functionality.. Visit the Testkube documentation and learn more about Testkube and its capabilities. If you have any questions, reach out to us on Slack.