As teams adopt cloud-native practices and technologies, it is often desirable to migrate to new solutions while keeping existing technology in place during a (sometimes lengthy) transition period. One area commonly a target for these migrations is CI/CD, where existing/legacy approaches don’t always harmonize with the needs of cloud-native application delivery pipelines. More specifically, introducing GitOps and tools like ArgoCD or Flux can pose many challenges in regard to maintaining the integrity of existing (often declarative) pipelines while exploring and perhaps transitioning to a more fluid GitOps approach. 

One aspect of any CI/CD pipeline is test execution, which is crucial to ensuring that code, artifacts, and deployments fulfill both functional and non-functional requirements before they move into production. Running tests consistently from both old/legacy CI/CD pipelines and new ones is key to ensuring the integrity of the deployed applications and the pipelines themselves.

This article will outline how the introduction of a dedicated tool for test execution, Testkube, not only facilitates the migration between any CI/CD solutions but also provides Dev and QA times with other benefits related to centralized triggering, troubleshooting, and reporting for any tests executed in a cloud-native application delivery pipeline. We will use Jenkins and ArgoCD as examples, but the approach outlined here can be applied to any migration/adoption of new CI/CD tooling and workflows.

Testing in a Traditional Jenkins Pipeline

Jenkins is a popular and widely used open source automation server and CI/CD system. It is built on a Java-based architecture, making it compatible across various operating systems, including Windows, Linux, and macOS. Jenkins features a user-friendly web interface that allows for easy setup and configuration, complete with on-the-fly error checks and built-in help. Its extensive plugin ecosystem enables integration with numerous tools, enhancing its functionality and adaptability to various workflows.

It is good practice to run tests as part of your Jenkins pipelines, and with Jenkins GitHub support, it is straight-forward to automate the execution of tests stored in a GitHub repository, for example to validate that the code/artifacts that were built or deployed work as expected. In this section, we are going to execute performance tests with k6 for our application from a Jenkins pipeline.

You can find files related to the demonstration in this repository.

Prerequisites

1. Install k6 - We have installed it on a local machine
2. Install Jenkins - We have set up on a local machine
3. ngrok - Expose Jenkins running on local. Skip this step if you have Jenkins running on a cloud-managed platform.
4. Create a Personal Access Token (PAT) for your GitHub account.
5. Configure a repository webhook on GitHub. Use the URL provided by ngrok and append /ghprbhook to it while adding the payload URL.

Create Jenkins Pipeline

1. Create a new pipeline in Jenkins and provide a name.
   

2. In the pipeline option, select Pipeline Script and then enter the following Groovy script:

pipeline {
  agent any
    stages {
        stage('New Performance Testing') {
            steps {
                script {
                    // Run a script to create test
                    sh """
                    echo 'Check k6 version'
                    k6 version
                    rm -rf k6-test && git clone https://github.com/cerebro1/k6-test/
                    cd k6-test
                    k6 run ./get.js
                    """
                }
            }
        }
    }
}

In this script, we have cloned `k6-test` repository and running k6 test available in `get.js`. This test will send a GET request to 'https://test.k6.io'. 

3. Click on Save and trigger the pipeline.

4. Select the Console Output to view the details.

Once the test is executed, you can view the results in your Jenkins pipeline as shown above. You can see in the output that the repo clone was successful and the test file `get.js` is running.

As you can see, we were able to automate the k6 test execution in Jenkins, and in a real-life scenario you would probably run this test at the end of a Jenkins pipeline that builds and deploys your application before running the test itself. 

One drawback of this approach is that the execution is now tightly coupled to Jenkins and the containing Pipeline; if we wanted to introduce a new CI/CD system to our software delivery pipelines we would have to create new scripts in the corresponding language/framework used by that CI/CD system - leaving us with two separate scripts to run the same test (in different contexts). In this scenario, having a tool dedicated to executing tests would greatly reduce the burden of maintaining these scripts and make it easy for us to adopt novel approaches for delivering and testing our applications.

Migrating Test Execution to Testkube 

Testkube is a platform specifically built for test-execution in heterogeneous cloud-native environments. Testkube decouples test execution from any specific CI/CD solution, allowing you to run any test in a consistent execution environment whenever needed, be it from CI/CD, on a schedule, in reaction to an event or just manually via the CLI or UI. 

Driving better innovation requires you to think out of the box and integrate with solutions that are designed for testing. With a tool like Testkube, not only are you automating test execution the cloud native way, but you are also powering your application with DevOps and GitOps adaptability. 

In our effort to decouple our k6 test execution from Jenkins so we can trigger it from other CI/CD pipelines, we are going to automate its execution via Testkube instead.

Pre-requisites

• Get a Testkube account.
• Kubernetes cluster - we’re using a local Minikube cluster.
• Testkube Agent configured on the cluster.
• Testkube API token, Org ID, and Environment ID: Available in Testkube Dashboard -> Settings.
• Jenkins pre-configured with GitHub. Refer to Using Testkube in Jenkins article for set up.
• Testkube CLI installed

Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube agent configured. 

Creating a Test Workflow

Navigate to the Test Workflows tab and click on “Add a new test workflow”:

‍

‍

This will provide you with four options:

• Create from Wizard - use the wizard to create a Test Workflow.
• Start from an example - use existing k6, cypress, and playwright examples
• Combine existing workflows - use with existing workflows.
• Import from YAML - import your own Test Workflow.

We’ll choose the “Import from YAML” option to create a workflow with the following configuration.

kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
  name: k6-testkube-1
  namespace: testkube
  labels:
    docs: example
spec:
  content:
    git:
      uri: https://github.com/cerebro1/k6-test/
      paths:
        - get.js
  steps:
    - name: Run Tests
      workingDir: /data
      run:
        image: grafana/k6:0.49.0
        env:
          - name: K6_WEB_DASHBOARD
            value: 'true'
          - name: K6_WEB_DASHBOARD_EXPORT
            value: k6-test-report.html
        args:
          - run
          - /data/repo/get.js
      artifacts:
        paths:
          - k6-test-report.html

In this configuration, we have set the value of `git.uri` as the same `k6-test` repository that is configured with Jenkins and set `git.paths` as the test file path which is `get.js`. In the `steps`, we have defined the k6 image to be used, the command to execute the test, and a path to store test artifacts. This is how it looks in the Testkube Dashboard.

Select the "Create” option from the dropdown to create the Test Workflow.

The Test Workflow with the name `k6-testkube-1` is created in the Testkube Dashboard which has the configuration to run the k6 test.

Configure Jenkins with Testkube

To allow Jenkins to communicate with Testkube, we will make use of Testkube CLI. The following command will set the context with which you can trigger the Test Workflow execution.

testkube set context --org-id <MY_ORG_ID> --env-id <MY_ENV_ID> -c cloud -k <MY_API_KEY>

These details are available in Testkube Dashboard-> Settings. We have created a Testkube API token with a week expiration date. We need to add this token in Jenkins to allow the Testkube CLI command execution. 

Add Testkube API Token in Jenkins

Select Manage Jenkins->Credentials and add a new credential of “Secret text” Kind as shown below with API token.

Update Jenkins Script

Since we already have a pipeline configured, let us go ahead and update the Groovy script as shown below.

pipeline {
    agent any

    environment {
        TK_ORG = 'tkcorg_xxxxxxxxxxxxxxxx'
        TK_ENV = 'tkcenv_xxxxxxxxxxxxxxxx'
        TK_API_KEY = credentials("TK_API_KEY")
    }
    stages {
        stage('Testing') {
            steps {
                script {
                    sh 'testkube set context --api-key $TK_API_KEY --org-id $TK_ORG --env-id $TK_ENV'
                    // Run k6-local
                    sh 'testkube run testworkflow k6-testkube-1 -f'
                }
            }
        }
    }
}

Here we have provided two commands in the `steps.script`:

1. Set the Testkube CLI context.
2. Run the Test Workflow with name “k6-testkube-1”. Here we have passed “-f” that will ensure that the pipeline has the complete details of the test execution.

Replace environment variables with details specified in your Testkube Dashboard and click Save and Apply.

Trigger k6 Test from Jenkins

We have successfully configured Jenkins with the k6 test. Let's see how this integration works. 

1. Trigger the Jenkins pipeline and head over to Builds. Select the build we triggered and you will see the pipeline script executed.
   

2. Select the Console Output to view the details.

You can see in the above screenshot Jenkins is integrated with Testkube and the Test Workflow is in execution.

3. In the console output, you can view the status of the k6 test execution using logs as shown below.

This seems way too complex to gain some insights to test, compare with other test executions, and troubleshoot the tests.

4. Let us navigate to the Testkube Dashboard the access the Test Workflow we created.

Here you can get a clear picture of what happened with the test and whether the artifacts were uploaded or not.

5. Select the “Run” to view the test execution details.

6. View the artifacts using the Artifacts tab.

7. Click on the HTML file to directly view the artifacts.

With Testkube integration in Jenkins, you were able to leverage the benefits of a cloud native testing platform that gives you dedicated features to manage your tests easily. Businesses that use Testkube capabilities can improve their testing capabilities and avoid the pitfalls of traditional testing approaches.

As mentioned above, Testkube modernizes your application testing by making it easy to migrate from one CI/CD tool to another. Suppose you now have a requirement to align with GitOps principles and want to use the same tests with ArgoCD without disturbing the current Jenkins pipeline. As shown next, Testkube makes this easy, allowing you to run the same Test Workflow from ArgoCD.

Running Testkube Tests from ArgoCD

Using a modern approach to application development helps reduce the time-to-market and stay ahead of the curve. Thanks to our efforts to migrate our k6 test execution from Jenkins to Testkube above, we are now able to easily run the same k6 test from ArgoCD also.

Since the ArgoCD application configuration is out-of-scope of this blog, check GitOps-Powered K8s Testing Machine: ArgoCD + Testkube for how to set up ArgoCD with Testkube.

We have ArgoCD running on a cluster. In the same k6-test repository, we have created a directory that has an ArgoCD job manifest. This job will trigger the same Test Workflow `k6-testkube-1` which we triggered via Jenkins pipeline.

Note: Make sure you create secrets on your cluster as shown here.

kubectl create secret generic testkube-secrets \
  --from-literal=apiToken=tkcapi_7 \
  --from-literal=environmentId=tkcenv_5 \
  --from-literal=organizationId=tkcorg_3

Create an application using ArgoCD UI and provide the required details. For reference, check Testkube GitOps Tutorial.

We have created an ArgoCD application with the name: “testkube-migrate”.

Once the Application has been synced to the cluster, you will want to run your tests. Testkube allows you to trigger them in multiple ways: using the Testkube CLI, Dashboard, Kubernetes triggers, and even ArgoCD post-sync hooks.

We are going to use ArgoCD post-sync resource hooks to trigger the test execution in Testkube. 

Here is the job that we have defined with name `testkube-execute-testworkflow` :

apiVersion: batch/v1
kind: Job
metadata:
  generateName: testkube-execute-testworkflows-
  namespace: default
  annotations:
    argocd.argoproj.io/hook: PostSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  template:
    spec:
      containers:
      - name: execute-testworkflows
        image: kubeshop/testkube-cli:2.1.19
        env:
        - name: API_TOKEN
          valueFrom:
            secretKeyRef:
              name: testkube-secrets
              key: apiToken
        - name: ENVIRONMENT_ID
          valueFrom:
            secretKeyRef:
              name: testkube-secrets
              key: environmentId
        - name: ORGANIZATION_ID
          valueFrom:
            secretKeyRef:
              name: testkube-secrets
              key: organizationId
        command:
        - /bin/sh
        - -c
        - |
          testkube set context \
            --api-key ${API_TOKEN} \
            --root-domain testkube.io \
            --org-id ${ORGANIZATION_ID} \
            --env-id ${ENVIRONMENT_ID}

          testkube run tw k6-testkube-1 -f
      restartPolicy: Never
  backoffLimit: 2

This Job will spin a container that connects to the configured Testkube account and executes the same Test Workflow `k6-testkube-1` that we executed via the Jenkins pipeline. The Job is defined in the Git Repository in this file, so it can be synced into our cluster by ArgoCD. Argo will trigger the Test Workflow as shown below.

Select the executing job `testkube-execute-testworkflow-` and view the Logs.

Yayy!! You can see the `get.js` k6 test running successfully on the cluster using ArgoCD and Testkube.

Let us check the Testkube Dashboard to see how it records triggers from different pipelines of the same Test Workflow. In the Executions, you can see which of your tests were triggered from the Jenkins and ArgoCD pipeline. This was automatically handled by Testkube.

‍

This integration enables you to keep your Jenkins pipeline and still integrate with the GitOps tool like ArgoCD with Test Workflow in Testkube helping you migrate easily.

Benefits of decoupling Test Execution from CI/CD 

As shown in the article so far, Testkube has helped us decouple test execution from a specific CI/CD tool making it easy to onboard any CI/CD tool while maintaining test integrity across multiple tools and pipelines. 

Above that, Testkube brings more benefits to the table, including

• Centralized Dashboard: Trigger tests from Jenkins, ArgoCD, or any other CI/CD tool, or manually from Testkube, you can view the execution details from a single point of reference which is the Testkube Dashboard. Here you can troubleshoot and report on your test-executions in a single pane-of-glass no matter how they were triggered.

Here, we have shown the “k6-testkube-1” Test Workflow that was triggered by Jenkins and ArgoCD to perform the test for an application.

• Decouple testing from infrastructure: Now, suppose you have a QA engineer who has to perform testing. You do not have to worry about giving them access to all the deployments like ArgoCD or Jenkins. You can give them controlled access to the Testkube and let them play around the test easily. Here is a document that can help you configure your organization member’s access to Testkube.

Conclusion

In this blog, we have seen the traditional approach of automating tests with a single CI/CD tool like Jenkins where test execution is tightly coupled to CI/CD, limiting our possibility to explore and adopt new approaches to CI/CD and test-execution.

Testkube provides a cloud-native testing control plane that decouples test execution from CI/CD which allows us to run the same tests easily from multiple CI/CD tools like Jenkins and ArgoCD. Testkube further democratizes testing by providing a single pane-of-glass for test troubleshooting and reporting, which makes our application development and testing more efficient as the complexity of our infrastructure increases. 

Head over to testkube.io to learn more, sign up for Testkube to get started, and join our Slack community to start a conversation, or read the testkube documentation to see how Testkube can improve your Continuous Testing and TestOps processes. 

The future of software testing is here. Are you ready to be a part of it?