Leveraging Testkube for Complex System Testing

Applications today span multiple servers and services which requires a multifaceted approach to ensure reliability and performance. Testing such distributed applications has its own challenges due to their inherent complexity. 

To perform comprehensive testing of such applications, you must run various functional and non-functional tests. Moreover, different load, API, and UI tests should preferably be executed simultaneously to ensure consistent system behavior under complex usage scenarios and provide a thorough validation of your system before it goes into production.

However, without proper tools, managing multiple tests simultaneously can be difficult. Coordinating with different types of tests, analyzing their results, and maintaining consistency is difficult. 

This learn article will show how Testkube can help you create custom Test Workflows combining multiple tests for seamless system testing.

Challenges With System Testing

Performing a system test for your application is crucial to ensure it always works as expected. This often involves running different tests, including unit, functional, and non-functional tests. Managing these tests and integrating multiple tools to replicate a real-world scenario is challenging. Let us look at some other challenges with system testing.

• Complexity: As the number of tests and their types increases, managing and analyzing them becomes complex. Furthermore, when each test is to be performed by a different tool, it is even more complex and demands advanced tools.
• Resource Optimization: Running multiple tests simultaneously means increased resource usage and requires careful orchestration of tests and allocation of resources. 
• Logs & Analysis: With so many tests running simultaneously, getting a complete picture of the test outcome can be difficult. Teams struggle to collate results from different tests and environments without proper tools. This inefficiency can lead to missed bugs and affect the application quality. 

You need a versatile tool to orchestrate and manage multiple tests to overcome the challenges of performing comprehensive system testing with different tests. Enter Testkube.

System Testing with Testkube

Testkube is a Kubernetes-native testing framework that makes end-to-end testing in Kubernetes a breeze. Using Testkube, you can orchestrate and automate complex testing workflows using different testing tools, all from a single intuitive UI. 

Benefits of Using Testkube For System Testing

Testkube enhances system testing by integrating seamlessly with Kubernetes, allowing teams to leverage its full potential for running your tests: 

• Orchestrate Complex Test Workflow: Test Workflows allow you to define complex workflows that enable sequential and parallel test executions to mimic real-world scenarios. Testkube does so without complex scripting and facilitates the creation of detailed test workflows, allowing for better control and customization of test executions.
• In-Cluster Test Execution: Unlike other testing frameworks and tools, Testkube executes your tests within the Kubernetes clusters, ensuring a secure and production-like environment and thus improving the reliability of your test outcomes.
• Leverage your own infrastructure: You can run Testkube on your existing infrastructure, which helps maintain consistency across test and production environments and decreases infrastructure costs.
• Integration with Testing Tools: Testkube integrates with all popular testing tools, including k6, Cypress, and Postman, to name a few. Furthermore, Testkube allows you to combine tests for any of these tools in any way required to perform your system tests.

With these out-of-the-box benefits, Testkube simplifies the orchestration of complex system testing scenarios with a drastically shorter implementation time than other approaches like CI/CD-based solutions or DIY frameworks.

Building A System Test Workflow Using Testkube

With Testkube, you can not only create standalone test workflows but also combine different test workflows that use different tools to run sequentially or in parallel. To put it simply, you can run a system test, load test, and API test all at the same time.

Let’s have a look at how this can be done with Testkube. We’ll create a similar scenario where we’ll have a cURL test to login a user as the first step. The test fetches the access token and passes it on to the next step. Next, we configure Cypress, k6, and Postman tests to run parallelly. All these tests will use the token to perform the tests.

To summarize, we have created the following test workflows: 

• cURL - to authenticate a user, fetch the access token, and save it in a file.
• Cypress - to perform end-to-end testing of the application.
• Postman - to perform API testing for the application.

Pre-requisites

• Get a Testkube account.
• Kubernetes cluster - we’re using a local Minikube cluster.
• Testkube Environment with Testkube Agent configured on the cluster.
• Some Test Workflows used to test all aspects of your application - we’re using Cypress, Postman, and distributed k6 test workflows. Check out our documentation to learn how to create a test workflow.

Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube agent configured and some Workflows ready for execution. 

This video below provides a visual guide to the concepts we'll be exploring in the following sections.

Creating a System Test Workflow

Login to Testkube, navigate to the Test Workflows tab for your local environment, and click the “Add a new test workflow” button.

This will provide you with four options:

• Create from Wizard - use the wizard to create a Test Workflow.
• Start from an example - use existing k6, cypress, and playwright examples.
• Combine existing workflows - use with existing workflows.
• Import from yaml - import your own Test Workflow.

We’ll choose the “Combine existing workflows” option to create this custom workflow and choose the existing test workflows we’ve created. 

In the new tab, provide the name for the test workflow and click on the “Add the first workflow” button. Testkube provides an easy-to-use and intuitive interface for creating test workflows. This will give you a list of test workflows that you have already created, and you can choose one from the list. We’ll choose a cURL test workflow that we created, which tests login. 

After adding your first test workflow, Testkube will allow you to add more test workflows to execute in sequence or parallel. You can click the “+” buttons on either side of the current test workflow to add a new test workflow. Let us add a “cypress-workflow.”

Similarly, let’s add “distributed-k6” and “postman-example” test workflow, but parallelly so that we have cypress, k6, and postman test workflows execute in parallel.

Finally, you’ll have something like this - a cURL test followed by cypress, k6, and postman that will run parallelly. Click on the “Next” button to view the spec file it generates. 

Passing Parameters Between Tests

A common need in System testing is to reuse output from one test as input to other tests. In this case, our initial test authenticates a user. The resulting authentication token is then passed to all subsequent tests to ensure those are all running as the same user account. 

Let’s do just that in our generated Workflow: we will modify the spec generated and add an initial cURL test step that adds “token” as the config variable that will be passed by the cURL test to the other tests. Below is the updated spec file.

kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
 name: end-to-end-test-workflow
 namespace: testkube
spec:
 container:
   env:
   - name: USERNAME
     value: emilys
   - name: PASSWORD
     value: emilyspass
 steps:
 - name: curl-emilys
   run:
     image: curlimages/curl:8.7.1
     shell: |
       curl -s -X POST https://dummyjson.com/auth/login \
        -H "Content-Type: application/json" \
        -d '{
          "username":  "'"$USERNAME"'",
          "password":  "'"$PASSWORD"'",
          "expiresInMins": 30
        }' | grep -o '"token":"[^"]*"' | sed 's/"token":"\([^"]*\)"/\1/' > /data/http_output.txt
 - execute:
     workflows:
     - name: cypress-example
       config:
         token: '{{ file("/data/http_output.txt") }}'
     - name: postman-testkube
       config:
         token: '{{ file("/data/http_output.txt") }}'
     - name: distributed-k6
       config:
         token: '{{ file("/data/http_output.txt") }}'
status: {}

Let us understand what the above spec file does:

• It configures the “USERNAME” and “PASSWORD” as environment variables for the user to perform authentication.
• The cURL test defines the end point for authentication and stores the received token in a file named http_output.txt
• Under workflows, we configure `token: '{{ file("/data/http_output.txt") }}' that takes the token from http_output and passes it as a variable to the other test workflows.

Executing a System Test Workflow

The spec file lists down all the test workflows that we chose from the UI and the order of their execution. Plus it now has the config parameters that we added. Click on the “Create and Run” button to create and execute the test workflow. You’ll see that your custom workflow has started executing. 

You can click on an individual workflow to see its status. For instance, we can check our cypress-example’s execution and see that it has fetched the token from the cURL test and executed the steps successfully.

Similarly, we can check the status and logs for each of the test workflows. Once the entire test workflow has finished executing, you’ll see the status in the UI. In this case, we had a failed Postman test workflow. 

That’s how you can create end-to-end system test workflows by combining multiple test workflows to reproduce a realistic, close-to-product scenario to test your application. 

Summary

One of the most difficult things in testing is to test in production-like environments. It’s difficult to create complex test scenarios that replicate a real-world scenario. Furthermore, if you use different testing tools, it’s nearly impossible to configure all of them to work in tandem and perform comprehensive system tests on your application. 

That’s where Testkube shines by allowing you to create your own test workflows through an intuitive UI and execute those workflows from inside your Kubernetes cluster(s). Irrespective of what testing tools your test workflows use, you can combine all of them to run sequentially or parallelly to perform system testing to test your application.

We would love to hear all about the custom test workflows that you created using Testkube. If you face any issues, remember that the entire Testkube team, plus a vibrant community of fellow Kubernetes testers, are on Slack. We’re just getting started in building the most comprehensive (and friendliest!) cloud-native testing framework for Kubernetes so feel free to follow us on Twitter @testkube_io.