As testers and developers, releasing quality software at speed is important. Parallel testing is an approach that transforms testing efficiency by simultaneously executing tests in parallel across environments, reducing testing time, optimizing resource usage, and improving scalability. But what if you’re working with Kubernetes applications? In order to perform parallelization in containerized applications, you may come across a few specific hurdles.
In this webinar, we’ll walk you through preparing for parallelization with well-structured test suites and a stable, scalable testing infrastructure - so you and your teams can get started running massive number of tests in minutes, using the tools you know and love.
We’ll cover:
Who should attend?
DevOps Engineers, Software Developers, QA Engineers, SREs - or anyone in-between. This event is perfect for those who wish to dive deeper or are curious about the world of testing applications in Kubernetes.
Hi everyone! I was a couple of minutes late, so sorry for that. I hope you guys can hear me all right. Welcome back to Testkube’s office hours, where we open our virtual office doors to you. My name is Ali, and I am the Developer Advocate at Testkube. Today, we will be talking a little bit about parallel testing and Testkube’s new feature that allows us to perform this practice with ease.
If you are new here, Testkube is a Kubernetes-native testing framework for testers and developers that really helps you orchestrate, execute, and display your tests and test results for your apps that are running in Kubernetes. If you haven't checked it out yet, now is the perfect time since all of our Pro features are available for free upon sign-up. You don't need to do much or put in any credit card details—simply sign in at cloud.testkube.io and start testing, which is pretty neat. If this is your first time joining one of our office hours, we usually cover a testing practice or a testing tool, show you hands-on examples with Testkube, and just have a conversation while answering any questions you might have along the way. Feel free to chime in at any time during the chat, because we are always very happy to hear from you.
Our agenda for today is very short and straight to the point. We are going to quickly brush over what parallel testing is, see what we have to do to set it up in Kubernetes using Testkube, and then look at how to monitor and report the results generated by these parallel tests. At the end, we can leave some room for questions, but as I said, feel free to chime in at any point during this conversation to ask questions or leave comments.
Let’s get right into it and quickly cover what parallel testing actually is. Parallel testing is the practice of executing multiple software tests at the same time, simultaneously. It is the exact opposite of sequential or serial testing as we know it, which requires one test to finish completely before you can start the next one. Normally in the testing world, when we hear the term parallel testing, we immediately associate it with executing the same test simultaneously but across different environments. For example, as we see in the diagram, let's say we have an automated functional test that we want to perform across 50 different browsers, devices, or operating system configurations. In the QA world, parallelizing these tests normally means following an approach of grouping them into test suites or test collections consisting of the same test conditions, and then running that same test across different environments like Chrome, Firefox, and Edge.
Obviously, the main benefit of parallel testing is reduced testing time. Let’s just think about that same scenario where we have to run one test across 50 different environments. If that test takes two minutes to complete, running it 50 times sequentially would equal 100 minutes of total runtime. However, if we were to group them to run in parallel—say, in groups of five—then our total execution time would go down to only 20 minutes. Along with this reduced timeframe comes the advantage of faster bug detection and optimization of our resource usage, because we are intelligently distributing workloads instead of throwing everything together in one massive queue.
When it comes to Kubernetes, though, we can take advantage of cluster features to bring parallelization to the next level. Instead of being limited to running a single test multiple times, we can run multiple tests of completely different test types simultaneously. You are not restricted to just running end-to-end or UI tests; you can also do API tests, load tests, performance tests, or any type really, in parallel with each other. You can genuinely mix and match your testing workflows when you take advantage of Kubernetes capabilities, which in this case means executing native Kubernetes jobs in parallel.
Let's see how this translates using Testkube and Testkube's Test Suites. Commonly in an engineering team, you will have your front-end and back-end divisions. Both of their services are deployed within Kubernetes clusters, and each team is responsible for testing their own work. Naturally, they are not going to be using the same tools. You might have the front-end team using Cypress tests, while the back-end engineers utilize Postman to test their APIs. At the same time, you have the QA team, who is responsible for all of these releases and wants to make sure that everything is in check and every single test completes successfully.
This brings up the question of how to orchestrate all these different types of involved tests. This can be done very easily with Testkube's Test Suites. Each team can run their individual tests against the clusters on their own, and then QA can create overarching test resources to aggregate tests written by all teams so they can be run together or used separately. Test Suites are pretty much just the orchestration of different test steps, which can be done sequentially or in parallel—which is the exact feature I want to show you today. Under the hood, this is defined as a clean JSON structure where you can define one or multiple steps and add delays or other configurations. Managing this is even easier through the dashboard because you get to visually interact with this JSON file. This is pretty simple and allows your QA team to organize workflows without having to worry about all of the complex intricacies happening behind the scenes in Kubernetes.
I am going to share my screen now so we can see what this looks like. Right now, I am signed into Testkube Cloud. In a scenario where you want to test your application against different environments, devices, or simply want to run heavy load testing concurrently, you don't want to test sequentially because it can take too long if you have massive amounts of tests. Let's head over to the Test Suites tab.
We can create a new suite here and call it something like "Load Validation Suite," and then add a description noting that this suite tests different loads on our web app. The cool thing about Testkube is that it is super intuitive; we recently added drag-and-drop capabilities directly through the dashboard. Previously, you could still do all of this through your terminal by creating a JSON file outlining the execution batch process order you wanted your tests to follow. The dashboard feature is essentially a visual JSON builder. Let's add our first test: I'll select a k6 test for load testing. At the same time, I also want to check a service response with a curl test, so I'll add that to the same step. I also want to check an API response, so let's add a Postman test. All of these can run at the exact same time; you just pile them up in a single block. Next, let's say I want to introduce a brief delay, so I will add a delay step and tell it to wait for one second. After that intermission, I'm going to add a bunch of curl tests again—let's add four of them to run simultaneously—just checking if everything is going as it should be. To close things off, I'll finish the pipeline with a final high-intensity k6 test.
By default, the concurrency level for parallel tests in Testkube is set to 10, but you can redefine this ceiling through the terminal using the concurrency option in your command. Now I'll save the configuration, and our test suite is created. We can look at the underlying definition to see that everything is cleanly mapped out in the JSON file, and now we can trigger our first execution.
For monitoring and reporting, Testkube gives you a centralized way to simplify running tests for your apps in Kubernetes. You have real-time access to all of your logs, you can watch live executions as they progress, and you can instantly access the artifacts produced by your testing tools. Looking at this current execution, we can see that it is actively running the first step, firing off the k6, curl, and Postman tests simultaneously. Once those finish successfully, it hits the designated delay, runs the four concurrent curl tests all at once, and will finally close out with the last k6 test.
The first three tests have completed successfully, the delay has processed, and now the four curl tests are running simultaneously. While they are running, you can click on any individual execution to watch its logs stream in real time. As you can see, it is incredibly fast. If we had run this sequentially, it would have taken a lot longer, but in total, this parallelized test suite only took one minute and nine seconds to execute. That is pretty incredible and really easy to set up without having to worry about complex configurations.
We have a question from the chat: "Can these tests run automatically once I set up a suite of tests?" Yes! Testkube allows you to set up automated scheduling for your tests. You can go right over to your settings, open the Scheduling tab, and create scheduled templates to have these suites run under certain conditions or at specific times. You can also leverage our test triggers feature. If you have a web application that you are constantly pushing updates to, you can configure a trigger to automatically run this entire load testing suite every time a new deployment is pushed to your app. This allows you to verify that your APIs are consistently returning the exact status or response you expect. Under the Test Triggers tab, you simply create a new trigger, select your condition, and choose the target Kubernetes resource. As I mentioned, you can trigger workflows by deployments, changes in your ConfigMaps, updates to a service, or any specific Kubernetes event that you want to validate against. This keeps your test suites completely ready for any automated scenario. All of this data is immediately available to you in a centralized area, giving you clean analytics and individual logs for every single test run. If you are running tests like Cypress, SoapUI, or Playwright that produce external artifacts, those files are readily available in the dashboard as well. k6 doesn't naturally produce physical artifacts, which is why I can't show you any here, but for your other tools, you will have access to everything exactly as you would when running a test individually. It doesn't change anything except for the fact that you are grouping everything together for optimal performance.
That is pretty much it, so I am going to head back to the presentation. If you have any other questions, feel free to leave them in the comments or reach out to us directly through Discord. We are always super happy to chat, hear your feedback, and see how we can improve our features for you. As you can see, Testkube makes parallel testing extremely easy thanks to its intuitive UI. Adding and executing tests is incredibly quick, and you get direct access to logs and results in just a few clicks. For a QA manager or a QA lead, this is fantastic for setting up unified testing standards across different development teams.
Make sure to check it out and try it for yourself at cloud.testkube.io. Of course, if you want to stay on top of new features or see us demo them live, make sure to sign up for our bi-weekly office hours. You can also check out testkube.io/learn, where we regularly share code snippets, highlight small features, and explain how to take full advantage of various testing practices with Testkube.
That wraps it up for today. If there is anything else you need, feel free to contact us or reach out to our product leader, Bruno. Thank you so much for joining, and I will see you all in a couple of weeks. Bye-bye!