Distributed Load Testing with JMeter in Testkube

Aug 21, 2024
read
Bruno Lopes
Product Leader
Testkube
Read more from
Bruno Lopes

Table of Contents

Start your free trial.

Aug 21, 2024
read
Bruno Lopes
Product Leader
Testkube
Read more from
Bruno Lopes
Bruno Lopes
Product Leader
Testkube
Here we show how Testkube+JMeter can be leveraged to simplify the distributed load-testing process, resulting in efficient resource allocation, horizontal scalability, & simplified test orchestration.

Table of Contents

Executive Summary

ExDistributed Systems face various challenges due to the system’s complexity, such as partial system failures, data inconsistency, deadlocks, high latency, and packet loss. These challenges can often be proactively addressed by proper functional and non-functional testing, including distributed load-testing to ensure your system can handle a high number of users at any given time. Using tools like JMeter, you can simulate real-world conditions and failures to ensure robustness and reliability.

However, simulating production-like conditions often requires running load tests in a distributed setup, generating load from multiple sources, which is resource-intensive and requires careful orchestration of tests and resources. Thus, you need specialized tools to make that process simpler. 

In this blog, we will show how Testkube can be leveraged together with JMeter to simplify the distributed load-testing process, resulting in efficient resource allocation, horizontal scalability, and simplified orchestration of tests.

Distributed Load Testing with JMeter

Distributed testing is the practice of executing tests concurrently from multiple machines or environments. The load is distributed across multiple nodes to more closely simulate real-life usage scenarios. This method is beneficial for testing complex systems, cloud-native, and large-scale applications like banking or e-commerce websites. 

In this section, we will discuss JMeter, a distributed load-testing tool, dig deeper into its distributed systems testing architecture, and understand the associated challenges.

What is JMeter?

Apache JMeter is an open source software for distributed, performance, and load testing of applications. JMeter supports protocols like HTTP/HTTPS, FTP, SOAP, JDBC, TCP, UDP, and much more, allowing it to support the testing of different applications. It enables distributed testing with controller-worker configuration, spreading the load across multiple nodes. This enables an extensive test of an application’s behavior under heavy traffic. Let us understand the controller-worker architecture supported by JMeter for distributed testing.

JMeter Controller/Worker Architecture

In the controller-worker architecture of JMeter, multiple users are simulated across various machines. This setup helps test applications where you need to generate significant loads but have hardware limitations. 

‍

Topology of jmeter distributed tests
Distributed Test Topology, Source: JMeter Distributed Testing

Here is an overview of each component in this architecture:

  • Target: An application, service, or servr under test.
  • Controller Node: A controller that manages the execution of the test. It initiates the test execution, distributes the load, and collects the results from the workers.
  • Worker Nodes: A load generator that takes requests from the controller, generates the load, and executes the test on the target.

How Does JMeter Execute a Test on a Target?

There is one controller that handles the execution of tests using multiple remote workers. For proper communication, the controller, worker, and target must be on the same network. The controller communicates with the workers using Java RMI (Remote Method Invocation). Each of these workers generates a load and executes tests. The test execution by all the workers on the target starts at the same time. The execution details are sent back by the workers to the controller, which performs the result aggregation.

Challenges With Distributed Testing with JMeter

While this setup seems easy, there are some challenges associated with it:

  • Resource allocation overhead: Even though the controller-worker architecture distributes the load, which reduces the chances of overloading a single worker, it is quite challenging to allocate resources to the workers for optimal usage.
  • Complex controller-worker configuration: Ensuring the same version of JMeter on all the nodes, configuring the network to allow RMI traffic, and synchronizing time between nodes are complicated.
  • Monitoring and debugging issues: JMeter requires configuring external plugins or tools for real-time analytics or monitoring. With a large-scale system setup and complicated configurations, this becomes more of an overhead for the team.

Thus, for performing distributed load testing on complex systems in a cloud-native environment, you require a tool to help with optimum resource allocation, easy configuration of controller and workers, scalability, and proper observability. The good news is that Testkube can do it all and gives you both a Dashboard and command line utility for test orchestration. 

In the following section, we will take a detailed look at the features of Testkube that can help achieve better test orchestration for JMeter distributed testing. 

Automate JMeter Distributed Load Testing with Testkube

Testkube is a Kubernetes-native testing platform that orchestrates and executes tests for cloud-native applications. It allows you to automate test execution irrespective of your testing framework, tool, or script using a powerful Test Workflows engine, and a unified dashboard for centralized test creation, execution, and result aggregation, helping you manage tests better and gain observability in overall testing.

How Does Testkube Automate JMeter Distributed Load Testing?

Testkube supports all popular testing frameworks, including JMeter, for distributed load testing. Testkube runs the test directly in your Kubernetes cluster, ensuring secure execution of your system or new features. Here are some of the benefits that Testkube offers when it comes to running distributed JMeter tests:

  • Easy RMI configuration: Testkube simplifies the major challenge of configuring RMI in JMeter’s controller-worker architecture by automating the network configuration needed, such as firewall and port setup. It also automates the installation of the same JMeter version on all nodes, saving time and reducing the risk of error. Testkube provides a sample configuration template that you can use to get started easily.
  • Scalable system design: With Testkube, you can leverage Kubernetes native scaling capabilities and dynamically scale the number of workers without changing your test scripts. Testkube uses Kubernetes resource management to distribute load. This way, you can easily simulate a large amount of traffic to test your system or application without worrying about resource allocation.‍
  • Centralized monitoring dashboard: While testing, it is important to have the ability to view the details of previous test executions and current ones for comparison. In the case of JMeter, you also need test execution details, errors, and performance metrics per worker. The Testkube Dashboard aggregates all the test executions in real-time and helps you track them for easy monitoring and debugging.

Testkube handles all the complexities of performing distributed tests in JMeter. This lets you focus on developing the application and testing rather than figuring out the infrastructure. The next section walks through JMeter distributed testing in Testkube

.

How Do You Execute JMeter Distributed Testing in Testkube?

Using Testkube Test Workflows, we are going to execute the test with JMeter. We have created a JMeter test that runs on our Testkube website and performs the distributed test. Let us get started with the prerequisites and create a Test Workflow to execute the test.

Prerequisites

The basic requirements while configuring the controller-worker architecture in JMeter with Testkube are as follows:

Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube runner.

Creating a Test Workflow

In the Testkube Dashboard, we are going to create a Test Workflow.

  1. Log in to Testkube and select Workflows from the left menu bar.
  2. Click on “Add a new Test Workflow” and select “Start from an example”.
Setting up test workflow in Kubernetes
  1. Scroll to the right and select “Distributed JMeter”. Testkube loads an example YAML.

‍

  1. We are going to update this YAML to also process the JMeter artifacts. Provide the YAML given below:
kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
  name: distributed-jmeter-example-config-artifacts
  namespace: testkube
spec:
  config:
    slavecount:
      type: integer
      default: 3
  content:
    git:
      uri: https://github.com/kubeshop/testkube
      revision: main
      paths:
      - test/jmeter/executor-tests/jmeter-executor-smoke.jmx
  container:
    workingDir: /data/repo/test/jmeter/executor-tests
  services:
    slave:
      use:
      - name: distribute/evenly
      count: config.slavecount
      logs: always
      timeout: 30s
      image: anasoid/jmeter:5.6-plugins-21-jre
      command:
      - jmeter-server
      - -Dserver.rmi.localport=60000
      - -Dserver_port=1099
      - -Jserver.rmi.ssl.disable=true
      readinessProbe:
        tcpSocket:
          port: 1099
        periodSeconds: 1
  steps:
  - name: Run tests
    run:
      image: anasoid/jmeter:5.6-plugins-21-jre
      shell: |
        jmeter -n \
          -X -Jserver.rmi.ssl.disable=true -Jclient.rmi.localport=7000 \
          -R {{ services.slave.*.ip }} \
          -t jmeter-executor-smoke.jmx \
          -j /data/artifacts/jmeter.log \
          -o /data/artifacts/report \
          -l /data/artifacts/jtl-report.jtl -e
    artifacts:
      paths:
      - /data/artifacts/**/*
status: {}

You can view the same running in the Testkube Dashboard as shown below.

Let us split the above file into parts to understand the changes specific to JMeter.

  1. Defining the slaves to have a dynamic value: Here we have used config to define the worker count. The name of to define the slave count. The name of the field is set as slavecount of type integer. We have set the default value as 3. This will be updated with the value you provide at the time of execution. Testkube allows easy scalability without touching the test script.
spec:
  config:
    slavecount:
      type: integer
      default: 3
  1. Path to the test that will run on target: Using the content, we have defined the GitHub repository path that has the test that will be executed on the target. In this test, the target is the Testkube website: testkube.kubeshop.io. With this, Testkube helps us version control the tests.
content:
  git:
    uri: https://github.com/kubeshop/testkube
    revision: main
    paths:
    - test/jmeter/executor-tests/jmeter-executor-smoke.jmx
container:
  workingDir: /data/repo/test/jmeter/executor-tests
  1. Configuring the slave: Testkube creates JMeter workers as Kubernetes pods, defined as a service named slave in the YAML. The worker count decides how many pods to spin up. Here, we have passed the JMeter image to be used while creating the containers in the pod. Also, we have defined the command that will run when each container spins up.
  services:
    slave:
      use:
      - name: distribute/evenly
      count: config.slavecount
      logs: always
      timeout: 30s
      image: anasoid/jmeter:5.6-plugins-21-jre
      command:
      - jmeter-server
      - -Dserver.rmi.localport=60000
      - -Dserver_port=1099
      - -Jserver.rmi.ssl.disable=true
      readinessProbe:
        tcpSocket:
          port: 1099
        periodSeconds: 1

This is the minimal configuration needed, and it is already part of the example that loads in the Testkube Dashboard when you select to get started with an example. Testkube internally does the firewall configuration to ensure secure and seamless communication between worker and controller.

  1. Running test and processing the artifacts: The final step is to define the test. In the controller-worker architecture, the controller initiates the test execution on the workers using the jmeter command with arguments that allow it to connect to the created workers. We also add arguments for generating a report for our execution, which we then capture using the artifacts configuration.
steps:
  - name: Run tests
    run:
      image: anasoid/jmeter:5.6-plugins-21-jre
      shell: |
        jmeter -n \
          -X -Jserver.rmi.ssl.disable=true -Jclient.rmi.localport=7000 \
          -R {{ services.slave.*.ip }} \
          -t jmeter-executor-smoke.jmx \
          -j /data/artifacts/jmeter.log \
          -o /data/artifacts/report \
          -l /data/artifacts/jtl-report.jtl -e
    artifacts:
      paths:
      - /data/artifacts/**/*

We used the sample provided by Testkube to run distributed testing in JMeter and tweaked it to process the artifacts.

  1. From the dropdown, select “Create” to create the Test Workflow.
using jmeter bin for testing
  1. We set the default worker count to 3. At the prompt, enter the number of workers you want to create and click on “Run”. We have entered the value as 6 here.
Starting the test
  1. Testkube does the automatic resource allocation and creates 6 worker nodes along with performing the test execution as shown below. 
automated test execution plan

After the test execution is completed successfully, you can view the execution logs of each worker by selecting it in the Testkube Dashboard. The image below shows the execution logs of the first worker (slave #1).

command line results

Viewing the artifacts

Artifacts help with the analysis of an application under heavy load. Testkube integrates with JMeter and loads the JMeter report UI, which gives a detailed view of the application test. In the Testkube Dashboard, click on “Artifacts”.

From the artifacts dropdown, select report and click on index.html.

more info on test results

Testkube loads the JMeter Dashboard as shown below. This has a detailed view of the Application Performance Index, Requests Summary, Statistics, etc., which you did not have to set up explicitly. Testkube configures all this for you, making it easier for you to gather results.

Apache JMeter Dashboard

Testkube provides utilities to make testing easier so you can focus on application development.

Executing the Test Workflow using Testkube CLI

Testkube provides the ability to connect to the Testkube account using the command line. In the previous section, we created and executed the Test Workflow from the Testkube Dashboard. We will learn here how to manage the Test Workflows using Testkube CLI. Install the Testkube CLI on your machine and configure the API Token. Once the context is set, you can view your Test Workflows and run them using the following command:

testkube run tw <testworkflow_name> --config <item=value>

For our use case, here is what the execution from the CLI looks like:

$ testkube run tw distributed-jmeter-example-artifacts --config slavecount=4
Context: cloud (1.17.54)   Namespace: testkube   Org: SONALI SRIVASTAVA-personal-org   Env: SONALI SRIVASTAVA-personal-env
--------------------------------------------------------------------------------------------------------------------------
Test Workflow Execution:
Name:                 distributed-jmeter-example-artifacts
Execution ID:         66a2315f7fc8371cd690a9d3
Execution name:       distributed-jmeter-example-config-artifacts-2
Execution namespace:  testkube
Execution number:     2
Requested at:         2024-08-13 21:05:03.083826997 +0000 UTC
Status:               queued

You can view the same running in the Testkube Dashboard as shown below.

Kubernetes distributed test worflow

This level of automation, from creating to executing the Test Workflow, allows you to work on the test and leave the execution to Testkube. With Testkube, we could run the JMeter test with minimal configuration and view the application performance-related metrics.

Conclusion

JMeter is a commonly used tool for distributed load testing. Testkube abstracts the complexities associated with it and handles the network configuration, resource allocation, and processing of the artifacts. In this blog, we have seen, using an example available in the Testkube Dashboard, how we can easily set up the controller-worker architecture for distributed testing of an application with JMeter.

Using the Testkube Dashboard, we were able to configure and execute distributed JMeter tests easily, and Testkube helped us view the execution logs of each worker and an aggregated results report that helps debug issues quickly.

By leveraging the Kubernetes features, Testkube simplifies configuring workers and lets you set the number of workers dynamically. Testkube also supports k6 for distributed testing. So, to standardize testing for you, we invite you to try Testkube today. Witness firsthand how Testkube simplifies and empowers your testing process with its Kubernetes-native test execution capabilities. Join our active Slack community for guidance and support.

Frequently Asked Questions

How many workers do I need for JMeter distributed testing?

It depends on your target load and how much one worker can generate. Run a single worker, raise threads until CPU or memory saturates, then divide your target concurrency by that number. In Testkube, the worker count is a config parameter, so you can change it per run without editing the test plan.

Do the JMeter controller and workers need the same version?

Yes. The Apache JMeter remote testing guide says to run the same JMeter and Java versions on every node, because mismatches can break RMI communication. Testkube avoids version drift by starting every worker from the same container image defined in the Test Workflow, such as anasoid/jmeter:5.6-plugins-21-jre in this example.

Which ports does JMeter remote testing use?

JMeter workers listen on the RMI registry port, 1099 by default. RMI also opens random ports unless you pin them, so set server.rmi.localport on workers and client.rmi.localport on the controller. The Test Workflow in this post uses 1099 and 60000 on the workers and 7000 on the controller.

Do I need SSL for JMeter RMI traffic?

Since JMeter 4.0, RMI uses SSL by default, which means generating and sharing a keystore across nodes. Inside a private cluster network, many teams set server.rmi.ssl.disable=true on the controller and workers, as this example does. Keep SSL enabled if test traffic crosses networks you do not trust.

Can Testkube run distributed load tests with tools other than JMeter?

Yes. Test Workflows can distribute any tool that runs in a container. The Testkube docs include distributed examples for k6 and Locust, and the same services and parallel step patterns apply to other load testing tools. Results from each tool are collected as artifacts in the Testkube Dashboard.

{{banner-get-started}}

Bruno Lopes
Product Leader
Testkube
Read more from
Bruno Lopes

About Testkube

Testkube is the open testing platform for AI-driven engineering teams. It runs tests directly in your Kubernetes clusters, works with any CI/CD system, and supports every testing tool your team uses. By removing CI/CD bottlenecks, Testkube helps teams ship faster with confidence.
Get Started with a trial to see Testkube in action.