Run Kubernetes Tests with SoapUI and Testkube

I am Lilla, and I have been developing software in Go since 2016. I joined [Kubeshop](http://www.kubeshop.io) this April as a backend engineer.

Before joining Kubeshop, I used to work with cloud infrastructure. I did not know much about software testing tools. I was familiar with some types of tests from my university days, just enough so that I could get along well in my bubble. Testing infrastructure code is very different from testing the applications that run on top of it. I *happen to know* that application developers are also struggling when coming to the cloud. The more complex your application is, the more you will have to think about dependencies. You have to see which parts are worth refactoring (sometimes all of it) to fit the cloud.

At Kubeshop, I started working on Testkube, a cloud native testing framework.. My very first task at Testkube was [Issue #216 called "SoapUI executor"](https://github.com/kubeshop/testkube/issues/216). The description says "I want to run my SoapUI tests with kubtest". “kubtest” was an older experimental name for Testkube. I did not know much about either of those when I picked it up, but I was looking forward to learning everything. The brevity of the description gave me lots of freedom. There was enough documentation and examples to get started with the first features. There was another contributor to Testkube executors at the same time. It gave me faith that it is not an impossible task.

In what comes, I will show you how to move and run your SoapUI tests in your Kubernetes cluster using Testkube.

What is Testkube?

[Testkube](https://testkube.kubeshop.io/) is a Kubernetes-native testing framework. It enables running tests directly in your Kubernetes cluster and then saves the results to a centralized space. It abstracts away all the troubles of containerising different test types. Testkube makes testing in Kubernetes a smooth cloud-native experience.

Kubernetes Testing with SoapUI

What I've learnt is that API testing could be troublesome and frustrating at first. Knowing “*what*” to test is not enough. SoapUI simplifies the "*how*" of testing. Testkube gives the answer to the "*where*” - directly in your Kubernetes cluster. The tests created with these two tools are reproducible, easy to read, and well structured. That's exactly what I used to want from my infrastructure tests.

SoapUI covers [multiple test types](https://www.soapui.org/getting-started/):

• Functional Testing
• Load Testing
• Security Tests
• Mocking

I am not going to dive into the features of SoapUI here - the [documentation](https://www.soapui.org/docs/) does a better job at that. Just note that SoapUI, despite the name, is great for testing APIs using a variety of popular protocols and technologies like SOAP, REST, GraphQL, MQTT, JDBC, etc.

The SoapUI executor integrated into Testkube allows for functional testing. It is the most widespread type of tests, and there was already a [Docker image provided by SmartBear](https://hub.docker.com/r/smartbear/soapuios-testrunner). So I needed to add a wrapper around it, and make sure that I called the executor correctly there.

SoapUI tests fit perfectly into Testkube

In a [2021 study published by RedHat](https://www.redhat.com/en/resources/kubernetes-adoption-security-market-trends-2021-overview), 88% of RedHat customers use Kubernetes with 74% in production. Kubernetes is at the base of most modern cloud-native applications. The number of users will keep growing with all the major cloud providers coming out with their own managed version.

Running SoapUI tests in a Kubernetes cluster should be as simple as one command - that’s what Testkube enables. Writing complex scripts for automation is a solution – as well as time consuming and error-prone. It involves updates every time a user discovers an edge case. It is worth doing only when there is a reason to have complete control over the flow.

Some APIs should not be public on the internet. Testing on the same Kubernetes cluster does not need a sophisticated firewall setup. They still have enough distance from the service to emulate incoming requests.

What I find fascinating about Testkube, is that it addresses both these concerns. It executes tests with one command (plus one for creating the test, but that's still O(1), right?), and can run in any Kubernetes cluster. It is a natural step in migrating SoapUI tests to Kubernetes. Especially now that there's an executor for it!

Below you can see how Testkube tests your website and microservices using SoapUI project xmls.

Running your first SoapUI tests

[Installing Testkube](https://cloud.testkube.io) is enough to equip your environment for [SoapUI tests](https://kubeshop.github.io/testkube/executor-soapui/).

As an example, I have created some sample tests in SoapUI to make sure that the website kubeshop.io contains the word “Kubernetes” . I will not detail how to create tests in SoapUI as SmartBear does a good job at that. I called the first test case “Kubeshop”. I added a method to check the response's status code, and whether the word is there. As a second test case, I wanted to check whether Testkube's page contains the word “kubectl”. Saving the project resulted in an [xml file](https://gist.github.com/vLia/869427d4a184ad79c2e514a2a3729b53). We will use Testkube to run these exported tests.

Running this project using the Testkube CLI is as easy as:

• Creating a test with:

-- CODE language-bash --
kubectl testkube create test --file "example-project.xml" --type "soapui/xml" --name example-test

‍

• Starting and following the output of the test

-- CODE language-bash --
kubectl testkube run test -f example-test

• (Optional) Running specific test cases:

-- CODE language-bash --
 kubectl testkube run test -f example-test --args '-I -c "Testkube"'

As the last command shows, Testkube forwards arguments to the SoapUI executor. The full list of accepted parameters is in the official [SoapUI documentation](https://www.soapui.org/docs/test-automation/running-functional-tests/). Keep in mind that these tests are running ultimately in a containerized environment. Thus, some of the original parameters will behave differently, for example -I will likely not be able to prompt the usual way.

SoapUI executor CRD

<p>Behind the scenes, Testkube works with Kubernetes' Custom Resource Definitions (CRDs). Tests and test suites have their own CRDs defined, as well as Executors. An executor is basically a Testkube-specific test runner. There are already many executors implemented to run Cypress, Postman, K6, etc. tests.</p>

The definition of the SoapUI executor CRD looks the following way:

-- CODE language-bash --
apiVersion: executor.testkube.io/v1
kind: Executor # defines the type of the resource
metadata:
  name: soapui-executor # name of the executor
  namespace: testkube # all Testkube resources are in the 'testkube' K8s namespace
spec:
  executor_type: job # the only possible type, tests are run in k8s jobs
  image: kubeshop/testkube-executor-soapui:latest # Docker image
  types:
   - soapui/xml # defines the input types, 'soapui/xml' is the only supported one
content_types:
   - string # test content is a string
   - file-url # test content is in a file
features:
   - artifacts # the logs from SoapUI will appear as artifacts for easier access

‍

SoapUI tests in Testkube UI

<p>Testkube also offers a neat browser app to show you an overview of all your tests. Here's what one of my (finally) successful tests looks like in the dev environment:</p>

The UI also gives you quick access to CLI commands to work with tests and test executions. There's a separate tab for the definition of the test. Clicking on an execution shows further details: logs, relevant files (artifacts) and commands.

Limitations

<p>As mentioned before, running SoapUI in a container limits its functionalities. Opening up a browser with the results, and working with prompts will be challenging.</p>

As of now, using volumes with Testkube tests is not implemented. Therefore, anything related to volumes is not supported:

- mounting a project directory
- getting the generated report directories and files in a volume
- mounting extensions or plugins

Third-party libraries are not supported. For some, like using JDBC drivers, the [Maven executor](https://github.com/kubeshop/testkube-executor-maven) provides a workaround.

Further steps & getting started with Testkube

Now you can leverage existing SoapUI tests in your Kubernetes infrastructure using Testkube.

Sign up to Testkube and try one of our examples or head over to our documentation - if you get stuck or have questions, we’re here to help! Find an answer to your questions in the Testkube Knowledge Base or reach out to us on Slack. We’re eager to hear how you use our integrations!