Testkube 0.7.0: Spit and polish and then some

Hear ye, hear ye, Testkube has a new release out - with something to show off.

• The dashboard received a visual overhaul. Check it out at https://demo.testkube.io - we’d love to get your feedback on our Discord channel.
• Result artifacts produced by tests are now available via the kubectl plugin and dashboard. For example, videos recorded during Cypress test execution appear here.
• Long-running test output can now be followed in real-time via the CLI. No more waiting until the test execution finishes!

And under the hood we’ve done our homework:

• Script executions are now 100% Kubernetes jobs.
• Test artifacts are scrapped from the execution pod and stored in a Minio bucket.

We also add a bunch of fixes:

• Executors sub command which allows users to create, delete and list executors.
• Scripts delete fix.
• Executions watch.
• Artifacts list and download.
  

If you’re new around here, Testkube provides a Kubernetes-native framework for test definition, execution, and results. It can easily run tests created with popular testing tools like Postman or Cypress, as well as run simple Curl commands or use a custom Testkube executor you’ve built yourself. Learn more about Testkube on Github and explore our Get Started guide.

On to the details:

Dashboard

We redesigned the dashboard!

Executions overview:

Execution details:

In execution details now you can view execution steps and see which step has passed or failed, and most importantly you can fetch execution artifacts that were generated by script execution. This feature is currently only available for Cypress scripts.

Result Artifacts

One of the really important details of testing is the ability to provide steps on how to reproduce the issue, especially in the UI. Many testing tools produce artifacts to help in this regard - screenshots, videos, logs, etc.. In order to collect these artifacts, we added a Minio storage  component to TestKube.

After each completed execution, its pod will scrape output files and store them in S3 storage in the cluster. By default, we are using Minio to achieve this - but you can configure this to use your own account S3 for better and more scalable artifact handling.

Result artifacts are available:

• Via the Testkube kubectl plugin - see …
• Via the updated Testkube dashboard; select a result and the Artifacts tab - click on artifacts to download/view them in your browser:
  

Realtime Test Execution Output

Not all test executions are short-lived. Sometimes tests can run for a long time (minutes to hours). Previously you would have to wait until the execution finishes to see its results and output. New in Testkube 0.7.0 is the possibility to follow test-out in real-time via the Testkube kubectl plugin

Now grab the output of the watch command:

Read more in our documentation at https://kubeshop.github.io/testkube/

Under the Hood

Previous Testkube version 0.6.0 consisted of REST based executors which would run the scripts in the same pod with the REST service. After lengthy use of the executors, pods would become huge and slow. So we decided to go the other way - Kubernetes jobs.

A Kubernetes Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. As pods successfully complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (ie, Job) is complete. Deleting a Job will clean up the Pods it created. Suspending a Job will delete its active Pods until the Job is resumed again.

What does this mean for the user?

Nothing really, apart from the fact that Testkube can now run hundreds or even thousands of tests more reliably. How you are using Testkube will remain the same. Magic happens somewhere in the cluster.

What’s next for Testkube?

We have a bunch of items on our road-map:

• Improved integration with CI/CD for triggering test executions
• Log collection from services under test
• The possibility to orchestrate multiple test scripts into a single test execution
• Notification mechanism to things like Slack, email, etc
• More executors

...and so much more. Check out our issues on Github.

Feedback

Why not give it a go yourself? Sign up to Testkube and try one of our examples or head over to our documentation - if you get stuck or have questions, we’re here to help! Find an answer to your questions in the Testkube Knowledge Base or reach out to us on Slack.