Over the years, technology has advanced rapidly and changed how we build and deploy our applications. Today, many applications are distributed, running on Kubernetes, and have multiple microservices, often deployed on different clusters, making it difficult to test the application end to end. 

While many developers use AI to help them write code, have you wondered if we could use AI to write and manage tests for us? Perhaps not surprisingly, it turns out we can very well use AI to write and manage tests. 

In this blog post, I’ll show how we can build a Testkube AI assistant using GPTScript to help us perform all the tasks for interacting with Kubernetes clusters and writing the tests for us. 

This blog post relates to my talk at DevOpsDays Kerala, where I spoke on “Is Your Testing Stuck In the Past? Modernizing Kubernetes Testing With GPTScript and Testkube”.

What is GPTScript?

We all have used ChatGPT, Claude, Perplexity, and similar AI services for various tasks. The interactive UI is suitable for generating text and getting assistance for coding. However, leveraging its full potential effectively requires understanding the art of crafting prompts and interacting with APIs, which is a steep learning curve.

That’s where GPTScript comes in handy. It is an open source scripting language designed to bridge this gap. It is tailored to automate your interactions with an LLM like OpenAI by building AI assistants using natural language, making the process much easier. 

At the core of GPTScript are tools. Tools are like functions that perform a particular set of actions. Further, there’s a concept of Context, which is a means to share prompt details among different tools. We can provide further guardrails and prompts to our assistant to complete a specific task. All of it is written in a `.gpt` file in simple English.

In the following section, we’ll create a Testkube AI testing assistant using GPTScript. 

Building a Testkube AI Assistant Using GPTScript

Test Workflows are at the heart of Testkube. These are single-file specifications where you can define everything about the execution of your tests, from the test code itself to resource requirements and parameters. These Test Workflow specifications can be version-controlled and triggered from your CI/CD pipeline or GitOps workflow.

You can write the Test Workflow from scratch or use the Testkube dashboard to create a test workflow using a wizard. This wizard speeds up test creation. Read our documentation on creating Test Workflows.

We’ll see how we can use GPTScript to build a Testkube AI assistant to generate and manage Test Workflows. We’ll deploy a pod with a custom Nginx application and use this assistant to write a Postman Test Workflow to test the response from the Nginx application. We’ll then update the application's image, which will cause our Postman Test Workflow to fail. Finally, we’ll ask the assistant to figure out why it failed and fix the Test Workflow.

Pre-requisites

• An OpenAI API Key that you can get from here.
• Latest version of GPTScript configured.
• Kubernetes cluster - we’ll be using kind.
• Testkube account and a Testkube Environment configured with Testkube Agent.
• Testkube CLI and kubectl, helm configured on your system.

You can validate GPTScript’s installation by running `gptscript version`, which will return the version of GPTScript installed.

Writing GPTScript

The first step is to create the GPTScript for the Testkube assistant. Below is the testkube.gpt file.

Name: Testkube
Description: A tool to help you perform testing of your application on your Kubernetes clusters using Testkube.
Context: learn-testkube, learn-kubectl
Tools: sys.exec, sys.http.html2text?, sys.find, sys.read, sys.write, github.com/gptscript-ai/browse-web-page
chat:true

You are an assistant for Testkube and help the user create, manage and execute test workflows. You can also perform kubernetes related tasks.

Rules
1. Access the testkube workflow docs at https://docs.testkube.io/articles/test-workflows and remember the latest specification to create testworkflows.
2. Use testkube CLI to interact with Testkube.
3. Use kubectl CLI to interact with the Kubernetes cluster.
4. Based on the user's request, perform actions on the Kubernetes cluster and create, manage, delete test workflows.


---

Name: learn-testkube
Description: A tool to help you learn testkube cli
#!/bin/bash
testkube --help
testkube create --help
testkube create testworkflow --help
testkube run --help

---

Name: learn-kubectl
Description: A tool to help you learn k8s and related commands
#!/bin/bash

CMDS="kubectl helm"
echo 'The additional CLI commands are available locally, use the `exec` tool to invoke them:'
for i in $CMDS; do
if [ -e "$(command -v $i)" ]; then
    	echo '  ' $i
fi
done

‍

Let us understand what this file does:

• We create a Testkube Assistant and define the tools and contexts it has access to - learn-testkube and learn-kubectl in this case.
• We enable chat mode and describe the guidelines for the assistant to follow:some text
  • We instruct the assistant to refer to the latest Testkube documentation on Test Workflows.
  • We also provide it with access to kubectl and Testkube CLI required to perform the tasks.
• For the learn-testkube context, we provide access to the testkube CLI.
• For the learn-kubectl context, we provide access to kubectl, helm CLI.

The script is also available in the GPTScript repo on Github.

Executing the GPTScript

To execute this script, you must first configure the “OPENAI_API_KEY” environment variable. 

Execute the script using the following command:

`gptscript testkube.gpt`

This will initialize the assistant and start in chat mode. 

Note: Because this is an AI assistant, the responses will differ every time, even if the same prompts are provided. Hence, you might have to do some extra prompts to get this demo to work.

Let’s first ask it to list the pods running in the current namespace.

Our application is deployed as a hello-world deployment pod. Let's ask it to make a curl request to test the response.

It figured out the service bound to the hello-world-deployment pod and made a curl request to validate the response, which is “Hello Kerala.”

Let us ask it to write a Postman Test Workflow to test this response.

Below is the complete Postman Test Workflow it generated.

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
 name: postman-hello-world-validation
spec:
 content:
   files:
     - path: hello-world.postman_collection.json
       content: |
         {
           "info": {
             "name": "Hello Kerala Validation",
             "_postman_id": "12345-67890-abcdef",
             "description": "Collection to validate the response of the hello-world service",
             "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
           },
           "item": [
             {
               "name": "Validate Hello Kerala Response",
               "request": {
                 "method": "GET",
                 "header": [],
                 "url": {
                   "raw": "http://hello-world-service.default.svc.cluster.local",
                   "protocol": "http",
                   "host": [
                     "hello-world-service",
                     "default",
                     "svc",
                     "cluster",
                     "local"
                   ]
                 }
               },
               "response": []
             }
           ],
           "event": [
             {
               "listen": "test",
               "script": {
                 "exec": [
                   "pm.test(\"Status code is 200\", function () {",
                   "    pm.response.to.have.status(200);",
                   "});",
                   "pm.test(\"Response is 'Hello, Kerala!'\", function () {",
                   "    pm.response.to.have.body(\"Hello, Kerala!\");",
                   "});"
                 ],
                 "type": "text/javascript"
               }
             }
           ]
         }
 container:
   resources:
     requests:
       cpu: 256m
       memory: 128Mi
   workingDir: /data/repo
 steps:
   - name: Run Postman Test
     run:
       image: postman/newman:5-alpine
       args:
         - run
         - hello-world.postman_collection.json

‍

In the above Test Workflow, we can see that it has followed the latest specification guidelines to generate the Test Workflow. It has added a Postman collection and a test to validate the response of the pod based on the cURL request it performed initially.

We’ll now ask it to create and execute this Test Workflow.

We can see that it has executed the Postman Test Workflow and passed. Thus, we saw how we could generate a Test Workflow using the Testkube AI assistant. 

Let’s now change the image of the hello-world deployment, which will now respond “Hello World”

‍

We’ll ask it to rerun the Test Workflow, and this time, it will fail because the response will be different.

We’ll now ask our assistant to determine why the test failed and provide a resolution. 

It explains why the Test Workflow's failure and suggests possible steps to rectify it. We’ll choose the third option, which updates the yaml specification and the existing Test Workflow. It automatically rewrites the yaml specification and updates the response validation section to check for “Hello World” instead of “Hello Kerala”.

Now, when we execute the Test Workflow, it passes as expected. It was now looking for a “Hello World” response, and it got “Hello World”.

And that’s how we can build an AI Assistant using GPTScript! 

Conclusion

In this post, we saw how to create an AI testing Assistant using GPTScript for testing with Testkube. We could “talk” to the AI assistant to manage our Testkube Test workflows smartly and effortlessly. It was similar to talking to a teammate and asking them to create and manage tests. The difference is that I interacted with an LLM, and the process was faster and wiser.

Testkube is a robust framework for orchestrating Test Workflows, and its ability to bring in any testing tool and run it under Kubernetes makes it a versatile tool for testing any cloud-native application. Thanks to Testkube and our AI Assistant, it is now easier than ever to ensure that our applications are both thoroughly tested and resilient to change. 

As we embrace the new technological advancements, it’s evident that the future of testing in Kubernetes lies in leveraging powerful tools like Testkube and AI to adapt and innovate. 

We invite you to try Testkube, and if you face any issues, remember that the entire Testkube team, plus a vibrant community of fellow Kubernetes testers, are on Slack. We’re just getting started in building the most comprehensive cloud-native testing framework for Kubernetes so feel free to follow us on Twitter @testkube_io.