Leveraging AI to Analyze Test Results in Kubernetes

Introduction

Artificial Intelligence is one of the remarkable forces that has been leading the changing landscape of the tech industry. From revolutionizing health care with computer-assisted imaging to improving customer experiences using chatbots, AI's growth knows no bounds.

Within software development itself, Artificial Intelligence has taken multiple roles, one example is assisting developers with better coding to perform predictive analytics. This has also made coding more efficient as it offers assistance to developers. One of the areas within software development where Artificial Intelligence's prowess is growing is testing. 

The convergence of Artificial Intelligence with testing calls for a new era where AI analysis enhances testing methodologies, offering smarter and faster log analysis along with a suite of advantages. In this blog post, we will see how Testkube’s AI Analysis feature for Test Results can make your testing process faster and smarter.

The Role of Artificial Intelligence When Analyzing Test Data

Artificial Intelligence’s ability to process and comprehend large amounts of data, understand patterns, and make intelligent decisions is an invaluable asset in the world of testing. It can assist with pattern recognition, anomaly detection, and predictive analysis to identify potential pitfalls. Artificial Intelligence can accelerate the identification of vulnerabilities that might otherwise be difficult to detect with traditional testing practices.

Amongst all the AI features, one of the remarkable features in testing lies in its capability to expedite the analysis of test logs. Traditional test logs have different messages and patterns that require exhaustive manual analysis which is not only time-consuming but also prone to errors. However,  current Artificial Intelligence machine learning models are trained on extensive data sets and can recognize patterns that have occurred across different testing scenarios. This brings precision and predictability to the analysis done by AI.

Following are some key benefits that Artificial Intelligence brings to the table for test log analysis:

• Faster Root Cause Analysis: Artificial Intelligence’s ability to process massive amounts of logs, scrutinize every entry, and identify patterns accelerates the overall time it takes to find the root cause. Moreover, it can identify patterns that the human eye can easily miss. What was once a time-consuming puzzle is transformed into a faster and more precise process. 

• Troubleshooting Suggestions: Going through logs and issues is a tedious task and that’s when Artificial Intelligence can become the smart assistant. It can provide actionable insights and recommendations, from syntax errors to configuration issues, that empower testers on how to address different issues faster. This not only saves time but also improves accuracy. 

• Efficient Feedback Loop: Artificial Intelligence ensures insights derived from logs are rapidly shared with the developers for them to make the necessary decisions more quickly. The integration of AI analysis with logs and developer action shortens the feedback loop allowing for more iterations in the same time frame. This results in a faster development process with fewer bottlenecks.

AI Test Insights in Testkube

Testkube is a Kubernetes-native testing framework that lets you store, manage, and execute tests on Kubernetes clusters, abstracting away all the complexities from your team. It leverages Kubernetes custom resource definitions (CRDs) to define tests as Kubernetes native objects. Testkube seamlessly adapts to your testing needs and scales as required. It comes with a comprehensive list of features, however, our focus today is the newest one - AI Test Analysis Analysis.

With the help of Artificial Intelligence, Testkube provides smart insights and suggestions for all your failed tests. Whenever you have failing tests, use the AI Insights tab to get the possible reasons for the failure of the tests along with troubleshooting suggestions to fix errors. 

What distinguishes Testkube from other AI-based testing solutions is its incorporation of AI directly within the framework. This means that you don’t have to worry about enabling it in your testing tools individually. Simply bring in your testing tool to Testkube and leverage the benefits of AI Insights right out of the box. Check out our extensive list of supported tests. 

One of the primary reasons for adding AI test Insights to Testkube is that it helps to considerably reduce the time needed for root cause analysis. By analyzing test logs, identifying patterns, and suggesting fixes, Testkube’s AI Test Insights makes the entire feedback process faster. This leads to faster bug resolution, resulting in faster shipping of features and products. 

For developers, Testkube integrates with OpenAI to provide the AI Analysis feature. The test logs are modified and only the relevant information is extracted and sent to OpenAI for analysis. The response is crafted for your requirements and presented along with a list of suggestions on how to fix it. This integration is baked into Testkube so users don’t need to worry about how it works and the related configurations. 

AI Test Insights Tutorial

To better understand how AI Test Insights works in Testkube, we’ll create a cURL test to validate a response from an endpoint. We’ll use Testkube’s AI capabilities to understand the failure of the test and fix it based on the suggestions provided. 

Pre-requisites

• Kubernetes cluster - we’re using Minikube
• Testkube Account
• Testkube CLI & Testkube agent configured
• Set up an environment on Testkube Cloud

Perform the pre-requisite steps of creating Kubernetes clusters, configuring Testkube CLI, and Agent, and creating an environment on the Testkube cloud before proceeding. 

Creating a cURL Test

Testkube allows you to create tests for different frameworks using different ways. For this blog post, we’ll create a simple cURL test using the Testkube Cloud UI.

Login to your Testkube account and navigate to the Tests section to create a test. Click on the Tests tab and choose `Add a new test`.

We’ll create a test that will send an HTTP GET request to an endpoint and validate the response - an IP address - is received.

Provide the following details: 

• Name: curl-url-test
• Type: curl/test
• Source: String

{
    "command": [
      "curl",
      "http://ip.jsontest.com/",
      "-H",
      "'Accept: application/json'"
    ],
    "expected_status": "200",
    "expected_body": "{\"ip\": \"120.88.40.210\"}"
}

‍

Execute Test

Click the Run Now button to execute the test case. The Testkube agent will execute the test on the target Kubernetes cluster and display the results.

Validate the Result & Use AI Analysis

After the test has finished executing, you can click on the test to view the results. In this case, the test has failed. First, we will look at the logs to understand why the test has failed.

‍

It shows that the IP address that we are looking for in the request does not match and caused the test to fail. Now we can see how the AI Analysis feature evaluates the issue and what is suggested for a resolution.

Navigate to the AI Analysis Tab. Testkube will automatically collect the relevant details from the log and analyze them.

‍

‍

As per the AI Analysis, the assessment is “The test execution is failing because the expected result does not match the actual result. The expected result was not received from the API”. This means that the response that we have received is different from what is expected, which is spot on. 

AI Analysis also provides you with a list of suggestions that you can follow to fix the failing test. In this case, apart from checking the URL, headers, and the internet connection, it is also suggested we validate the response. 

Let us go back and update the expected IP address in the test and execute it again.

{
    "command": [
      "curl",
      "http://ip.jsontest.com/",
      "-H",
      "'Accept: application/json'"
    ],
    "expected_status": "200",
    "expected_body": "{\"ip\": \"120.88.40.232\"}"
 }

‍

If you execute the test again, it passes. Note that the AI Analysis tab is not present this time. This is because AI Analysis is best suited to analyze failed tests.

This was a simple demo to show you how to use Testkube’s AI Analysis feature to quickly analyze logs and fix failing tests. You can create complex tests to test your applications and infrastructure. The feature aids in faster troubleshooting of the issue with relevant suggestions leading to faster root cause analysis for your team.

Summary

The current pace of evolution in software development requires agility and precision. With the integration of Artificial Intelligence into software testing, we have ushered in a new era of software testing. Faster root cause analysis leads to swift bug resolution, and accelerated feedback loops, all leading to faster shipping of the product. 

AI Analysis in Testkube brings the power of Artificial Intelligence to your testing without the need for you to worry about how it will work. Simply bring in your testing tool, integrate it with Testkube, and experience the power of AI-assisted testing. To experience the full potential of AI analysis on Testkube, we invite you to try Testkube today. 

Try out the various test types in Testkube and witness firsthand how Testkube’s AI Analysis makes your testing faster and more precise. We love to hear from you! Join our Slack community for guidance and support.