Continuous Integration and Continuous Deployment (CI/CD) have become essential components of modern software development cycles, allowing teams to automate and streamline their build, test, and deployment processes. Continuous testing is crucial in CI/CD pipelines, ensuring adequate testing of code changes throughout the software development lifecycle. 

By introducing continuous testing into CI/CD workflows, teams can create high-quality software in shorter periods, reduce risks, and address issues ahead of time. Our previous blog posts discussed integrating Testkube with CI/CD tools like GitHub Actions, Jenkins, Azure DevOps, Bitbucket, and CircleCI. 

In this blog post, we'll look at utilizing TeamCity build pipelines and Testkube to continuously deploy test cases as part of the CI/CD pipeline!

What is TeamCity?

TeamCity is a robust build management and CI/CD server developed by JetBrains. It automates the build, test, and release processes, ensuring your application remains deployable. TeamCity's web-based interface allows you to configure and manage builds and review and evaluate build logs and statistics.

The TeamCity build pipeline is a series of automated processes that monitor application development, testing, and deployment. It is a component of the CI/CD practices that TeamCity facilitates.

It streamlines the development process by continuously updating the application's status. This real-time feedback allows developers to quickly detect, evaluate, and address issues before they escalate into major ones. Its compatibility with a wide range of version control systems, build tools, and testing frameworks makes TeamCity an ideal choice for CI/CD pipelines.

TeamCity offers multiple installation methods, including Docker and TeamCity Cloud. Your specific requirements and infrastructure determine your preferred installation method. Refer to the TeamCity Cloud documentation to learn more about the installation process. 

We'll use TeamCity Cloud to integrate with Testkube for this blog post.

Benefits of Using Testkube with CI/CD Pipelines 

Testkube is a cloud native testing control plane that helps with test orchestration and execution. Integrating this capability with various CI/CD pipelines yields multiple benefits, as outlined below:

• Integration with Various CI/CD Tools: Easy integration with well-known CI/CD tools like GitHub Actions, Jenkins, Azure DevOps etc.
• Test Execution and Reporting: Perform tests effectively and generate comprehensive reports.
• Enhanced Security: Testkube ensures that your tests run securely by preventing unnecessary exposure in the CI/CD pipeline, which improves overall system security.
• In-depth Insights: Testkube captures and analyzes data from your executed tests. It leverages this information to generate detailed insights and dashboards, offering a clear view of test performance and trends.

Now, let's learn how to integrate Testkube and TeamCity build pipelines for a consistent and effective testing process.

Integrating TeamCity Build Pipelines with Testkube

We will integrate Testkube with TeamCity to streamline and optimize the testing process. For this demo, we will store a test workflow in a Git repository and configure the TeamCity build pipeline to automatically run whenever a new pull request (PR) is created in the repo. As a result, the test workflow will be deployed and executed on a target Kubernetes cluster, ensuring seamless integration and efficient testing within the CI/CD pipeline.

You can find the test workflow in this repo, which we’ll use for this blog post.

Prerequisites

• A TeamCity server account. 
• Get a Testkube account.
• Kubernetes cluster - we’re using a local Minikube cluster.
• Testkube Agent configured on the cluster.
• A GitHub repo containing test scripts and files.
• Testkube API Token, Org ID, and Environment ID

Once you meet the prerequisites, you can start a target Kubernetes cluster using a Testkube agent setup. You should also have a GitHub repository with the k6 test workflow configured.

To create the API token, refer to our API Token document. To find the Org ID and environment IDs, go to your Testkube Dashboard and click Settings from the menu bar.

For security reasons, you must store these IDs and credentials as variables in the Parameters section of TeamCity to ensure their secure access and utilization during pipeline execution. Refer to the TeamCity documentation for additional details. 

‍

Creating a Test Workflow in Testkube

Testkube allows you to create Test Workflows, which are an easy and quick way to define and run tests within a Kubernetes cluster. With these YAML-based specifications, the test and any associated images, artifacts, and resource parameters can be defined in a single file.

It gives you more flexibility and control over the tests and lets you build multi-step workflows that manage requirements end-to-end.

Discover and learn more about Test Workflows, which enable you to perform efficient and seamless testing in Kubernetes frameworks.

The first step is to create a test workflow. Here's an example of how to create a test workflow:

• Log in to your Testkube dashboard and navigate to the Test Workflow section.
• Click the "Add New Test Workflow" button. 
• A "Create a Test Workflow" dialog box appears, and you can build the sample workflow by selecting the "Start from an Example" option.

Here's a sample Test Workflow we created for k6:

‍

kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
	name: basic-k6-workflow
    namespace: testkube
    labels:
    	docs: example
spec:
	content:
    	files:
        - path: /data/example.js
        content: |-
        import http from 'k6/http';
        import { sleep } from 'k6';
        export default function () { 		
       		http.get('https://test.k6.io');
       		sleep(1);        
       	};  
    steps:
    - name: Run Tests
    workingDir: /data
    run:
    	image: grafana/k6:0.49.0
        env:
        - name: K6_WEB_DASHBOARD
        value: "true"
        - name: K6_WEB_DASHBOARD_EXPORT
        value: k6-test-report.html
        args:
        - run
        - example.js
        artifacts:
        paths:
        - k6-test-report.html
status: {}

We’ll save this k6 test workflow configuration as a yaml file in our GitHub repo and configure TeamCity’s build configuration to refer to this repo.

Configuring TeamCity Build Pipelines

After logging into your TeamCity account, you can create a project either:

• Manually
• Using one of the listed tools
• Providing a repository URL

Here, we'll choose from a repository URL option

Enter the details required to create a project from the URL, then click Proceed.

On the "Create a Project from a URL" page, enter the project name "Testkube Demo" and other details, then select the Proceed button.

After creating the project, the Build Steps page will automatically detect the build steps and their settings by scanning the specified VCS repo. This auto-detection will work only if a build file is in the repo. If the repo does not have a build configuration, the auto-step will not work. In this case, you can set up the build steps manually.

Since our repo doesn’t have a default build configuration, we will manually create a new build step. You can add a new build step to the build configuration by selecting a runner from the list or searching for it directly. Here, we'll use "Command Line" as a runner to execute shell commands or scripts.

When you select the Command Line runner, a “New Build Step: Command Line” page displays, which allows you to create a build configuration. 

‍

Enter the details required for configuring and running the build step, including the Step name, Step ID, Custom script, etc., and then click Save to add it to the configuration. Click on “Show advanced options” to configure additional settings.

Let’s take a look at the custom script:

echo "Execute Test Workflow"

# Setting Testkube Context
testkube set context --api-key $TESTKUBE_API_TOKEN --org-id $TESTKUBE_ORG_ID --env-id $TESTKUBE_ENV_ID
     	 
# Verify Testkube CLI installation
testkube version
              	 
# Run Testkube command to validate the connection
testkube create testworkflow --name testworkflow-actions -f k6-basic.yaml
testkube run tw testworkflow-actions

The mentioned workflow carries out the following actions:

• Configures the Testkube CLI context to use the specified environment by setting the context with the provided API key, organization ID, and environment ID..
• testkube version: Verifies that Testkube CLI has been installed correctly and is operational.
• testkube create testworkflow --name testworkflow-actions -f k6-basic.yaml: Creates a new test workflow in Testkube named testworkflow-actions and the -f k6-basic.yaml flag specifies the configuration file for the test workflow, which defines the workflow steps and parameters. 
• testkube run tw testworkflow-actions: Executes the test workflow

After configuring your build steps, click on the “Triggers” section to configure the existing VCS Trigger. Once configured, TeamCity will detect changes in the VCS whenever they occur on the repo and add them to the build queue for execution. 

To save your changes, click the Save button. 

‍

After configuring the Triggers section, click on the “Build Features” section to add features that extend the build process, such as connecting external tools, managing resources, and pushing statuses to VCSs. 

We want TeamCity to publish the build status to the GitHub PR for our use case.

‍

Click “Add Build Feature” and choose “Commit Status Publisher”. 

‍

In “Publisher,” select the repo for publishing the build status. From the drop-down menu, select GitHub.

Select “Access Token” from the drop-down menu under Authentication Type to authenticate with the repo.

To obtain an access token, a GitHub personal access token with specific permissions is necessary, including:

• Repo: status for repository-level status updates.
• public_repo for public repositories or repo for private repositories.

Refer to the GitHub documentation to create a Personal Access Token for your GitHub repo. 

You can verify the configuration by connecting to the given VCS repository by clicking the “Test Connection” button.

‍

Once the build feature is configured, click the Save button to apply the changes. The build feature will be added to the list.

‍

You can add environment variables to the Parameters section by clicking the “+Add new parameter” button. Configuring environment variables before configuring build steps, triggers, or other settings is generally recommended.

‍

By following these steps, we have successfully configured your TeamCity pipeline to integrate with Testkube. This setup ensures that your build steps are properly defined and the VCS trigger is configured to monitor changes in your repo.

Triggering Testkube Test

Once you have configured your pipeline, make some changes to the repo and commit them by raising a pull request to trigger the pipeline.

‍

Once the pull request is created, the configured VCS Trigger in TeamCity automatically triggers the pipeline. The pipeline begins executing the steps defined in the build configuration file.

On TeamCity, navigate to the project and click on the “Build” option to view the build execution status and build log. 

‍

Based on the configuration, we can see that the pipeline is triggered with each new PR. It connects to our target Kubernetes cluster, deploys the k6 test defined in the repo, and runs the test workflow using Testkube.

Monitoring the Test Workflow

You can use the Testkube CLI or the Testkube dashboard to monitor and report on the workflow's progress. Let's look at how to use the dashboard to monitor the status of the Test Workflow.

Navigate to the Test Workflows page to see the k6 test workflow named “testworkflow-actions,” which was deployed via the pipeline.

Click on a step to see the status of the test workflow.

Testkube collects all artifacts upon execution, providing a stats report containing all outputs and results.

You can also verify the status of the pipeline execution on GitHub PR. This is because of the Build Feature that we configured, which enables TeamCity to update the build status for the GitHub PR.

The example above shows how easy it is to configure Testkube with TeamCity. You can execute advanced scenarios by modifying the workflow and setting repositories to meet specific requirements.

Summary

Continuous testing is paramount in the modern software development industry, as it optimizes the efficiency of test practices, thereby improving the testing experience. We automated the testing process by seamlessly integrating TeamCity build pipelines' automation capabilities with Testkube's testing capabilities while creating a robust CI/CD pipeline framework. 

We demonstrated the process by successfully implementing a k6 test using Testkube and executing it through TeamCity build pipelines, resulting in a practical workflow that improves testing efficiency and reliability. 

So, now that you've learned about these two powerful tools—TeamCity and Testkube—and how they can be seamlessly integrated to help with testing, take your CI/CD pipeline experience to the next level by leveraging their combined strengths to deliver reliable releases quickly and effectively.

Reach out to me to discover how Testkube can transform your testing strategy. You can also join our Slack community to start a conversation or read Testkube documentation to see how Testkube can enhance your GitOps testing workflow, ensuring seamless, efficient, and automated testing within the Kubernetes environments. The future of software testing is here. Are you ready to be a part of it?