Building Ephemeral Test Environments using vCluster, GitOps and Testkube

Testing Kubernetes applications is notoriously challenging. Shared development environments lead to conflicts, traditional CI/CD pipelines can't replicate production complexity, and debugging failed tests in ephemeral containers feels like detective work.

What if every pull request you opened could spin up its own isolated Kubernetes environment with real GitOps workflows and comprehensive testing—then vanish when done?

This is exactly what we'll build using vCluster, ArgoCD, and Testkube. This solution expands on our previous post about End-to-End PR Testing in Kubernetes by adding ArgoCD as the deployment orchestrator, creating a complete GitOps pipeline where both infrastructure and tests are managed as code.

‍

Why GitOps and Ephemeral Clusters?

Traditional Kubernetes testing approaches fail at scale due to environment conflicts, infrastructure drift, and limited test observability. This architecture solves these problems by combining:

• GitOps: Declarative, auditable deployments that mirror production workflows
• Ephemeral environments: Complete isolation per PR without infrastructure overhead
• In-cluster testing: Native Kubernetes test execution with full resource visibility

To achieve this, we will use the following tools:

• GitHub Actions: Orchestrates the entire pipeline when PRs are created or modified
• vCluster: Creates lightweight, isolated virtual clusters inside a host Kubernetes cluster
• ArgoCD: Handles GitOps-driven application deployment, watching Git for changes and syncing manifests into vCluster
• Testkube: Orchestrates test execution inside Kubernetes, collecting logs and results that persist beyond cluster lifecycle
• Chainsaw: Validates Kubernetes policies and resource configurations
• K6: Executes performance and load testing scenarios

Key Benefits

• Zero infrastructure management: Developers focus on code, not cluster operations
• Complete isolation: Each PR gets its own environment, eliminating cross-contamination
• Production parity: GitOps workflows match exactly how applications deploy to production
• Deep observability: Full visibility into test execution, resource usage, and failure diagnostics
• Fast feedback loops: Automated testing delivers results within minutes of code changes

‍

Hands-On: Building a Complete GitOps Testing Pipeline

To understand this better, we'll build an automated testing system that creates isolated environments for every pull request. 

When a developer opens a PR against our sample repository, GitHub Actions automatically:

• Provisions a lightweight vCluster inside your existing GKE cluster
• Configures ArgoCD to deploy the nginx application using GitOps
• Installs a Testkube runner inside the vCluster for isolated test execution
• Runs comprehensive testing, including K6 performance tests and Chainsaw policy validation
• Collects detailed test results, logs, and metrics
• Completely destroys the environment when testing completes

Below is a sequence diagram to help you understand the sequence of events better.

Prerequisites

• A host Kubernetes cluster (GKE, EKS, AKS, or on-premises)
• Helm and kubectl installed
• ArgoCD is installed on the host cluster
• Testkube account with appropriate permissions
• Testkube API Token, Org ID, and Environment ID
• GitHub repository for code, TestWorkflow, and manifest files

To create the Testkube API token, refer to our API Token documentation. To find the Org ID and environment IDs, log in to your Testkube Dashboard and select Settings from the menu bar.

Further, we are using GKE for this demo, and we had to create some GitHub environment secrets with GKE cluster details like `GKE_PROJECT_ID`, `GKE_CLUSTER_NAME`, `GKE_ZONE`, and `GCP_SA_KEY` that are required to communicate with the GKE cluster. 

You can find all the code, TestWorkflows, and manifests used in this blog post in this GitHub repo.

Application & Test Setup

Sample Application: Nginx Web Service

For this article we're using a simple nginx web application deployed via Kubernetes manifests (in real-life your application will most likely be much more complex). The application consists of a deployment, service, and ingress configuration. You can find all the nginx manifests here.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: nginx-deployment
spec:
 replicas: 2
 selector:
   matchLabels:
     app: nginx
 template:
   metadata:
     labels:
       app: nginx
   spec:
     containers:
     - name: nginx
       image: nginx:latest
       ports:
       - containerPort: 80
       resources:
         requests:
           memory: "64Mi"
           cpu: "250m"
         limits:
           memory: "128Mi"
           cpu: "500m"

Test Workflows

Our testing strategy combines performance and policy validation using two Testkube Workflows. You can create these TestWorkflows using the Testkube dashboard. You can also find these manifests here.

k6 Test Workflow

k6 is a load testing tool designed for testing APIs and web applications. Our k6 workflow simulates 10 concurrent users hitting the nginx service for 30 seconds, measuring response times and validating that the application can handle expected traffic loads without performance degradation. You can find the k6 TestWorkflow here. 

kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: k6-sample
namespace: testkube
labels:
  docs: example
spec:
content:
  git:
    uri: https://github.com/techmaharaj/testkube-vcluster-test.git
steps:
- name: Run Tests
  workingDir: /data
  run:
    image: grafana/k6:0.49.0
    env:
    - name: K6_WEB_DASHBOARD
      value: "true"
    - name: K6_WEB_DASHBOARD_EXPORT
      value: k6-test-report.html
    shell: k6 run /data/repo/test/k6/k6-sample.js --vus 100 --duration 10s
  artifacts:
    paths:
    - k6-test-report.html
status: {}

‍Chainsaw TestWorkflow

Chainsaw is a Kubernetes-native testing tool from the Kyverno project that validates cluster resources against policies and best practices. Our Chainsaw workflow verifies that the deployed nginx application has a certain number of replicas. You can find the actual Chainsaw TestWorkflow here.

kind: TestWorkflow
apiVersion: testworkflows.testkube.io/v1
metadata:
name: chainsaw-nginx
namespace: testkube
spec:
content:
  git:
    uri: https://github.com/techmaharaj/testkube-vcluster-test.git
    paths:
    - test/chainsaw
steps:
- name: Test ArgoCD deployment
  workingDir: /data
  run:
    image: ghcr.io/kyverno/chainsaw:latest
    shell: |
      chainsaw test --report-format XML --report-path /data/repo --report-name chainsaw-report /data/repo/test/chainsaw
  artifacts:
    paths:
    - /data/repo/chainsaw-report.xml
status: {}

GitHub Actions Workflow

Our GitHub Actions workflow orchestrates the entire testing pipeline through these key phases:

• Infrastructure Setup: Create an isolated vCluster environment
• GitOps Deployment: Configure ArgoCD to deploy from the PR branch
• Test Orchestration: Install Testkube runner and execute tests
• Results Collection: Gather logs, metrics, and artifacts
• Environment Cleanup: Destroy all resources automatically

‍

Below are the key implementation details. You can look at the complete GitHub Actions workflow file here. Please note that we have added logging and debug statements in the workflow to improve understanding and help with debugging should it fail.

‍

vCluster Creation

- name: Create vCluster
 run: |
   echo "=== Creating vCluster for PR $PR_NUMBER ==="
   
   # Save host context
   HOST_CONTEXT=$(kubectl config current-context)
   echo "HOST_CONTEXT=$HOST_CONTEXT" >> $GITHUB_ENV
   echo "Host cluster context: $HOST_CONTEXT"
   
   # Create namespace
   kubectl create namespace demo-$PR_NUMBER --dry-run=client -o yaml | kubectl apply -f -
   
   # Create vCluster with proper configuration
   vcluster create testkube-vcluster-$PR_NUMBER \
     --namespace demo-$PR_NUMBER \
     --connect=false \
     --expose

‍

ArgoCD Integration

- name: Configure ArgoCD to Deploy to vCluster
 run: |
   echo "=== Configuring ArgoCD to Deploy to vCluster ==="
   
   # Ensure we're on host cluster for ArgoCD operations
   kubectl config use-context $HOST_CONTEXT
   echo "Current context: $(kubectl config current-context)"
   
   # Create ArgoCD application targeting vCluster
   argocd app create nginx-vcluster-pr-$PR_NUMBER \
     --repo "https://github.com/techmaharaj/testkube-vcluster-test" \
     --path k8s \
     --dest-server "$VCLUSTER_SERVER" \
     --dest-namespace nginx \
     --revision ${{ github.head_ref || github.ref_name }} \
     --sync-policy automated \
     --upsert
   
   # Sync the application
   argocd app sync nginx-vcluster-pr-$PR_NUMBER --force --timeout 300

‍

Installing Testkube Runner

- name: Install Testkube Runner in vCluster
 run: |
   echo "=== Installing Testkube Runner in vCluster ==="
            
   # Create unique runner name for this PR
   RUNNER_NAME="vcluster-runner-pr-$PR_NUMBER"
   echo "RUNNER_NAME=$RUNNER_NAME" >> $GITHUB_ENV
   echo "Creating runner: $RUNNER_NAME"
   
   # Install runner in vCluster - hardcode the known environment
   echo "Installing runner with environment: testkube-gke"
   testkube install runner "$RUNNER_NAME" \
     --create \
     --floating \
     --label environment=vcluster \
     --label pr_number="$PR_NUMBER" \
     --label purpose=ephemeral_testing \
     --namespace testkube \
     --env testkube-gke

‍

Executing TestWorkflow

- name: Run Tests in vCluster
 id: run-tests
 continue-on-error: true
 run: |
   # Runner Name
   RUNNER_NAME="vcluster-runner-pr-$PR_NUMBER"
   
   # Execute k6 test workflow targeting vCluster runner
   testkube run testworkflow k6-sample -f \
     --target name=$RUNNER_NAME \
     --tag runner=$RUNNER_NAME \
     --tag environment=vcluster \
     --tag test-type=k6 \
     --tag pr=PR-$PR_NUMBER \
     --tag cluster=demo-$PR_NUMBER \
     --tag demo=gitopscon-isolated

Workflow Triggers

The pipeline automatically triggers on:

• Pull request creation
• Pull request updates (new commits)
• Manual workflow dispatch for testing

‍

Seeing the Workflow in Action

With all the required steps in place, it’s time to see the workflow in action.

To trigger the workflow, simply create a new PR in the repo. The workflow begins immediately after PR creation and shows all steps in the Actions tab.

‍

You will see the creation of vCluster on the GKE host cluster and the installation of ArgoCD on the host cluster. 

‍

‍

‍

ArgoCD deploys the nginx application directly from the PR branch to ensure we test the exact code changes.

‍

‍

The workflow first configures Teskube on the cluster using the API key, org ID, and env ID that you saved as variables. 

‍

‍

Testkube runner is then installed on the Testkube environment that you associate it with using the credentials. You can also validate the runner setup under “Environments” on the Testkube dashboard. 

‍

‍

Once this step is successful, you can see the Testkube runner created under “Settings” -> “Agents” on the Testkube dashboard.

‍

‍

After all the setup is ready and verified, the workflow executes the two Testworkflows - k6 and chainsaw - using the runner that we configured earlier. 

‍

Below is the successful execution of the k6 TestWorkflow that validated the nginx service. 

‍

‍

‍

Below is the chainsaw TestWorkflow that validated the number of replicas of the nginx deployment in the vCluster. 

‍

‍

‍

Once the execution is complete, you can check the status of the tests in the Testkube dashboard or in the GitHub actions workflow execution as well.

‍

The status of the GitHub Action is automatically updated and sent back to the PR. For advanced use cases, you can create branch rules that can act based on the result of the GitHub Action. 

With this we've built a complete GitOps testing pipeline that automatically creates ephemeral environments for every pull request. Our workflow provisions a vCluster, deploys applications using ArgoCD's GitOps principles, executes comprehensive tests with Testkube, and cleans up everything automatically—all without touching the host cluster. You can view the complete GitHub workflow here.

While our example used a simple Nginx deployment with basic K6 performance tests and Chainsaw policy validation, this architecture supports far more complex scenarios. You can integrate any Testkube-compatible testing framework, from Cypress end-to-end tests and Postman API collections to custom security scans and chaos engineering experiments. 

‍

Conclusion

The combination of ArgoCD, Testkube, and vCluster creates a robust GitOps testing machine that addresses the most fundamental challenges of testing in Kubernetes environments. By treating infrastructure, applications, and tests as code, this GitOps approach eliminates the environment conflicts, flaky tests, and infrastructure overhead that plague traditional testing workflows.

This is the next stage in cloud-native testing, transitioning from manual, shared test environments to declarative, automated, isolated, and ephemeral testing processes that match the speed of development.

Get started using Testkube and ArgoCD to create a modern, scalable PR testing workflow. Join the Testkube Slack community or read the Testkube documentation to begin creating your own automated testing stack!