Scaling Cypress Tests: Parallelize E2E Tests

Users today want applications that are snappier and have an intuitive interface. Building and shipping such applications require thorough testing features to ensure they work as expected. 

One of the most popular end-to-end testing tools for this is Cypress. Its rich feature set and developer-friendly APIs make testing your entire application easy. However, running tests in sequence is time-consuming for large, complex applications. 

Hence, teams turn to test parallelization, which allows them to run multiple tests simultaneously. In this blog post, we’ll look at how test parallelization works in Cypress. We’ll also explain why parallelization is critical for end-to-end testing and how Testkube helps with test parallelization for Cypress, providing an example. 

The Need For Parallelization In Testing

Test parallelization runs multiple tests simultaneously across different environments. Instead of executing tests sequentially, one after another, parallel testing divides the test suite into smaller groups that can be run concurrently.

Here’s why parallelization in testing is necessary:

• Faster Results: Test parallelization reduces the time taken to execute all the tests, thus reducing the overall execution time and allowing teams to get faster feedback. 
• Resource Efficiency: By running tests in parallel across different environments, you can optimize your resource utilization and better use the available resources.

Test Parallelization in Cypress

As mentioned earlier, Cypress has become a popular choice for end-to-end testing. To enhance its capabilities, Cypress provides Cypress Cloud with an intuitive dashboard with additional functionalities like test parallelization and result analytics. These features allow developers to speed up test execution.

Below are some salient features of running tests in parallel in Cypress

• Multi-browser Testing: Cypress allows tests to be run concurrently across different browsers, enhancing cross-browser coverage.
• CI Integration: Cypress easily integrates with CI tools and allows you to execute tests on different environments in an automated manner.
• Dashboard Service: Cypress provides a dashboard that helps you manage and visualize parallel test runs.

Having said that, running tests in Parallel in Cypress can be complex. Managing test data, environment setup, and teardown across parallel instances can be difficult. Further, in their free tier, the limit on the number of tests you can execute in parallel and the results you can collect is relatively low.

Cypress With Testkube

Testkube is a Kubernetes-native testing framework that allows you to create testing workflows in a declarative, version-controlled manner. It allows you to plug in any testing tool and leverage the power of Kubernetes.

Key benefits

• Simplified Test Workflow Creation: Without complex scripting, Testkube facilitates the creation of detailed Test workflows that allows for better control and customization of your Test Executions. Refer to our Test Workflows documentation to learn more. 
• Scales your Testing Tools by leveraging Kubernetes: Testkube integrates with any testing tool, including Cypress and Playwright, and allows you to leverage your own infrastructure to run tests at scale!
• Single Pane of Glass: Testkube gives you a simple dashboard that allows you to observe and troubleshoot all of your tests.

Further, Testkube integrates with existing CI/CD pipelines, enhancing end-to-end testing capabilities. Furthermore, it provides a straightforward process for incorporating custom testing tools, enabling native execution on Kubernetes with minimal setup. Read more about Testkube.

Key advantages of Testkube for Cypress test parallelization:

1. No dependency on Cypress libraries: It doesn't rely on any Cypress-specific libraries that could be subject to blocking or restrictions.
2. Flexibility: Testkube allows users to run tests without being tied to specific versions of Cypress or worrying about compatibility issues.
3. Container-based execution: Testkube runs tests inside containers, providing isolation and consistency.

Thus, Testkube provides stable and reliable options for teams looking to parallelize their Cypress tests.

Cypress Test Workflow using Testkube

Let's see how we can run Cypress tests in parallel using Testkube. We’ll create a Test workflow for Cypress tests and configure it to run in parallel. 

Pre-requisites

• A Testkube account, either on prem or in the cloud.
• Kubernetes cluster - we’re using a local Minikube cluster.
• Testkube Agent configured on the cluster.

Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube agent configured. 

Creating a Test Workflow

Navigate to the Test Workflows tab and click on “Add a new test workflow”

This will provide you with three options:

• Create from scratch - use the wizard to create a Test Workflow.
• Start from an example - use existing k6, cypress, and playwright examples
• Import from yaml - import your own Test Workflow.

We’ll choose the “Import from yaml” option to create this workflow. Below is the yaml file used to create the Test workflow for running Cypress tests in parallel.

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
  name: cypress
  namespace: testkube

spec:
  content:
    git:
      uri: https://github.com/kubeshop/testkube
      revision: main
      paths:
      - test/cypress/executor-tests/cypress-13

  container:
    image: cypress/included:13.6.4
    workingDir: /data/repo/test/cypress/executor-tests/cypress-13

  steps:
  - name: Run tests
    parallel:
      maxCount: 3
      
      shards:
        testFiles: 'glob("cypress/e2e/**/*.js")'

      description: '{{ join(map(shard.testFiles, "relpath(_.value, \"cypress/e2e\")"), ", ") }}'

      transfer:
      - from: /data/repo

      fetch:
      - from: /data/artifacts

      container:
        resources:
          requests:
            cpu: 1
            memory: 1Gi

      run:

        args:
        - --env
        - NON_CYPRESS_ENV=NON_CYPRESS_ENV_value
        - --config
        - '{"video":true,"screenshotsFolder":"/data/artifacts/screenshots","videosFolder":"/data/artifacts/videos"}'
        - --spec
        - '{{ join(shard.testFiles, ",") }}'

        env:
        - name: CYPRESS_CUSTOM_ENV
          value: CYPRESS_CUSTOM_ENV_value

    artifacts:
      workingDir: /data/artifacts
      paths:
      - '**/*'

The above file creates a Cypress Test workflow and configures it to run 3 tests in parallel, specifying other details like resource requirements and artifacts folders.

Paste the yaml file's contents and click Create & Run to create and run the test workflow.  This will trigger the Test workflow, which you’ll see on the dashboard.

After the Test workflow has finished, it will update the status and give you a link to view the artifacts, such as logs, screenshots, and videos.

In this case, it has generated logs and videos. Clicking on any artifacts will open a new tab/window to view them.

This was a simple demo of creating and running Cypress tests in parallel using Testkube. To take advantage of test workflows more, you can create custom workflows and import them to Testkube.

Cypress & Its Alternatives

Cypress offers features like test parallelization and analytics that help teams quicken their testing process. While initially expensive, it now has a free tier with limitations on the number of parallel tests you can run. 

To overcome this barrier, open-source alternatives like SorryCypress and managed solutions like Currents.dev emerged, offering unlimited parallelization and features previously exclusive to Cypress's enterprise plans.

However, starting with version 12, Cypress blocked projects using the 'cypress-cloud' module, impacting these third-party services. SorryCypress now only works with older Cypress versions, while Currents.dev has ended its official support for Cypress. This change has disrupted many teams' testing workflows.

However, with Testkube, you can still run your Cypress tests in parallel without worrying about Cypress blocking Testkube. 

Summary

To summarize, Cypress is a versatile and feature-rich tool for end-to-end testing. With Testkube, you can run Cypress tests in parallel and leverage its advanced test orchestration capabilities to perform end-to-end testing.

Further, third-party services like SorryCypress and Currents.dev are no longer helpful as they don’t work with the latest version of Cypress or are forcing their users to use Playwright. But with Testkube, you’re guaranteed an uninterrupted testing experience.

Check out the Testkube website to learn more about Cypress or other testing tools you can integrate and get started with Testkube today. Feel free to post a note in our active Slack community if you struggle with anything.