Load Testing

Load Testing vs Stress Testing

Both load and stress tests simulate a high level of traffic to see how the applications and systems respond. The difference is in the purpose and objectives.

Load Testing:

• Purpose: Load testing simulates normal or expected user loads (including high peak loads) on a system to evaluate its performance under typical usage conditions.
• Objective: The objective of load testing is to assess how the system handles the anticipated workload, measure response times, throughput, and resource utilization, and ensure that performance meets predefined criteria.

Stress Testing:

• Purpose: Stress testing involves subjecting a system to extreme or beyond-normal loads to evaluate its behavior and performance under such conditions.
• Objective: The objective of stress testing is to identify the system's breaking points, measure its stability, and understand how it recovers from failures or abnormal conditions.

Load tests are primarily designed to ensure that user expectations, such as Service Level Agreement (SLA) contract conditions, are met. Stress tests aim to determine how a system would behave under an extreme load, such as a DDoS attack, and determine a maximum limit rather than identify bottlenecks, to aid disaster and extreme event planning.

What are popular load testing tools?

There are several popular load testing tools available that help simulate user loads and assess system performance. Here are some widely used load testing tools:

• Apache JMeter: Apache JMeter is an open-source load testing tool that allows for performance testing of web applications, APIs, and various protocols. It offers a user-friendly GUI, supports distributed testing, and provides a wide range of performance testing capabilities.
• Gatling: Gatling is an open-source load testing framework written in Scala. It focuses on performance testing for web applications and supports scripting in a highly scalable and reactive manner. Gatling offers a lightweight and code-centric approach to load testing.
• LoadRunner: LoadRunner by Micro Focus is a popular commercial load testing tool. It supports testing for a wide range of applications, including web, mobile, API, and enterprise applications. LoadRunner provides comprehensive features, including load generation, monitoring, and analysis.
• NeoLoad: NeoLoad by Neotys is a commercial load testing tool that focuses on user experience and real-world simulation. It supports various application types, including web, mobile, and API. NeoLoad provides advanced features like automatic script maintenance, dynamic infrastructure scaling, and integrated reporting.
• Apache Bench (ab): Apache Bench is a command-line tool included with the Apache HTTP Server distribution. It is a simple and lightweight tool for load testing web servers and can generate significant loads on a single machine.
• Locust: Locust is an open-source, Python-based load testing tool that emphasizes ease of use and scalability. It allows developers to write load test scenarios using Python code and supports distributed load generation.
• k6: k6 is an open-source load testing tool with a developer-friendly approach. It is designed to be scriptable in JavaScript and provides a user-friendly command-line interface. k6 offers real-time and detailed test result analysis.
• Artillery: Artillery is an open-source, extensible load testing tool focused on testing APIs and backend systems. It allows users to write test scenarios using YAML or JavaScript and supports distributed testing and reporting.
• Tsung: Tsung is an open-source distributed load testing tool specifically designed for stress testing web applications. It supports various protocols, including HTTP, WebSocket, and XMPP.

These are just a few examples of popular load testing tools available. The choice of tool depends on factors such as the application type, scripting capabilities, scalability requirements, reporting features, and budget considerations.

What load testing tools are available for testing digital workspaces like Citrix and VMware?

For load testing specifically targeting Citrix and VMware environments, there are specialized tools available. Some load testing tools commonly used for Citrix and VMware include:

• Login VSI: Login VSI is a commercial load testing tool designed specifically for testing virtualized desktop environments, including Citrix virtual apps and desktops, and VMware Horizon. It focuses on simulating real user workloads to assess the performance and scalability of virtualized environments.
• Citrix Virtual Apps and Desktops (formerly XenApp and XenDesktop) Provisioning Services (PVS): Citrix PVS includes built-in load testing capabilities for testing the scalability and performance of virtualized applications and desktops. It allows administrators to simulate different user workloads and measure the impact on the environment.
• VMware View Planner: VMware View Planner is a load testing tool specifically designed for VMware Horizon View virtual desktop infrastructure (VDI) environments. It enables the creation of realistic workloads to assess the performance and scalability of VMware Horizon deployments.
• Automai: This is another commercial load testing tool designed for digital workspaces.

What causes performance bottlenecks when load testing?

If you find a bottleneck or performance issue when load testing, you will need to pinpoint the root cause of the issue. For example, if testing web page or service availability and performance - different web pages could be slow due to a variety of reasons. These bottlenecks could lurk anywhere in your system. Common root causes you will need to eliminate when investigating load testing issues include:

Code

• Inefficient frontend code (example, React or Angular suboptimal rendering)
• Inefficient server-side code (Java thread deadlocks)

Database

• Suboptimal SQL queries
• Poorly tuned database engine
• Application Infra
• JVM bottlenecks (garbage collection, leaks)
• App server bottlenecks (connection pools, worker threads)
• Slow external dependencies

OS / Infra

• Network issues such as faulty NIC card or improper routing rules
• Suboptimal infrastructure (OS, VM, containers)
• Docker containers and Kubernetes orchestration
• … and many more.

How can using monitoring tools in addition to load testing help identify performance bottlenecks and optimize the system?

Load testing tools such as JMeter can highlight that the system under test is not able to handle the load, but will not tell you the reason why. Monitoring tools are needed to highlight where the performance bottlenecks like and how to resolve them. This is why IT monitoring tools complement load testing tools. See more: How APM Solutions Enhance JMeter Load Testing Visibility (eginnovations.com).

In an ideal situation, monitoring tools will be deployed in the IT environment and used during the load test. Once the performance bottlenecks are remediated, load tests can be re-executed to confirm that the system is operating well.

Are there any industry standards or guidelines for load testing that I should follow?

To some extent there are industry standards and guidelines for load testing that can help ensure effective and reliable load testing practices. There are a number of bodies that offer guidance or recognized certification programs for professionals involved in software testing. Here are a few well know organizations:

• The International Software Testing Qualifications Board (ISTQB)
• The Software Engineering Institute (SEI) - Performance Testing: SEI, is a part of Carnegie Mellon University
• The Performance Testing Council (PTC)
• The World Wide Web Consortium (W3C) - Web Performance Working Group: W3C's Web Performance Working Group develops standards and specifications related to web performance.

If looking to formulate a load testing strategy the materials, certifications, training and courses offered by these organizations may be of interest.