k6 - Performance Automation Testing

What is Performance testing?

The objective of performance testing is to not to test the functionality but to identify and remove the performance bottlenecks in the application

The goal of performance testing is to verify:

• Application Performance (pointing out bugs when more users using the app)

• Stability (Resource scarcity like CPU, Memory, Configurations and Connections)

• Scalability ( Scale users from ‘1’ number of users to ‘n’ number of users)


Smoke Testing: To check whether the system is able to handle the required basic and necessary minimum load and verify how it behaves

Smoke test intends to:

  • Check your Test Script do not have any errors
  • Check that system do not throw any errors with your minimum load


Load Testing: To evaluate the system performance at normal and peak loads in terms of specific number of concurrent users or request per second

Load test intends to:

  • Evaluate the current system performance under typical and peak load
  • Ensure the performance standards are maintained continuously when changes are made to system’s code and infrastructure


Stress Testing: Verifying the stability and reliability of system under a heavy load or extreme conditions like to check how the system behaves when sudden increase in active users traffic to arbitrate the limits of the system

Stress test intends to:

  • Determine system’s performance and behavior under maximum load conditions
  • Determine what is the maximum load condition that causes the system to fail
  • Determine the breaking and failure point of the system
  • Determine how the system recovers this with manual intervention or automatic way at this point


Spike Testing: Load testing with ramping up and ramping down of the sudden load which is basically to suddenly increase and decrease in the load. Spike testing is like a subset of stress test.

Spike test intends to:

  • Determine systems performance under sudden rise in the user’s traffic
  • Determine whether the system recovers when the traffic is abated.


Soak Testing: Load testing a system to verify the reliability and stability of the system under a specific load over a longer period of duration time.

Soak test intends to see:

  • To identify memory or resource leaks, Insufficient storage
  • To identify Incorrect configurations or Infrastructure failures
  • To identify where there’s an incorrect database tuning or high volumes of data

What is K6?

K6 is an open-source performance testing tool usually referred as ‘Load tests as code’. K6 is earlier known as Load Impact. It is written in Golang supporting ES6 of JavaScript for writing your load tests with features like Metrics, Checks and Thresholds (https://k6.io/docs/using-k6/).

K6 is easily integrated with CI/CD unlike JMeter.

K6 idea is to dispense a good developer experience in writing load tests or from tester’s perspective who know basics of JavaScript programming language. K6 is now acquired by Grafana labs in June 2021.

K6 Supports:

  • Databases Influx DB
  • k6 Test Script Recording Test Recorder
  • Orchestration tools Docker, Kubernetes and Prometheus
  • Test case development in JavaScript
  • Well integrated with CI Tools like Gitlab, Circle CI, Travis, Jenkins etc.
  • Grafana for creating performance reports and metrics

  • Protocols: Supports Http and Web sockets

  • Design of Test cases: Testers can develop test cases with the basic knowledge of JavaScript
  • Ramping up Virtual users: It’s default 
  • Resource Consumption: As it is lightweight the memory consumption is less
  • Test case recording: Recording and auto generates test code in JavaScript for your test cases

K6 Official documentation https://k6.io/docs/

Automated Performance Testing

  • Load Testing
  • Stress Testing
  • Spike Testing
  • Soak/endurance Testing but takes very long time

Automated Performance Testing factors:

  • Check on particular load level
  • Evaluation of metrics and results
  • Require a dedicated single test environment
  • Require basic JavaScript and tool knowledge
  • Integration with CI/CD pipeline
  • Test report on performance of the system

When do we prefer k6 over other performance testing tools?


With K6 :

  • When our Tech-stack is in JavaScript/TypeScript
  • Good features like Checks and Thresholds work very well for goal-oriented, automation-friendly load testing
  • Open source, good amount of users and good documentation
  • HAR files to generate and record sessions
  • Lightweight uses JavaScript and supports TypeScript
  • Does not run in NodeJS and doesn’t run in a browser is a huge benefit
  • Extremely lightweight
  • Integrate with CI/CD
  • Automation enthusiastic
  • Maintained easily and Goal based load testing
What is HAR ? 

HAR - HTTP Archive Format is a JSON archive file format that stores session data over many browsers (IE, Firefox, Chrome, Safari, etc.). HAR files can contain sensitive data, including the content of the pages you downloaded when creating the file and your cookie data.

Anyone with the HAR file can emulate your account and has access to the information submitted (e.g. personal details, passwords, credit card numbers, etc.)


Influx DB and Grafana:  K6 can send output of the results to Influx DB and when Influx DB is integrated with Grafana, We can see the Test performance metrics on Grafana Dashboards. However, K6 have a problem with Influx DB version 2.0 and have to fix this. Until Influx DB version 1.8 this metrics on Grafana is supported. However, you can do this setup on k6 cloud easily.


GitHub issues: https://github.com/grafana/k6/issues/1730#issuecomment-797360302


Sample boilerplate code: k6 Repo


Conclusion: 

It usually depends which tool is a good fit for our respective projects. Apache JMeter is quite popular from long time and people prefer it. On the other hand, there are projects in python language using Locust performance testing tool. 


Some others use Scala language in their projects so they prefer using Gatling. Similarly if the tech-stack is in JavaScript/TypeScript using k6 would be a good option. 

However all the performance testing tools have their own unique features and benefits and selecting a specific tool depends on our own project requirements.


Happy Testing Guys :) !


Comments

Post a Comment