We now have over 4,000 customers using RDStation, so before making changes that could impact the product, we need to simulate system-like loads in real time to assess acceptability and options before putting into production. We recently considered removing the cache from one of our endpoints, to assess the effectiveness of that cache before taking any action, a benchmark was created to simulate multiple users interacting with the endpoint. This simulation happened with the JMeter tool, which is the subject of this post in question.
What are load tests for?
Load tests are used to evaluate and validate the operational limits of a software’s processing, simulating a real environment, considering data transfer time and transaction response time. In addition, it is an ally in the development of more aware metrics, together with more professional tools such as JMeter, which provides several resources for the development of test plans, whether complex or not.
Our endpoint tests were performed in a plan simulating 100 users logged in simultaneously, where each user fires 100 requests at 5 second intervals.
JMeter: an opensource tool for load testing
The JMeter is an open source tool designed to create and run load tests in computer services. For the preparation of test plans, JMeter can help you with:
- Configure various types of requests
- Create loops and logic conditions for each request
- Import data into the plan via csv files (users, passwords)
- Configure parallelism through the number of threads, the amount of execution of each thread and the interval between each one
- Create more efficient tests by simulating multiple users and independent requests
- Simulate an attack on your server
- Present test results in various ways (tree, table)
Installing the tool
For installation just download JMeter and unzip the file. In the software folder structure, we have the folder binwhere the executables are: jmeter.batfor Windows and jmeter.shLinux systems.
The test plan is a container for running tests, defining what and how it will be tested. A plan is composed of one or several elements, as we can see in the image above, each line below the Test Plan is its own element, such as thread groups, logic controllers and configuration elements. When we open JMeter, an example default plan is suggested, ready to be adapted to your needs.
Adding configuration elements
The first element we will add is Padrões de Requisição HTTP (HTTP Request Defaults), which sets default values for HTTP requests. To add the element, just right-click on the Test Plan then Add > Configuration Elements > HTTP Request Patterns .
HTTP Request Defaults
In this configuration element we can define the default values of the HTTP requests that will be added later in the test, such as the address and the port. With this element, it is not necessary to repeat information that is standard in all HTTP request elements. Below, the element with the added information.
If necessary, configuration for proxy server can also be done.
HTTP Cookie Manager (HTTP Cookie Manager)
CSV Data Set Config (CSV Data Set Config)
This configuration element allows reading CSV files and assigning this data to JMeter variables that can be used during testing, such as user login data. The file txtmust be saved in the same location as the test plan file.
User group (Thread Group)
The thread group is the beginning of any test plan, it is below it that all controllers and testers of the plan will be.
In it we can establish settings such as: number of users (threads), interval time for each user ( ramp-up period ) and the number of times the test will be executed ( loop count ).
HTTP request (HTTP Request)
The HTTP request is the main element of a test plan for a web system, there can be one or several in the same plan. In it, we can define the verb that will be used ( POST, GET, PUT, etc.), send parameters with the request , such as the email and password of a user in a POST, among others that will be commented shortly afterwards. Below is the image of the request element that we used to GET on the Login page.
Attributes for Configuration:
- Name (Name) : Use the name to remember the element’s responsibility. Example: “Request new password email” or “login”.
- Server Name or IP (Server Name or IP) : Address or IP of the site that will be tested, for example: meusite.com.br. If you have not previously set the default value in Padrões de Requisição HTTP, the HTTP Request will use the information from this element.
- Port Number : Here you must specify the port that the server responds to, if it is not the port 80, like when we run a rails app server locally, the port could be changed to 3000.
- Protocol (Protocol) : The default protocol is HTTP , but it can be overwritten if it is necessary to use the HTTPS protocol .
- Method (Method) : If the request is making a request, the GET protocol must be maintained, but if you are sending data you will possibly need a POST or PUT, or if you are deleting, a DELETE.
- Content Encoding : This attribute establishes the encoding format of the data that are sent with the request, by default this format is UTF-8 , but it can be changed according to your needs.
- Path (Path) : The path is the page that will be accessed, complementing the server name or IP. For example in a POST request to login, we could establish the path to /login.
- Send parameters with request : Here the parameters that must be sent together with the request can be added, specifying the name and its respective value, as shown in the image below, where the request logs a user through one POSTsending the email and password, in addition to the token for authentication of the request. The values in this request are from JMeter variables created during the execution of the plan, such as USERand PASSWORDwhich were created in that configuration element for Dados CSV, which we added earlier to read from a file txt.
Regular Expression Extractor
This element can be used to obtain data from a page through regular expressions , this data is assigned to variables that can be used in other stages of the plan later.
In this example we are extracting the authentication token from the GET request made on the login page and saving it in a variable called TOKEN, which is used in the POST request commented above, which logs the user in by sending his data.
Iteration Controller (Loop Controller)
The Loop Controller will make the elements contained within the Controller execute as many times as Contador de Iteraçãospecified, or forever if the check-box Infintois checked. Next is the image that shows the controller in the plan, with an HTTP request element inserted. This makes the request GET campaignsrun 100 times for each user, that is, for each Thread Group.
Format for viewing results
The element for viewing the results can be added by right clicking on the Test Plan then Add > Listener > View Results in Table or View Results Tree . The information in each element is presented as follow
running the test
With the element to visualize the results added and selected, we can execute the plan through the menu, clicking on Run > Start and following the results.
In the visualization of the results in table , used here, we can observe some important data that can be analyzed:
- Sample time: Total request time in m/s .
- Status: Shows if the request was successfully executed or if there were failures.
- Bytes: Amount of data returned by the server.
- Latency: Difference between the time the request was sent and the time the response was received.
In order to be able to better analyze the data, we export it to an electronic spreadsheet. To export, just select all requests in the results visualization table (one CTRL + Aworks), copy them with CTRL + Cand paste them into the spreadsheet.
analyzing the results
Tests were run in the RDStation staging environment, which runs in a production-like environment with only reduced hardware.
At the end of the test plan execution, we export data from all requests and group by cached/uncached. We found that removing the cache cost approximately 25% .
Analyzing the results we see that the cache has an efficiency of 25% for this scenario. Which is interesting for RDStation with a rapidly growing customer base.