JMeter Integration with Jenkins

This Integration will reduce the risk of performance degradations whenever you add a new feature or fix a bug on your product. if your latest commit inadvertently causes response times to go above an acceptable threshold or SLA, your CI system will automatically raise a ‘red flag’ and mark the build as ‘failed’. This will make it very easy for you to identify the cause (and the responsible person! :) behind the problem.

 Integrate as below:

Step 1:
Download  and Setup Jenkins https://jenkins-ci.org

Step 2:
To get JMeter support on Jenkins, you need to Download the Performance Plugin
Put performance.hpi  file to Jenkins/Plugin folder.

Step 3:
Add the jmeter.save.saveservice.output_format=xml line to the user.properties file (this is located in /bin folder of your JMeter installation

Step 4:
Run Jenkins as java -jar jenkins.war and wait for the “Jenkins is fully up and running" message.

Open the http://localhost:8080 URL

How to Run a JMeter Test in Jenkins

Step 1:
Create JMeter  Test Plan.
Test has to be run in non-GUI mode.

Step 2:
Add new job in Jenkins Add New Item.

Step 3:
 Add Build Step and select execute Windows batch command.

Step 4:
 Put JMeter Non GUI mode command here.

Step 5:
Add Post Build Action and select Publish Performance Test Result Report.
Put result file name with full path in Report Files field. and SAVE

Step 6: 
Click on Build Now  and see the console output.

Running JMeter test in Tarus Framework

• Introduction
• Installation
• Running JMeter Test
• Reporting

Introduction

While running load test in  Non GUI mode (CLI) it is difficut to analyse real time result we can view result only at the end of run by uploading the .jtl file. This limitation can be achive by using Tarus Framework

• Taurus, an open source test automation tool that extends and abstracts JMeter.
• Using Taurus we can create easy readable script for JMeter test plan.
• Platform-independency - runs where Python and Java can run
• Console stats and pseudo-graphic charts during the test execution
• Real-time reporting

Installation (For Windows)

Step 1: Install Java.             

https://www.java.com/download/

Step 2: Install Python.          

http://www.python.org/downloads

Step3: Install Taurus Python Dependencies.

Open command prompt. 

Install pip                             pip install --upgrade pip

Install lxml Package            pip install lxml

Install psutil Package        pip install psutil

Install Taurus                       pip install bzt

Running JMeter Test

Save your existing JMeter script in following path

let assume the path of path of python directory under C drive.

C/Python/Scripts

run following command in cmd

cd C/Python/Scripts > bzt example.jmx

where example.jmx is your JMeter Script

Reporting

After runing script using Taurus following Console stats and pseudo-graphic charts will display .

By this we can view result at real time.

NON GUI MODE

Windows

Step  1: Open commond propmt by typing  CMD in Run prompt.
Step 2 : Go to bin folder of jmeter.
Step 3: type Jmeter  -n -t jmxfile.jmx -l logfile and hit enter.

A summarizer will stsrt  displaying that means load test got started in non gui mode.

To generate HTML file type
Jmeter  -n -t jmxfile.jmx -l logfile  -e -o foldername and hit enter.

 A HTML file will generated after completion of test.

Linux

Step 1: Go to bin folder of jmeter
Step 2 : type .jmeter -n -t jmxfile.jmx -l logfile  -e -o foldername and hit enter.

Log file can be use in GUI mode to track the results any point of time of load test

Distributed Mode

Distributed testing is a kind of testing which use multiple systems to perform stress testing

Nodes of this model

•  Master: the system running JMeter GUI, control each slave.
• Slave: the system running jmeter-server, receive command from  the master and send a request to server under test.
• Target: the web server under test, get request from slaves.

Preconditions

• The firewalls on the systems are turned off.
• All the machines should be on same subnet.
• Use the same version of JMeter.

                                                                   Rodmap

Step 1:

On the slave/Master systems, go to jmeter/bin directory and execute file "jmeter-server.bat

Step 2 :

On the master systems, go to /bin directory and edit file jmeter.properites, add IP slave machine.

Step 3 :

On the master machine, run JMeter GUI and open the test plan.Click Run on the menu bar; select Remote start -> select the IP address of slave machine .

Listeners

Lite Ners

Post-Processors

Post Processor

Pre-Processors

Pre Processors

Assertions

Assertions

Timers

Timers

Thread Group

                                      Introduction

• Thread group elements are the beginning points of any test plan.
• All controllers and samplers must be under a thread group.
• Other elements, e.g. Listeners, may be placed directly under the test plan.
• Thread group element controls the number of threads JMeter will use to execute your test.
• controls for a thread group allow you to set:

  number of threads

ramp-up period

number of times to execute the test.

Ramp Up Period

• Each thread will execute the test plan in its entirety and completely independently of other test threads.
• Ramp Up Periode : If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun.
• Loop Count − Defines the number of times to execute the test.

Action to be taken after sampler error

• Continue : Continue the test even after a sampler error.
• Start Next Thread group :Start execute next thread gruop after is any sampler error occurs.
• Stop Thread : Stop executing test immediate if error occurs.

                                 Scheduler Configuration 

     You can configure the start and end time of running the test.

Other Thread Group Elements

• Stepping Thread Group.
• Ultimate Thread Group.
• setUp Thread Group.
• tearDown Thread Group.
• Arrivals Thread Group.
• Free-Form Arrivals Thread Group.
• Concurrency Thread Group.

                              Stepping Thread Group

• Lets us create ramp-up and ramp-down scenarios in steps.
• Increase and Decrease load by portions.
• Initial threads can be delayed.
• Configure hold target load time.
• Real time Preview Graph.
• This is a custom Thread Group. need to setup Standard Plugin in /jmeter/lib/ext folder .

Adding  the Stepping Thread Group from the Test Plan.

Let take a scenario as

>1,00 threads as target load
>0 seconds waiting after the test starts
>0 threads run at the immediate beginning of the test
>10 threads are added every 30 seconds with a ramp-up (or step transition time) of 5 seconds
>The target load is held for 60 seconds (1 minutes)
>Finally, 5 threads are stopped every 1 seconds.

This means

>The test begins immediately when JMeter starts, since there is 0 seconds waiting.
> Every 30 seconds 10 users will be added, until we reach 1,00 users. The first step is 1-10, the second 11-20, etc., because we defined 0 threads to run at the beginning. If we defined 5 threads to run the first step would be 6 - 16 , the second 17 - 27  etc. and the test would be completed faster.
> It will take each of the steps (with 10 users each) 10 seconds to complete. After that JMeter waits 30 seconds before starting the next step.
> After reaching 1,00 threads all of them will continue running and hitting the server together for 1 minutes.
->At the end, 5 threads will stop every 1 seconds.

             Concurrency Thread Group

• Maintain the level of concurrency.
• No ramp-down here.
• No Initial Delay.
• Doesn't create all threads upfront, thus saving on memory.
• Real time Preview Graph.

Adding the Concurrency Thread Group to the plan

Let’s look at the following scenario:
- 100 threads
- 30 minutes Ramp Up Time
- 10 Ramp-Up Steps
- 30 minutes holding the target rate

This means that:
- Every 3 minutes 10 users will be added until we reach 100 users. ( 30 minutes divided by 10 steps equals 3 minutes per step. 100 users divided by 10 steps equals 10 users per step. Totalling - 10 users every 3 minutes).
- After reaching 100 threads all of them will continue running and hitting the server together for 30 minutes

Jmeter Elements

There are several elements presents in GUI mode and each one of them having their specific use
with their execution order.

Thread Group: Specifies number of virtual users,rampup tiime.

Config Elements :Use to set up defaults and variables for later use by samplers.

Logic Controllers:Determine order of user request.

Pre-Processors:Executes some action before making Sampler Request.

Timers: Use to specify the think time.

Samplers: Specify the type of  request.

Post-Processors:Executes some action after a Sampler Request has been made

Listeners: Shows the results of the samples

Introduction of JMeter and Installation

                                 What Is Apache JMeter

How JMeter Works

Installation

Install Java

Install JMeter  http://jmeter.apache.org/download_jmeter.cgi

                                    Directory Structure

                                 Prime Test Elements 

1. Thread Group
2. Controllers
3. Samplers
4. Timers
5. Assertions
6. Configuration Elements
7. Pre-Processor Elements
8. Post-Processor Elements

                        Execution order of Test Elements

•     Configuration elements
•     Pre-Processors
•     Timers
•     Sampler
•     Post-Processors (unless Sample Result is null)
•     Assertions (unless Sample Result is null)
•     Listeners (unless SampleResult is null)

Parameterisation /Correlation /Synchronization

Parameterisation 

Using Parameterization we can execute one test plan for more than one user at the same time. This is a method of generalizing an action for many users.

Suppose, we have to do load test of a search engine, for that we record the search scenario. Now to do effective load test, we can't just playback the script for say 100 users. We should simulate 100 users that search for 100 different things. Here we need parameterization, wherein we can have a CSV file that contains 100 different items. Now instead of the static search item that we had recorded in our script we can include the search items from the CSV file dynamically.

                          We will use 'CSV Data Set Config'. to achieve this in JMeter

Correlation 

• Correlation is an important aspect in creation of performance testing scripts. It fetches dynamic values from earlier responses and supplies those values to subsequent requests.
• It’s a procedure to fetch a dynamic value from earlier responses and using that value in subsequent steps
• Correlation in JMeter can be achieved using Regular Expression Extractor,HTTP URL Rewriting etc. post-processor. 

Suppose we have recorded a scenario in which 

User enters login details and click OK button

Home page opens and user take further actions 

      Now, if we just playback this script, the test will fail even for a single user. This is because of the authentication mechanism used. When we login to a website, session variables are dynamically created. These session variables are passed to the subsequent requests and help validation and authentication of the actions performed. So, one cannot just record and playback the requests having these variables. Here, we need to correlate the web requests with the dynamic variables. And for correlation,

 

    we need to use the "Regular Expression Extractor" which makes use of regular expressions.

Synchronization 

• Coordinating  two or more activities, devices, or processes in time.
• We need  Synchronization to avoid the condition like deadlock.

         We can achieve synchronization  by using 'Interthread commincation Preprocessor'and    'Synchroization Timer'

Introduction of Performance Testing

Performance Testing  

Performance testing is the general name for tests that check how the system behaves and performs. Performance testing examines responsiveness, stability, scalability, reliability, speed and resource usage of your software and infrastructure. 

When should you use Performance Testing?

Test Continuously in each iteration in Agile Methodology. 

Load Testing 

Load testing is testing that checks how systems function under a heavy number of concurrent virtual users performing transactions over a certain period of time

When should you use Load Testing?

When you want to determine how many users your system can handle

Stress Testing 

Stress testing is testing that checks the upper limits of your system by testing it under extreme loads. The testing examines how the system behaves under intense loads.tress testing also examines memory leaks, slowness, security issues and data corruption.

When Should You Use Stress Testing?

Stress tests and app stress tests are important before major number of user going to hit.

Spike Testing 

If your stress test includes a sudden ramp-up in the number of virtual users,

Soak Testing

If you stress test for a long period of time to check the system’s sustainability over time with a slow ramp-up

Start Journey with JMeter

Index

1: Introduction of performance testing and it's  types.

2: Metrics and other important concepts.

3: Introduction of JMeter and Installation

4: Briefing of JMeter Elements with Example

5: How to do distributed testing

6: Performance Testing in Non GUI mode

7: Database Testing 

8: Test your web services

9: Functional Testing using JMeter

10: IP spoofing

11. Taurus Ingratiation .

12. Jenkins Integration.

13. API Testing