Load Testing Examinator: JMeter

Yes, you can load test Examinator yourself. We tested it with 19,200 users and got really good response times.

Scenario

The scenario is that the users will login to the application and choose an exam. They will start taking exam, answer some of the questions, select few of them for review, can go back in mid of test to any other question or any question selected for review. The scenario wants to simulate the real-world situation where all users would be taking a particular exam for the exam duration. Each user waits for around 45 secs to answer the next question. The users start ramping up in 2 minutes.

Overview

# of User Sessions: 19,200
Thinktime: 45 secs
User Arrival Rate: 160 users/sec
Ramp-up Time: 120 secs
Throughput: 445 pages/sec
Response Time: 5 ms

Duration of Exam: 110 mins
# of Questions in the Exam: 100
No. of Choices/Question: 5

Load Testing Tool: JMeter

# of Tomcat nodes: 16
# of Terracotta nodes: 1
# of JMeter instances: 16

Terracotta Monitoring: Terracotta Admin Console
System Monitoring: nmon
Load Balancer Monitoring: mod_status Module
Tomcat Monitoring: JConsole
GC Monitoring: verboseGC

JMeter can be really a pain-in-ass if we are trying to load 20,000 users from a single JMeter instance or distributed JMeter instances. Distributed JMeter uses RMI to control the instances running remotely creating an overhead on JMeter. JMeter goes into series of Full GC for high number of users for a large test plan. The Examinator test plan includes a lot of HTTP Samples, around 280 HTTP request.

Tips to use JMeter for load testing Examinator:

• Avoid using GUI for load testing
• Avoid using distributed JMeter instances
  
• Reduce I/O by reducing data being saved under load per sample using bin/jmeter.properties

To make it happen, we ran 16 different JMeter instances after which we consolidated the results from each. Shell scripts was a great help to make it happen. Now we have a problem, each JMeter instance can't use same datapool (user login/passwords). We created copies of the JMeter Test plan and divided datapool of 19,200 into 16 files, each having 1200 users.

A small shell script does this for me:

if [ $# -lt 1 ];then
echo "Usage: "
exit
fi
name=`echo $1 | cut -d '.' -f 1`
while [ $i -lt $2 ]
do
echo "Creating ${name}${i}.jmx..."
cp $1 ${name}${i}.jmx
`sed -i 's/userList.csv/userList'$i'.csv/g' ${name}${i}.jmx`
done

Thus, each JMeter instances would be using a separate set of users and simulation 1200 users.