Performance Testing of a Sitecore Website

Finally, I am able to fetch some time for my blog writing and here i am with my first ever post as a Sitecore developer. In this post I delineate about performance testing of a Sitecore 8.1 based website which I performed few months ago for a client in Saudia Arabia.

Following are the goals, I wanted to achieve from this test

  • To determine the load of the maximum no. of users the infrastructure of the website can handle.
  • Determine the load of Simultaneous multi-geo generated VUs (virtual users).
  • Scenario based VU load.
  • Real-time performance counters, while running the test, and final results for the concurrent VUs (Virtual Users).
  • Validate no. of VUs load and the page visits by VU from Sitecore DMS.
  • How to improve performance of the website after the first iteration of the performance test results.

For testing I utilized Load Impact to generate virtual user’s traffic with specific scenarios.

Azure Based Infrastructure

Infrastructure used for the testing is the exact replica of Production Environment.It has two Content delivery servers with separate SQL database servers behind the load balancer. For the DMS, Three VMs were allocated one for Arbiter and two for Mongo Collection Servers.

Following are the specs of the Azure hardware used.

Infra

 

Azure Architectural Deployment Diagram

Environment

 

Testing Scenario

In this scenario, 24 pages were selected from the entire website. These pages had unique presentation layouts and data sources. These pages were from different sections of the website which included English and Arabic languages.

Performance Test – 500 VU (Virtual Users) for 15Mins – 24 Pages

Test was executed on website with:

  • 500 Virtual Users load.
  • Duration of 15 minutes.
  • 25 GB of data transferred.
  • 635382 requests generated.

Scenario was built for 24 pages. Total 202 were the unique URLs, including static assets like js, css, images etc., which were requested by the server during test execution.

Average Page Request Time

In the below screenshot of results, the transaction time metrics called “Test min”, “Test max” and “Last” are visible. These metrics are special: they are calculated by the client-side JavaScript based on all the individual samples it has retrieved.

  • Test max is the maximum transaction time seen at any point during the test, for the URL in question.
  • Test min is similarly the minimum transaction time seen at any point.
  • Last is the last average transaction time seen for that URL. The “Last” value is usually only interesting for tests that are still running, as it shows you the current state.
  • Count shows how many times a page accessed by a single user.

 

LoadImpact Pages Overview

PageStats

Page Statistics

Results – CPU, RAM and Bandwidth Utilization

On the load of 200 active users during the test on the website, following metrics were recorded

  • Bandwidth usage – 19 MB
  • 27 GB data received
  • 168749 requests
  • Request rate of 898 reqs/second.

GeneralStats-1

 

With 201 concurrent Virtual Users the CPU, RAM and Network utilizations of both the Content Delivery servers were as below

Counters-1

 

On 353 active users

  • Bandwidth usage – 29 MB
  • 79 GB data received
  • 402053 requests
  • Request rate of 862 reqs/second

GeneralStats-2

 

With 353 concurrent Virtual Users the CPU, RAM and Network utilization of both the Content Delivery servers were as below:

Counters-2

 

On 492 active users

  • Bandwidth usage – 94 MB
  • 47 GB data received
  • 619245 requests
  • Request rate of 715 reqs/second

GeneralStats-3

 

With 492 concurrent Virtual Users the CPU, RAM and Network utilization of both the Content Delivery servers were as below:

Counters-3

 

Results – Virtual Users & Load Time Graph

Graph

 

Understanding the Results

Table-Stats-1

 

Understanding – Test & Results

In the above screenshot of Page Statistics, count (no. of times each page accessed by UV) of the pages are between 1400 to 1800. So upon average calculation (1400 + 1800) / 2 = 1600 times Each page was accessed approximately 1600 times.

Conclusion 1: Each page from the above scenario (with 24 pages) was accessed 1600 times by 500 Virtual Users in the duration of 15mins. This means that each user executed scenario 3.2 times by the formula (1600 pages /500 VUs = 3.2).

Conclusion 2: Around 77 pages were accessed by each user in 15 mins of duration. Formula 77 pages = (3.2 * 24 (pages in given scenario) )

Conclusion: Above conclusions show that the scenario was very idealistic and extreme considering the load of the users and pages accessed by users in the duration of 15mins.

If we stick with the above scenario in order to determine the maximum number of users the system can handle. The table below gives a brief insight.
Moreover the statistics below are generated on the basis of concurrent users not higher than 500 on any given time.

Table-Stats-2

Note: Users column doesn’t show concurrent users but only 500 concurrent users at a time

The table above also shows that if the website has only 500 concurrent users in 15mins of each hour of the day (24 hours) then it can handle 48k users in a single day and 14,40,000 users can be handled by the website in a month.

Normal internet users usually do not access pages so frequently as depicted in the above scenario where 77 pages have been accessed in the duration of 15 mins. Even if we assume that a normal user access 3 pages in a minute then 3 * 15mins = 45 pages are accessed by VU in 15mins. Again it’s not normal that 500 or all the users keep accessing 45 pages throughout the visit of the website in 15mins.

Considering normal/actual user’s behavior it can be safely stated that the above test results can be the same with 1000 to 1500 normal concurrent users.

Test was simulated to put stress and load on the infrastructure and determine the breaking point of the system, however the results shows that even with such peak load It can sustain and keep serving the traffic.

 

VUs Recorded by Sitecore

Sitecore recorded the data of the load test performed which validates the test and proved that the load was on the servers.

 

Sitecore Path Analyzer

In order to support the test reliability, the utilization of the path analyzer shows the path followed by the VUs during test. It is the same path which was in the scenario created by LoadImpact utility.

Here is the screenshot with some details below

PathAnalyzer

In the left corner the Inverse Pyramid displays the Incoming Traffic count whereas the section on the right hand side depicts the flow / trend of how the pages were accessed by all 500 virtual users.

As per the screenshot above, you can see the page-view is starting from 1,903 and ending at 1,411. Upon average calculation (1903 / 1411) = 1657 would be the count of pages accessed by UVs during 15mins of test. 1657 is almost the same value calculated above with the help of LoadImpact screenshot “Page Statistics”.

 

Sitecore Experience Analytics

Below screenshot gives an overview of the same scenario explained in above screenshot (Path Analyzer).

It is also showing the same count of the pages accessed by users during test.

Dashboard

 

Not satisfied?

If you’re not satisfied by the performance of your servers and you’re expecting more traffic than what is mentioned in this test, then following further options can be taken into account

  • Adding 3rd or another Content Delivery server in the environment. This will share the load and the performance counter will show less utilization of the resources. If you have azure deployment, then adding another web server is a minutes job.
  • To boost the performance of the website and putting less load on server, you can configure CDN to deliver your static or CMS based content like images, js, css etc.

 

One thought on “Performance Testing of a Sitecore Website

  • I savor, cause I found just what I was looking for. You have ended my four day lengthy hunt! God Bless you man. Have a great day. Bye

Leave a Reply

Your email address will not be published. Required fields are marked *

rfwbs-slide