CopyDisable

Saturday 11 August 2018

HTTP2 Push benchmarking


To check the performance of HTTP2 push feature I used a very basic HTML page with some static content like images and to load some CSS and two heavy JavaScript files.
This HTML page will be served by Nginx webserver version 1.15.2 (since Nginx version 1.13.9, HTTP2 push is supported) on Ubuntu 16.04.

HTML file used for testing:

<!DOCTYPE html>
<html lang="en">
<head>
<title>My HTTP2 Test</title>
<script src="js/angular.js"></script>
<script src="js/jquery-latest.js"></script>
<link href="css/RatingStars.css" rel="stylesheet" type="text/css" />
<link href="css/speech.css" rel="stylesheet" type="text/css" />
<link href="css/style.css" rel="stylesheet" type="text/css" />
<link href="css/track.css" rel="stylesheet" type="text/css" />
<link href="css/speech1.css" rel="stylesheet" type="text/css" />
<link href="css/speech2.css" rel="stylesheet" type="text/css" />
<link href="css/speech3.css" rel="stylesheet" type="text/css" />
<link href="css/speech4.css" rel="stylesheet" type="text/css" />
<link href="css/speech5.css" rel="stylesheet" type="text/css" />
<link href="css/speech6.css" rel="stylesheet" type="text/css" />
<link href="css/speech7.css" rel="stylesheet" type="text/css" />
<link href="css/speech8.css" rel="stylesheet" type="text/css" />
<link href="css/speech9.css" rel="stylesheet" type="text/css" />
<link href="css/speech10.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div>
<h1>Images</h1>
<div><img src="images/1.jpg" /></div>
<div><img src="images/2.jpg" /></div>
<div><img src="images/3.jpg" /></div>
<div><img src="images/4.jpg" /></div>
<div><img src="images/5.jpg" /></div>
<div><img src="images/6.jpg" /></div>
<div><img src="images/7.jpg" /></div>
<div><img src="images/8.jpg" /></div>
<div><img src="images/9.jpg" /></div>
<div><img src="images/10.jpg" /></div>
</div>
</body>

Nginx configuration:

For simple static file testing I used the http2_push directive in Nginx configuration file.
Note: By default Nginx limits number of concurrent push requests in a connection to 10. As the number of objects that we will be pushing is more than 10 for this test, so I will be changing the http2_max_concurrent_pushes directive to some desired value. For this test I am setting this to say 50. Set this in the http section of the nginx config file
http2_max_concurrent_pushes 50;

Test Scenarios

I took four different scenarios for this benchmarking test
1. Webserver will serve the page over simple HTTP using protocol HTTP/1.1
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name _;
location /index.html {
try_files $uri $uri/ =404;
}
}
2. Webserver will serve the page over HTTPS using protocol HTTP/1.1
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate /etc/nginx/ssl/star_mkcl_org.crt;
ssl_certificate_key /etc/nginx/ssl/mkcl-org.key;
ssl_stapling on;
ssl_stapling_verify on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name _;
location /index.html {
try_files $uri $uri/ =404;
}
}
3. Webserver will serve the page over HTTPS using protocol HTTP/2
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate /etc/nginx/ssl/star_mkcl_org.crt;
ssl_certificate_key /etc/nginx/ssl/mkcl-org.key;
ssl_stapling on;
ssl_stapling_verify on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name _;
location /index.html {
try_files $uri $uri/ =404;
}
}
4. Webserver will serve the page using protocol HTTP2 with Push feature.
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate /etc/nginx/ssl/star_mkcl_org.crt;
ssl_certificate_key /etc/nginx/ssl/mkcl-org.key;
ssl_stapling on;
ssl_stapling_verify on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name _;
location /index.html {
http2_push /css/RatingStars.css;
http2_push /css/speech.css;
http2_push /css/style.css;
http2_push /css/track.css;
http2_push /css/speech1.css;
http2_push /css/speech2.css;
http2_push /css/speech3.css;
http2_push /css/speech4.css;
http2_push /css/speech5.css;
http2_push /css/speech6.css;
http2_push /css/speech7.css;
http2_push /css/speech8.css;
http2_push /css/speech9.css;
http2_push /css/speech10.css;
http2_push /js/angular.js;
http2_push /js/jquery-latest.js;
http2_push /images/1.jpg;
http2_push /images/2.jpg;
http2_push /images/3.jpg;
http2_push /images/4.jpg;
http2_push /images/5.jpg;
http2_push /images/6.jpg;
http2_push /images/7.jpg;
http2_push /images/8.jpg;
http2_push /images/9.jpg;
http2_push /images/10.jpg;
try_files $uri $uri/ =404;
}
}
Tests over internet connections are not that reliable. As I found two same test results gave big variation for the same configuration (e.g. one test gave 32 second load time and another test gave 7 second load time for the same scenario using internet connection).
So I decided to run the tests in two different environments:
· Hosted the above HTML page in public server and run the page load tests from Google Chrome and WebPageTest.org
· Hosted the HTML file in virtualbox and run the tests using Google chrome. Here I can get stable and reliable network link between the browser and the webserver. To simulate over the internet scenario, I used the tool Wondershaper to limit the incoming and outgoing bandwidth of the virtualbox VM to 1024kbps.

Test results:

Test run on Public Server using WebPageTest.org tool:



I have selected the Test Location Ireland and Browser as Chrome for my tests.
HTTP2 with Push
I run this test for 7 times and got the below results:
Performance Results (Median Run)
Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexTimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)6.925s0.777s7.000s70006.925s283,325 KB7.144s293,326 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)7.032s0.882s7.100s7100> 7.077s7.032s283,325 KB7.241s293,326 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)6.857s0.697s6.900s6900> 6.898s6.857s283,325 KB7.053s293,326 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 3)7.077s0.848s7.100s7100> 7.122s7.077s283,325 KB7.279s293,326 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)6.900s0.751s7.000s7000> 6.945s6.900s283,325 KB7.089s293,326 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)7.223s0.753s7.200s7200> 7.275s7.223s283,325 KB7.421s293,326 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)7.049s0.762s7.100s7100> 7.098s7.049s283,325 KB7.258s293,326 KB$$$$$

Considering the Fully loaded time of the webpage, we will calculate the average time: 7.212
HTTP2 without PUSH
I run this test for 3 times and got the below results:

Performance Results (Median Run)
Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)7.119s0.740s7.100s7100> 7.168s7.119s283,327 KB7.326s293,329 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)7.172s0.739s7.200s7200> 7.224s7.172s283,327 KB7.384s293,329 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)7.118s0.702s7.200s7200> 7.166s7.118s283,327 KB7.315s293,329 KB$$$$$

Considering the Fully loaded time of the webpage, we will calculate the average time: 7.342
HTTPS without HTTP2
I run this test for 3 times and got the below results:
Performance Results (Median Run)
Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 3)7.170s0.720s7.200s7200> 7.217s7.170s283,327 KB7.385s293,329 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)7.196s0.801s7.200s7200> 7.251s7.196s283,327 KB7.407s293,329 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 3)7.143s0.722s7.200s7200> 7.187s7.143s283,327 KB7.342s293,329 KB$$$$$

Considering the Fully loaded time of the webpage, we will calculate the average time: 7.378
HTTP 1.1 without HTTPS
I run this test for 8 times and got the below results:
Performance Results (Median Run)
Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)8.409s0.442s3.800s38003.806s8.409s273,327 KB8.605s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)6.760s0.410s3.400s34133.365s6.760s273,327 KB6.963s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)6.567s0.428s4.100s41004.110s6.567s273,327 KB6.768s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)6.601s0.407s4.000s40003.963s6.601s273,327 KB6.796s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)9.671s0.416s6.700s67006.740s9.671s273,327 KB9.870s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 1)6.765s0.443s3.800s38003.806s6.765s273,327 KB6.969s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)10.586s0.421s5.000s50005.055s10.586s273,327 KB10.781s283,328 KB$$$$$


Document CompleteFully Loaded
Load TimeFirst ByteStart RenderSpeed IndexFirst Interactive (beta)TimeRequestsBytes InTimeRequestsBytes InCost
First View (Run 2)6.553s0.409s3.800s38003.761s6.553s273,327 KB6.765s283,328 KB$$$$$

Considering the Fully loaded time of the webpage, we will calculate the average time: 7.9395
Final result of WebPageTest.org tool:
Here I considered the Full page load time returned by the tool.
For each test scenarios, I run the tool minimum 3 times and maximum 8 times and calculated the average of the results that I got.
The results are:
ScenarioFully loaded time (secs)
HTTP1.1 without SSL7.9395
HTTP1.1 with SSL7.378
HTTP2.0 with SSL7.342
HTTP2.0 with SSL and Push7.212






Conclusion: From the above results that we got from the WebPageTest.org tool, we can see that HTTP2 with Push feature has speed advantage over others.

Test run on Public Server using Google Chrome browser and using the developer tool:

First I tried to run the tests over our broadband connections, as broadband connections are not that reliable and I started to get weird results because of network link fluctuations. So I decided to use mobile phone 4G network for this test.




· HTTP1.1 without SSL
· HTTP1.1 with SSL



· HTTP2.0 with SSL




· HTTP2.0 with SSL and Push
Test1
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.310.918.3610.79
HTTP1.1 with SSL283.332.1932.0632.07
HTTP2.0 with SSL283.317.0411.5617.31
HTTP2.0 with SSL and Push283.37.747.637.64
Test2
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.316.9016.7616.77
HTTP1.1 with SSL283.37.235.827.12
HTTP2.0 with SSL283.310.706.8510.60
HTTP2.0 with SSL and Push283.35.784.595.68
Test3
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.310.127.6410
HTTP1.1 with SSL283.310.3910.2510.26
HTTP2.0 with SSL283.37.255.657.16
HTTP2.0 with SSL and Push283.35.795.655.66
Test 4
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.37.427.027.31
HTTP1.1 with SSL283.310.3210.1810.19
HTTP2.0 with SSL283.39.156.439.03
HTTP2.0 with SSL and Push283.312.068.6911.96

Averaging the results of the above 4 tests and selecting the Finish, DOMContentLoaded and Load times, we get the following results:
Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL11.33759.94511.2175
HTTP1.1 with SSL15.032514.577514.91
HTTP2.0 with SSL11.0357.622511.025
HTTP2.0 with SSL and Push7.84256.647.735





Conclusion: Here also we can conclude that HTTP2 with Push feature has loaded the page faster than others.

Test run on webpage hosted on virtualbox VM and requests sent using Google chrome.

In ideal scenario, the network should be reliable to get the proper results for benchmarks. But this is hardly possible in public network over internet. So I tried to simulate the scenario, using a tool named Wondershaper in my Ubuntu virtualbox VM. With the help of this tool I was able to limit the incoming and outgoing bandwidth of the virtualbox VM to 1024kbps, which is close to what we get in normal slow internet connections.
Results:
Test1
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.330.9530.8930.91
HTTP1.1 with SSL283.330.2530.5030.51
HTTP2.0 with SSL283.330.5913.9230.59
HTTP2.0 with SSL and Push283.330.7015.8030.70
Test2
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.331.1531.1131.13
HTTP1.1 with SSL283.330.8931.1631.18
HTTP2.0 with SSL283.331.8416.6831.84
HTTP2.0 with SSL and Push283.330.5715.5230.58
Test3
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.330.7926.6230.75
HTTP1.1 with SSL283.330.8531.1831.19
HTTP2.0 with SSL283.330.9115.7130.92
HTTP2.0 with SSL and Push283.330.7215.9030.72
Test4
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.331.2731.1931.20
HTTP1.1 with SSL283.330.5130.7530.77
HTTP2.0 with SSL283.330.0330.2430.26
HTTP2.0 with SSL and Push283.331.2716.4131.28
Test5
RequestsData Transfer (MB)Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL283.330.9930.9630.97
HTTP1.1 with SSL283.331.3531.5531.58
HTTP2.0 with SSL283.330.6515.4130.65
HTTP2.0 with SSL and Push283.330.4415.3130.44

Averaging the results of the above 5 tests and selecting the Finish, DOMContentLoaded and Load times, we get the following results:
Finish (Secs)DOMContentLoaded (Secs)Load (Secs)
HTTP1.1 without SSL31.0330.15430.992
HTTP1.1 with SSL30.7731.02831.046
HTTP2.0 with SSL30.80418.39230.852
HTTP2.0 with SSL and Push30.7415.78830.744







Conclusion: Here also we can see that HTTP2 Push shown speed improvement over the all the others.

Final Conclusion:

From all the above three types of tests, we can see that HTTP2 Push clearly had improved page load time. Also it reduces the time the browser spends waiting on the network.



Monday 6 August 2018

HAProxy and Traefik benchmarking

Last year I did some benchmarking test and checked the performance of two open source load balancer HAProxy and Traefik. I am publishing these tests as this might help you to choose your load balancer.
I used the following benchmarking tools:
· ab (https://httpd.apache.org/docs/2.4/programs/ab.html )
· wrk (https://github.com/wg/wrk )
· siege (https://www.joedog.org/siege-home/ )
Test environment I used:
· HAProxy 1.7
· Traefik 1.3.5
· Ubuntu 16.04
Note: These results are for tests run with default configurations for both the load balancer, without applying any optimization settings.
Results:
• In most cases we found that HAProxy performed better than Traefik.
• HAProxy yet to support HTTP/2, Traefik has full support of HTTP/2


Tool: ab

html file with size 811 bytes

ab Traefik HAProxy Observation
-c 100 -n 10000 Concurrency Level: 100
Time taken for tests: 5.468 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 9890000 bytes
HTML transferred: 8110000 bytes
Requests per second: 1828.68 [#/sec] (mean)
Time per request: 54.684 [ms] (mean)
Time per request: 0.547 [ms] (mean, across all concurrent requests)
Transfer rate: 1766.17 [Kbytes/sec] received
Concurrency Level: 100
Time taken for tests: 2.601 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 10520000 bytes
HTML transferred: 8110000 bytes
Requests per second: 3844.41 [#/sec] (mean)
Time per request: 26.012 [ms] (mean)
Time per request: 0.260 [ms] (mean, across all concurrent requests)
Transfer rate: 3949.53 [Kbytes/sec] received
HAProxy is winner here
-c 500 -n 10000Concurrency Level: 500
Time taken for tests: 7.547 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 9890000 bytes
HTML transferred: 8110000 bytes
Requests per second: 1324.97 [#/sec] (mean)
Time per request: 377.367 [ms] (mean)
Time per request: 0.755 [ms] (mean, across all concurrent requests)
Transfer rate: 1279.68 [Kbytes/sec] received








Concurrency Level: 500
Time taken for tests: 2.456 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 10520000 bytes
HTML transferred: 8110000 bytes
Requests per second: 4071.94 [#/sec] (mean)
Time per request: 122.792 [ms] (mean)
Time per request: 0.246 [ms] (mean, across all concurrent requests)
Transfer rate: 4183.28 [Kbytes/sec] received








HAProxy is winner here
-c 1000 -n 10000Concurrency Level: 1000
Time taken for tests: 8.153 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 9890000 bytes
HTML transferred: 8110000 bytes
Requests per second: 1226.58 [#/sec] (mean)
Time per request: 815.274 [ms] (mean)
Time per request: 0.815 [ms] (mean, across all concurrent requests)
Transfer rate: 1184.66 [Kbytes/sec] received








Concurrency Level: 1000
Time taken for tests: 3.368 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 10520000 bytes
HTML transferred: 8110000 bytes
Requests per second: 2968.76 [#/sec] (mean)
Time per request: 336.841 [ms] (mean)
Time per request: 0.337 [ms] (mean, across all concurrent requests)
Transfer rate: 3049.94 [Kbytes/sec] received








HAProxy is winner here
-c 2000 -n 10000Concurrency Level: 2000
Time taken for tests: 8.921 seconds
Complete requests: 10000
Failed requests: 355
(Connect: 0, Receive: 0, Length: 355, Exceptions: 0)
Non-2xx responses: 355
Total transferred: 9587740 bytes
HTML transferred: 7826100 bytes
Requests per second: 1120.92 [#/sec] (mean)
Time per request: 1784.243 [ms] (mean)
Time per request: 0.892 [ms] (mean, across all concurrent requests)
Transfer rate: 1049.52 [Kbytes/sec] received










Concurrency Level: 2000
Time taken for tests: 2.737 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 10520000 bytes
HTML transferred: 8110000 bytes
Requests per second: 3653.91 [#/sec] (mean)
Time per request: 547.359 [ms] (mean)
Time per request: 0.274 [ms] (mean, across all concurrent requests)
Transfer rate: 3753.82 [Kbytes/sec] received








HAProxy is winner with no failed request
-c 5000 -n 10000Concurrency Level: 5000
Time taken for tests: 8.062 seconds
Complete requests: 10000
Failed requests: 7288
(Connect: 0, Receive: 0, Length: 7288, Exceptions: 0)
Non-2xx responses: 3097
Total transferred: 7259056 bytes
HTML transferred: 5636250 bytes
Requests per second: 1240.34 [#/sec] (mean)
Time per request: 4031.166 [ms] (mean)
Time per request: 0.806 [ms] (mean, across all concurrent requests)
Transfer rate: 879.26 [Kbytes/sec] received










Concurrency Level: 5000
Time taken for tests: 2.500 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 10520000 bytes
HTML transferred: 8110000 bytes
Requests per second: 4000.46 [#/sec] (mean)
Time per request: 1249.856 [ms] (mean)
Time per request: 0.250 [ms] (mean, across all concurrent requests)
Transfer rate: 4109.85 [Kbytes/sec] received








HAProxy is winner with no failed request

gif file with size 160KB

abTraefikHAProxyObservation
-c 100 -n 10000Concurrency Level: 100
Time taken for tests: 21.366 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612040000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 468.03 [#/sec] (mean)
Time per request: 213.660 [ms] (mean)
Time per request: 2.137 [ms] (mean, across all concurrent requests)
Transfer rate: 73680.66 [Kbytes/sec] received








Concurrency Level: 100
Time taken for tests: 15.999 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612230000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 625.05 [#/sec] (mean)
Time per request: 159.987 [ms] (mean)
Time per request: 1.600 [ms] (mean, across all concurrent requests)
Transfer rate: 98410.42 [Kbytes/sec] received








HAProxy is winner
-c 500 -n 10000Concurrency Level: 500
Time taken for tests: 20.531 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612040000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 487.07 [#/sec] (mean)
Time per request: 1026.541 [ms] (mean)
Time per request: 2.053 [ms] (mean, across all concurrent requests)
Transfer rate: 76677.77 [Kbytes/sec] received








Concurrency Level: 500
Time taken for tests: 19.244 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612230000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 519.63 [#/sec] (mean)
Time per request: 962.215 [ms] (mean)
Time per request: 1.924 [ms] (mean, across all concurrent requests)
Transfer rate: 81813.51 [Kbytes/sec] received








HAProxy and Traefic performed almost similar way, but HAProxy has little edge
-c 1000 -n 10000Concurrency Level: 1000
Time taken for tests: 22.104 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612040000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 452.41 [#/sec] (mean)
Time per request: 2210.370 [ms] (mean)
Time per request: 2.210 [ms] (mean, across all concurrent requests)
Transfer rate: 71221.46 [Kbytes/sec] received








Concurrency Level: 1000
Time taken for tests: 21.513 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612230000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 464.84 [#/sec] (mean)
Time per request: 2151.265 [ms] (mean)
Time per request: 2.151 [ms] (mean, across all concurrent requests)
Transfer rate: 73186.86 [Kbytes/sec] received








HAProxy and Traefic performed almost similar way, but HAProxy has little edge
-c 2000 -n 10000Concurrency Level: 2000
Time taken for tests: 47.486 seconds
Complete requests: 10000
Failed requests: 100
(Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Non-2xx responses: 100
Total transferred: 1595934000 bytes
HTML transferred: 1593673750 bytes
Requests per second: 210.59 [#/sec] (mean)
Time per request: 9497.274 [ms] (mean)
Time per request: 4.749 [ms] (mean, across all concurrent requests)
Transfer rate: 32820.56 [Kbytes/sec] received










Concurrency Level: 2000
Time taken for tests: 27.242 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612230000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 367.08 [#/sec] (mean)
Time per request: 5448.332 [ms] (mean)
Time per request: 2.724 [ms] (mean, across all concurrent requests)
Transfer rate: 57795.42 [Kbytes/sec] received








HAProxy is winner with no failed request
-c 5000 -n 10000Concurrency Level: 5000
Time taken for tests: 24.660 seconds
Complete requests: 10000
Failed requests: 6382
(Connect: 0, Receive: 0, Length: 6382, Exceptions: 0)
Non-2xx responses: 6339
Total transferred: 588597280 bytes
HTML transferred: 586949329 bytes
Requests per second: 405.51 [#/sec] (mean)
Time per request: 12330.208 [ms] (mean)
Time per request: 2.466 [ms] (mean, across all concurrent requests)
Transfer rate: 23308.69 [Kbytes/sec] received










Concurrency Level: 5000
Time taken for tests: 54.773 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 1612230000 bytes
HTML transferred: 1609770000 bytes
Requests per second: 182.57 [#/sec] (mean)
Time per request: 27386.395 [ms] (mean)
Time per request: 5.477 [ms] (mean, across all concurrent requests)
Transfer rate: 28744.99 [Kbytes/sec] received








HAProxy took more time but it completed all the requests but in Traefic almost 60% requests failed.

zip file with size 850KB

abTraefikHAProxyObservation
-c 100 -n 5000Concurrency Level: 100
Time taken for tests: 44.798 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 4279650000 bytes
HTML transferred: 4278485000 bytes
Requests per second: 111.61 [#/sec] (mean)
Time per request: 895.965 [ms] (mean)
Time per request: 8.960 [ms] (mean, across all concurrent requests)
Transfer rate: 93292.62 [Kbytes/sec] received








Concurrency Level: 100
Time taken for tests: 45.018 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 4279745000 bytes
HTML transferred: 4278485000 bytes
Requests per second: 111.07 [#/sec] (mean)
Time per request: 900.367 [ms] (mean)
Time per request: 9.004 [ms] (mean, across all concurrent requests)
Transfer rate: 92838.50 [Kbytes/sec] received








HAProxy and Traefik performed almost similar way, but Traefik has little edge
-c 500 -n 5000Concurrency Level: 500
Time taken for tests: 49.729 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 4279650000 bytes
HTML transferred: 4278485000 bytes
Requests per second: 100.54 [#/sec] (mean)
Time per request: 4972.945 [ms] (mean)
Time per request: 9.946 [ms] (mean, across all concurrent requests)
Transfer rate: 84041.66 [Kbytes/sec] received








Concurrency Level: 500
Time taken for tests: 49.420 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 4279745000 bytes
HTML transferred: 4278485000 bytes
Requests per second: 101.17 [#/sec] (mean)
Time per request: 4942.045 [ms] (mean)
Time per request: 9.884 [ms] (mean, across all concurrent requests)
Transfer rate: 84569.00 [Kbytes/sec] received








HAProxy and Traefik performed almost similar way, but HAProxy has little edge
-c 1000 -n 5000Concurrency Level: 1000
Time taken for tests: 55.801 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 4279650000 bytes
HTML transferred: 4278485000 bytes
Requests per second: 89.60 [#/sec] (mean)
Time per request: 11160.222 [ms] (mean)
Time per request: 11.160 [ms] (mean, across all concurrent requests)
Transfer rate: 74897.18 [Kbytes/sec] received








Concurrency Level: 1000
Time taken for tests: 53.274 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 4279745000 bytes
HTML transferred: 4278485000 bytes
Requests per second: 93.85 [#/sec] (mean)
Time per request: 10654.734 [ms] (mean)
Time per request: 10.655 [ms] (mean, across all concurrent requests)
Transfer rate: 78452.24 [Kbytes/sec] received








HAProxy and Traefik performed almost similar way, but HAProxy has little edge
-c 2000 -n 5000Concurrency Level: 2000
Time taken for tests: 60.801 seconds
Complete requests: 5000
Failed requests: 4929
(Connect: 0, Receive: 0, Length: 4929, Exceptions: 0)
Non-2xx responses: 137
Total transferred: 4162407679 bytes
HTML transferred: 4161256678 bytes
Requests per second: 82.23 [#/sec] (mean)
Time per request: 24320.547 [ms] (mean)
Time per request: 12.160 [ms] (mean, across all concurrent requests)
Transfer rate: 66854.60 [Kbytes/sec] received










Concurrency Level: 2000
Time taken for tests: 60.992 seconds
Complete requests: 5000
Failed requests: 4971
(Connect: 0, Receive: 0, Length: 4971, Exceptions: 0)
Total transferred: 4254922479 bytes
HTML transferred: 4253669787 bytes
Requests per second: 81.98 [#/sec] (mean)
Time per request: 24396.949 [ms] (mean)
Time per request: 12.198 [ms] (mean, across all concurrent requests)
Transfer rate: 68126.51 [Kbytes/sec] received









HAProxy and Traefik performed almost similar way
-c 5000 -n 5000Concurrency Level: 5000
Time taken for tests: 39.571 seconds
Complete requests: 5000
Failed requests: 4354
(Connect: 0, Receive: 0, Length: 4354, Exceptions: 0)
Non-2xx responses: 2238
Total transferred: 2364417106 bytes
HTML transferred: 2363475652 bytes
Requests per second: 126.36 [#/sec] (mean)
Time per request: 39570.740 [ms] (mean)
Time per request: 7.914 [ms] (mean, across all concurrent requests)
Transfer rate: 58351.22 [Kbytes/sec] received










apr_socket_recv: Connection reset by peer (104)
Total of 1646 requests completed
No Clear Winner


Tool: wrk

html file with size 811 bytes

wrkTraefikHAProxyObservation
100 threads and 5000 connections
30 seconds
48985 requests in 30.10s, 39.74MB read
Socket errors: connect 0, read 0, write 0, timeout 31041
Non-2xx or 3xx responses: 8036
Requests/sec: 1627.39
Transfer/sec: 1.32MB



164998 requests in 30.10s, 162.55MB read
Socket errors: connect 0, read 169, write 0, timeout 0
Requests/sec: 5481.61
Transfer/sec: 5.40MB


HAProxy is winner here
500 threads and 5000 connections
30 seconds
47130 requests in 30.23s, 39.04MB read
Socket errors: connect 0, read 0, write 0, timeout 32929
Non-2xx or 3xx responses: 6739
Requests/sec: 1559.22
Transfer/sec: 1.29MB



164946 requests in 30.20s, 162.50MB read
Socket errors: connect 0, read 21, write 0, timeout 2
Requests/sec: 5462.42
Transfer/sec: 5.38MB


HAProxy is winner here
1000 threads and 5000 connections
30 seconds
51638 requests in 30.35s, 40.26MB read
Socket errors: connect 0, read 0, write 0, timeout 36486
Non-2xx or 3xx responses: 10532
Requests/sec: 1701.53
Transfer/sec: 1.33MB



213411 requests in 30.38s, 210.24MB read
Socket errors: connect 0, read 11, write 0, timeout 4
Requests/sec: 7024.07
Transfer/sec: 6.92MB


HAProxy is winner here

gif file with size 160KB

wrkTraefikHAProxyObservation
100 threads and 5000 connections
30 seconds
11108 requests in 30.10s, 1.19GB read
Socket errors: connect 0, read 0, write 0, timeout 6609
Non-2xx or 3xx responses: 5475
Requests/sec: 369.02
Transfer/sec: 40.57MB



12423 requests in 30.11s, 2.00GB read
Socket errors: connect 0, read 139, write 0, timeout 10848
Requests/sec: 412.62
Transfer/sec: 67.97MB


HAProxy is winner here
500 threads and 5000 connections
30 seconds
10930 requests in 30.29s, 1.31GB read
Socket errors: connect 0, read 28, write 0, timeout 6982
Non-2xx or 3xx responses: 4441
Requests/sec: 360.86
Transfer/sec: 44.33MB



8203 requests in 30.25s, 1.37GB read
Socket errors: connect 0, read 191, write 0, timeout 5636
Requests/sec: 271.13
Transfer/sec: 46.23MB


HAProxy and Traefik performed almost similar way
1000 threads and 5000 connections
30 seconds
7816 requests in 30.42s, 1.21GB read
Socket errors: connect 0, read 83, write 0, timeout 6097
Non-2xx or 3xx responses: 2088
Requests/sec: 256.93
Transfer/sec: 40.80MB



7622 requests in 30.43s, 1.28GB read
Socket errors: connect 0, read 132, write 0, timeout 5022
Non-2xx or 3xx responses: 9
Requests/sec: 250.46
Transfer/sec: 42.92MB



HAProxy and Traefik performed almost similar way

zip file with size 850KB

wrkTraefikHAProxyObservation
100 threads and 1000 connections
30 seconds
2135 requests in 30.10s, 2.17GB read
Socket errors: connect 0, read 0, write 0, timeout 1721
Requests/sec: 70.93
Transfer/sec: 73.92MB


2252 requests in 30.10s, 2.22GB read
Socket errors: connect 0, read 0, write 0, timeout 2252
Requests/sec: 74.82
Transfer/sec: 75.38MB


HAProxy and Traefik performed almost similar way
500 threads and 1000 connections
30 seconds
2192 requests in 30.11s, 2.18GB read
Socket errors: connect 0, read 0, write 0, timeout 1498
Requests/sec: 72.81
Transfer/sec: 74.31MB


2553 requests in 30.11s, 2.42GB read
Socket errors: connect 0, read 0, write 0, timeout 2545
Requests/sec: 84.79
Transfer/sec: 82.43MB


HAProxy and Traefik performed almost similar way
1000 threads and 1000 connections
30 seconds
2337 requests in 30.14s, 2.23GB read
Socket errors: connect 0, read 0, write 0, timeout 2308
Requests/sec: 77.53
Transfer/sec: 75.82MB


2432 requests in 30.14s, 2.34GB read
Socket errors: connect 0, read 0, write 0, timeout 2415
Requests/sec: 80.69
Transfer/sec: 79.59MB


HAProxy and Traefik performed almost similar way


Tool: Siege

html file with size 811 bytes

Siege TraefikHAProxyObservation
100 users
1 minute
Transactions: 11790 hits
Availability: 100.00 %
Elapsed time: 59.80 secs
Data transferred: 5.01 MB
Response time: 0.00 secs
Transaction rate: 197.16 trans/sec
Throughput: 0.08 MB/sec
Concurrency: 0.71
Successful transactions: 11790
Failed transactions: 0
Longest transaction: 0.05
Shortest transaction: 0.00










Transactions: 11828 hits
Availability: 100.00 %
Elapsed time: 60.06 secs
Data transferred: 5.03 MB
Response time: 0.00 secs
Transaction rate: 196.94 trans/sec
Throughput: 0.08 MB/sec
Concurrency: 0.55
Successful transactions: 11828
Failed transactions: 0
Longest transaction: 0.04
Shortest transaction: 0.00










HAProxy and Traefik performed almost similar way
500 users
1 minute
Transactions: 58530 hits
Availability: 100.00 %
Elapsed time: 59.55 secs
Data transferred: 24.90 MB
Response time: 0.01 secs
Transaction rate: 982.87 trans/sec
Throughput: 0.42 MB/sec
Concurrency: 6.11
Successful transactions: 58530
Failed transactions: 0
Longest transaction: 0.30
Shortest transaction: 0.00










Transactions: 58438 hits
Availability: 100.00 %
Elapsed time: 59.33 secs
Data transferred: 24.86 MB
Response time: 0.00 secs
Transaction rate: 984.97 trans/sec
Throughput: 0.42 MB/sec
Concurrency: 4.67
Successful transactions: 58438
Failed transactions: 0
Longest transaction: 1.08
Shortest transaction: 0.00










HAProxy and Traefik performed almost similar way
1000 users
1 minute
Transactions: 116628 hits
Availability: 100.00 %
Elapsed time: 59.98 secs
Data transferred: 49.61 MB
Response time: 0.01 secs
Transaction rate: 1944.45 trans/sec
Throughput: 0.83 MB/sec
Concurrency: 23.19
Successful transactions: 116628
Failed transactions: 0
Longest transaction: 0.59
Shortest transaction: 0.00










Transactions: 118018 hits
Availability: 100.00 %
Elapsed time: 59.70 secs
Data transferred: 50.20 MB
Response time: 0.01 secs
Transaction rate: 1976.85 trans/sec
Throughput: 0.84 MB/sec
Concurrency: 10.50
Successful transactions: 118018
Failed transactions: 0
Longest transaction: 0.49
Shortest transaction: 0.00










HAProxy and Traefik performed almost similar way

gif file with size 160KB

Siege TraefikHAProxyObservation
100 users
1 minute
Transactions: 11522 hits
Availability: 100.00 %
Elapsed time: 59.27 secs
Data transferred: 1768.85 MB
Response time: 0.01 secs
Transaction rate: 194.40 trans/sec
Throughput: 29.84 MB/sec
Concurrency: 2.56
Successful transactions: 11522
Failed transactions: 0
Longest transaction: 1.02
Shortest transaction: 0.00










Transactions: 11691 hits
Availability: 100.00 %
Elapsed time: 59.80 secs
Data transferred: 1794.80 MB
Response time: 0.01 secs
Transaction rate: 195.50 trans/sec
Throughput: 30.01 MB/sec
Concurrency: 2.42
Successful transactions: 11691
Failed transactions: 0
Longest transaction: 1.03
Shortest transaction: 0.00










HAProxy and Traefik performed almost similar way
500 users
1 minute
Transactions: 32702 hits
Availability: 100.00 %
Elapsed time: 59.23 secs
Data transferred: 5020.40 MB
Response time: 0.40 secs
Transaction rate: 552.12 trans/sec
Throughput: 84.76 MB/sec
Concurrency: 218.54
Successful transactions: 32702
Failed transactions: 0
Longest transaction: 7.28
Shortest transaction: 0.00










Transactions: 32489 hits
Availability: 100.00 %
Elapsed time: 59.61 secs
Data transferred: 4987.70 MB
Response time: 0.41 secs
Transaction rate: 545.03 trans/sec
Throughput: 83.67 MB/sec
Concurrency: 222.03
Successful transactions: 32489
Failed transactions: 0
Longest transaction: 7.58
Shortest transaction: 0.00










HAProxy and Traefik performed almost similar way
1000 users
1 minute
Transactions: 28082 hits
Availability: 100.00 %
Elapsed time: 59.96 secs
Data transferred: 4311.14 MB
Response time: 1.58 secs
Transaction rate: 468.35 trans/sec
Throughput: 71.90 MB/sec
Concurrency: 740.20
Successful transactions: 28082
Failed transactions: 0
Longest transaction: 9.96
Shortest transaction: 0.09










Transactions: 29985 hits
Availability: 100.00 %
Elapsed time: 59.74 secs
Data transferred: 4603.29 MB
Response time: 1.44 secs
Transaction rate: 501.92 trans/sec
Throughput: 77.06 MB/sec
Concurrency: 721.42
Successful transactions: 29985
Failed transactions: 0
Longest transaction: 8.63
Shortest transaction: 0.02










HAProxy and Traefik performed almost similar way

zip file with size 850 KB

Siege TraefikHAProxyObservation
100 users
1 minute
Transactions: 6034 hits
Availability: 100.00 %
Elapsed time: 59.50 secs
Data transferred: 4924.08 MB
Response time: 0.47 secs
Transaction rate: 101.41 trans/sec
Throughput: 82.76 MB/sec
Concurrency: 48.06
Successful transactions: 6034
Failed transactions: 0
Longest transaction: 4.76
Shortest transaction: 0.02










Transactions: 7725 hits
Availability: 100.00 %
Elapsed time: 59.38 secs
Data transferred: 6304.03 MB
Response time: 0.26 secs
Transaction rate: 130.09 trans/sec
Throughput: 106.16 MB/sec
Concurrency: 33.66
Successful transactions: 7725
Failed transactions: 0
Longest transaction: 3.57
Shortest transaction: 0.01










HAProxy performed little better
500 users
1 minute
Transactions: 5920 hits
Availability: 100.00 %
Elapsed time: 59.92 secs
Data transferred: 4831.05 MB
Response time: 4.32 secs
Transaction rate: 98.80 trans/sec
Throughput: 80.63 MB/sec
Concurrency: 426.36
Successful transactions: 5920
Failed transactions: 0
Longest transaction: 12.53
Shortest transaction: 0.10










Transactions: 6052 hits
Availability: 100.00 %
Elapsed time: 59.45 secs
Data transferred: 4938.77 MB
Response time: 4.19 secs
Transaction rate: 101.80 trans/sec
Throughput: 83.07 MB/sec
Concurrency: 426.20
Successful transactions: 6052
Failed transactions: 0
Longest transaction: 19.39
Shortest transaction: 0.22










HAProxy and Traefik performed almost similar way
1000 users
1 minute
Transactions: 5109 hits
Availability: 100.00 %
Elapsed time: 59.28 secs
Data transferred: 4169.23 MB
Response time: 9.74 secs
Transaction rate: 86.18 trans/sec
Throughput: 70.33 MB/sec
Concurrency: 839.22
Successful transactions: 5109
Failed transactions: 0
Longest transaction: 29.18
Shortest transaction: 0.05










Transactions: 5200 hits
Availability: 100.00 %
Elapsed time: 59.50 secs
Data transferred: 4243.49 MB
Response time: 9.80 secs
Transaction rate: 87.39 trans/sec
Throughput: 71.32 MB/sec
Concurrency: 856.13
Successful transactions: 5200
Failed transactions: 0
Longest transaction: 26.36
Shortest transaction: 0.24










HAProxy and Traefik performed almost similar way