Here is a quick benchmark of IIS 7.5 and IIS 8 to get an idea of the performance difference between these two Microsoft web servers for Windows on different numbers of CPU cores and different levels of concurrent connections. IIS 7.5 was tested on Windows Server 2008 R2 Datacenter, while IIS 8 was tested on the Release Candidate of Windows Server 2012.
For this test I had two virtual machines running on top of Microsoft’s Hyper-V virtualization platform, so note that performance of the hardware may be skewed based on the way the hypervisor itself works and the way it both manages and uses resources. Both virtual machines were base installs of the OS with IIS turned on and available Windows updates at the time applied.
The operating systems are different on both virtual machines, and this will obviously affect the outcome of the results.
- VM1: Windows Server 2008 R2 Datacenter
- VM2: Windows Server 2012 RC Datacenter
The underlying hardware was as follows:
- Intel(R) Xeon(R) CPU E5530 @ 2.40GHz
- 4GB Memory (RAM)
- 60GB Hard disk space
I have tested the performance of both virtual machines when they have been assigned 1 CPU core, 2 CPU cores, and 4 CPU cores. 4gb of RAM and 60GB of disk went unchanged the whole time.
The benchmark tool used was Weighttpd for Windows. The classic Apache Benchmark (ab) tool was not used as this is now quite ancient, and doesn’t support threading meaning that the results it gives are not accurate especially when trying to measure scalability over multiple CPU cores.
Weighttpd was run on the local machine as the purpose of this benchmark was to gauge the performance of the web server software only rather than the difference in networking implementations etc.
Many of you will say this is useless, I disagree. Sometimes you want to know how well the engine of your car can perform, the car is limited by this engine, however you need to remember you can’t only travel on the engine alone. I may do other tests in the future running the benchmarking tool on another server, but for now this is a local test only.
The benchmarking process
Before every benchmark run, the server was completely rebooted to ensure fairness throughout all testing.
I ran 3 tests on each virtual machine while on 1 CPU core, 2 CPU cores, and then 4 CPU cores in order to get a more of an average result. This means in total there was 12 tests run on each virtual machine, for a total of 24 tests. Only one virtual machine was on and running a test at a time, and reboots took place between each of the 24 tests.
The tests were set to load http://localhost/100.html and this file was a static page that was exactly 100 bytes in size.
I ran the tests using my PowerShell script for Weighttpd as this allows me to step through different levels of concurrency whilst automating the process.
The below command was run through the script:
WeigHTTP.exe -n 100000 -c $j -t 4 -k localhost/100.html
Basically $j was a variable that started at 10 meaning the executable file run with 100,000 requests (-n) to the web server at a concurrency level of 10 connections (-c), over 4 threads (-t) to the file called 100.html on the server itself.
After the 100,000 requests were complete, the loop would end and $j would increase by 10 meaning the next test would test 100,000 requests at a concurrency of 20. We step -c up by 10 concurrent connections each time until we get to 1,000 in the final test.
-t was always set to the amount of cores the virtual machine was assigned at the time, so for a test with 1 CPU core, -t was 1, for 2 CPU cores it was set to 2, and for 4 CPU cores it was set to 4. I found this also gave the best performance results.
I modified two things on each Windows server prior to recording my benchmark data in order to avoid TCP/IP port exhaustion which would be sure to happen with high concurrency benchmarking. The total amount of ephemeral ports was set to 50,000 and the TCP/IP socket connection timeout value was set to 30 seconds, down from the default of 240.
This basically means that the server had 50,000 ports to use to connect to itself on, and that it would release a port after 30 seconds of not being in use allowing for other connections to reclaim them.
This was done by adding DWORD values into HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters as outlined here.
The below graph shows the summary of all results of both servers when assigned 1 CPU core, 2 CPU cores and 4 CPU cores.
As you can see, IIS 8 is definitely outperforming IIS 7.5 in this test on all numbers of cores.
Yes, the operating systems were different, however this is something I wanted to show – the performance difference between Server 2008 and Server 2012 in combination with IIS 7.5 and IIS 8. While this is only loading a static file, it shows that IIS 8 has quite a lot of improvements with it which is definitely interesting, I plan on testing dynamic content such as PHP based pages in the future.