Performance difference between IIS 7.5 and IIS 8

Here is a quick benchmark of IIS 7.5 and IIS 8 to get an idea of the performance difference between these two Microsoft web servers for Windows on different numbers of CPU cores and different levels of concurrent connections. IIS 7.5 was tested on Windows Server 2008 R2 Datacenter, while IIS 8 was tested on the Release Candidate of Windows Server 2012.

Test environment

For this test I had two virtual machines running on top of Microsoft’s Hyper-V virtualization platform, so note that performance of the hardware may be skewed based on the way the hypervisor itself works and the way it both manages and uses resources. Both virtual machines were base installs of the OS with IIS turned on and available Windows updates at the time applied.

The operating systems are different on both virtual machines, and this will obviously affect the outcome of the results.

  • VM1: Windows Server 2008 R2 Datacenter
  • VM2: Windows Server 2012 RC Datacenter

The underlying hardware was as follows:

  • Intel(R) Xeon(R) CPU E5530 @ 2.40GHz
  • 4GB Memory (RAM)
  • 60GB Hard disk space

I have tested the performance of both virtual machines when they have been assigned 1 CPU core, 2 CPU cores, and 4 CPU cores. 4gb of RAM and 60GB of disk went unchanged the whole time.

The benchmark tool used was Weighttpd for Windows. The classic Apache Benchmark (ab) tool was not used as this is now quite ancient, and doesn’t support threading meaning that the results it gives are not accurate especially when trying to measure scalability over multiple CPU cores.

Weighttpd was run on the local machine as the purpose of this benchmark was to gauge the performance of the web server software only rather than the difference in networking implementations etc.

Many of you will say this is useless, I disagree. Sometimes you want to know how well the engine of your car can perform, the car is limited by this engine, however you need to remember you can’t only travel on the engine alone. I may do other tests in the future running the benchmarking tool on another server, but for now this is a local test only.

The benchmarking process

Before every benchmark run, the server was completely rebooted to ensure fairness throughout all testing.

I ran 3 tests on each virtual machine while on 1 CPU core, 2 CPU cores, and then 4 CPU cores in order to get a more of an average result. This means in total there was 12 tests run on each virtual machine, for a total of 24 tests. Only one virtual machine was on and running a test at a time, and reboots took place between each of the 24 tests.

The tests were set to load http://localhost/100.html and this file was a static page that was exactly 100 bytes in size.

I ran the tests using my PowerShell script for Weighttpd as this allows me to step through different levels of concurrency whilst automating the process.

The below command was run through the script:

WeigHTTP.exe -n 100000 -c $j -t 4 -k localhost/100.html

Basically $j was a variable that started at 10 meaning the executable file run with 100,000 requests (-n) to the web server at a concurrency level of 10 connections (-c), over 4 threads (-t) to the file called 100.html on the server itself.

After the 100,000 requests were complete, the loop would end and $j would increase by 10 meaning the next test would test 100,000 requests at a concurrency of 20. We step -c up by 10 concurrent connections each time until we get to 1,000 in the final test.

-t was always set to the amount of cores the virtual machine was assigned at the time, so for a test with 1 CPU core, -t was 1, for 2 CPU cores it was set to 2, and for 4 CPU cores it was set to 4. I found this also gave the best performance results.

Modifications

I modified two things on each Windows server prior to recording my benchmark data in order to avoid TCP/IP port exhaustion which would be sure to happen with high concurrency benchmarking. The total amount of ephemeral ports was set to 50,000 and the TCP/IP socket connection timeout value was set to 30 seconds, down from the default of 240.

This basically means that the server had 50,000 ports to use to connect to itself on, and that it would release a port after 30 seconds of not being in use allowing for other connections to reclaim them.

This was done by adding DWORD values into HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters as outlined here.

Overall results

The below graph shows the summary of all results of both servers when assigned 1 CPU core, 2 CPU cores and 4 CPU cores.

Windows Benchmark IIS 7.5 vs IIS 8

As you can see, IIS 8 is definitely outperforming IIS 7.5 in this test on all numbers of cores.

Summary

Yes, the operating systems were different, however this is something I wanted to show – the performance difference between Server 2008 and Server 2012 in combination with IIS 7.5 and IIS 8. While this is only loading a static file, it shows that IIS 8 has quite a lot of improvements with it which is definitely interesting, I plan on testing dynamic content such as PHP based pages in the future.

  1. If you this got your VM`s, please test some dynamic .net code. I can help you if you dont have time, to write test page with all most used functions like strings, converts, lists and so on..

    • Honestly I have no experience with .net, if you can provide something simple to set up and test I’ll be happy to check it out.

  2. Ofc, please send me some test email. You got mine from comment email field.

    Ill make mvc project with all most used commands in c#.

  3. Hi, May i know the Maximum no.of Requests that can be handled by IIS 7.5 running on below server configuration

    Intel(R) Xeon(R) CPU E5530 @ 2.40GHz
    4GB Memory (RAM)
    60GB Hard disk space

    • Hi Mohammed,

      As those are the hardware specifications I used for this particular test, you can get the answer from the above image if you take a look at the black line – as this represents IIS 7.5 with 4 CPU cores.

      The average appears to be approximately 25,000 requests per second for a static html page.

  4. Very interesting test: very few tests cover a large concurrency range while this is the only way to see how a server really behaves under load.

    Something that seems to have survived in this new IIS evolution is the lack of multicore scalability: there is a significant difference between 1 and 2 cores but almost no gain at using 4 cores instaed of 2.

    The IIS team has made progress about single-thread performance (this is good), but not about scalability (this is sad).

    For this, I guess that we will have to wait for a new version. At least, they are improving!

  5. Damien Jorgensen

    I think the multicore test would show a bigger difference if it were dynamic asp.net content being used as then you’d see the application itself using the cores

    I don’t see how having more than 2 cores will give you anything as its just a static file

  6. Now the differance clear to me.. Thank you :lol:

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>