JavaScript Engine Benchmark Test Results

All the rage seems to be who has the fastest JavaScript engine these days. Chrome has it's V8 engine, Firefox with their TraceMonkey, and there is also the WebKit project. What I wanted to do was get some hard data to compare all these engines along with older browsers.

In an earlier post, I showed the difference between the stable release of Firefox vs Firefox 3.1 Beta 1 w/ TraceMonkey enabled using the SunSpider benchmark. Google offers their own benchmark testing as well, so I figured why not try them both out?

The Results

After running the Google tests it became aparent these tests were not cross-browser balanced. This test went through a series of 5 different tests and the higher the score the better. Take a look at this chart illustrating the results:

Unbalanced Results

  • IE 8: 30
  • Firefox 3.04: 156
  • Firefox 3.1b1 (w/ TraceMonkey): 49
  • Chrome: 1659
  • Safari (Win): 132
  • Opera 9: 172
  • Safari w/ WebKit: 1259

As you can see these benchmark tests are specifically for what Chrome's V8 engine is working on. In fact one of the tests completely fail (returning a score of 1) when run in Firefox's TraceMonkey engine. The only engine that came close to Chrome was WebKit on Safari. I first planned to only run these tests but after looking at the results it became clear this is not a good comparison of JavaScript rendering power.

This lead me to use the SunSpider benchmark which is much more balanced and more of a "real world" test. These results are much more accurate in my opinion and are based on the time to complete things rather than who can get the biggest score. Here's a chart of the results:

Balanced Results

Now you can see a much clearer picture in how the engines stack up. TraceMonkey is almost a full eighth of a second faster than the V8 engine. That eighth of a second gives the Firefox engine a 1.31% gain in performance over Google's engine. Both of these engines are still in beta releases of software and will be interesting to see what happens in the future.

All I have to say is that I've never had a problem with Gmail load times until I started using TraceMonkey. Now whenever I'm on a slower JavaScript engine it feels very sluggish. I am thoroughly impressed by what the TraceMonkey team has come up with and most importantly this will cause rougher competition among browsers and bring full scale web apps up to par with desktop applications. It will definitely be an interesting next couple of years to see how this all plays out.

If you're skeptical about these results head over Brendan's Roadmap Updates where he runs similar tests and goes into much more detail to show TraceMonkey being 4x faster than V8. He also uses the latest nightly build to perform the tests where you will see even better results from the TraceMonkey engine. These two engines are no where near being final releases so who knows which one will outperform in the long run.

For WebKit, it does seem to have the best results and outperform all other engines, however, it is unlikely this will really be used in any browser. This engine is more of a proof of concept (and yes it works wonderfully) since Safari is only really used on Mac OS X.

Testing Environment

Testing was done in a virtual machine (VMware Workstation 6.5) running Windows XP SP2 with Ubuntu 8.10 as the host. I loaded each browser and ran the tests all individually and compiled the information using Google Docs. If you have any questions feel free to comment or drop me an email.

Tags: