There’s been a bit of hullabaloo this morning around a comment John Resig made on David Walsh’s blog where he asserts that Sizzle is 2x faster than MooTools. I posted this morning about how this is somewhat of an unfair comparison - that Sizzle is a selector engine and MooTools is a framework (that has its own selector engine).

The real question here is not which selector engine is faster (though that’s pertinent) but which framework is faster, by how much, and if it’s enough to matter. Valerio’s thoughtful post about why MooTools wouldn’t use Sizzle outlined several reasons why it doesn’t make sense, but the biggest reason is that the MooTools selector engine is actually faster than Sizzle when used as the MooTools selector engine. In other words, if MooTools dropped in Sizzle (which is larger in file size) for its current selector engine, our users would not see a benefit. This is really the main issue - he wasn’t arguing that MooTools is faster or as fast as jQuery - only that including Sizzle didn’t make sense.

So David posts instructions on how to use Sizzle, if you really want to, in MooTools. He asserts that doing so would provide no performance benefits and no functional benefits, which is really just an echo of Valerio’s argument. Again, he’s not comparing MooTools to jQuery - just that Sizzle + MooTools doesn’t really get you anything, and I agree.

So in an effort to understand that question, just that question - is MooTools w/ Sizzle faster or slower than MooTools without it, I put together a new SlickSpeed test. This test is the default SlickSpeed (Valerio made a version originally to test Sizzle that only ran the selector once because Sizzle used to cache but it doesn’t any more - this version of SlickSpeed runs each test numerous times).

In an effort to answer the main question, I’ve included in this test Sizzle, the MooTools selector engine on its own, and MooTools, and MooTools using Sizzle (as David outlined in his post). For good measure, I also include jQuery, though as it doesn’t post-process elements, you’ll see that it’s basically on par with Sizzle, as it should be.

I’ve made a google spreadsheet that others can add their results to. I’ll update this graphic below as more data comes in. I don’t have FF2 or IE8 handy, so if someone would be so kind as to fill that in…

As you can see, MooTools + Sizzle isn’t faster than MooTools on its own. This is the main question I wanted to answer.

This means that the selector engine in MooTools is the right choice for MooTools. If you look at the graphs though, you’ll see that in IE, MooTools really suffers. This is because IE doesn’t allow you to alter Element.prototype. As a result every time MooTools uses its selector engine in IE there is an overhead as each returned element must be extended with the new methods at that time. This is a design decision and it’s something we’re working on. Expect to see this aspect of performance improve in MooTools 1.3.

But as far as selector engines are concerned, Sizzle isn’t appreciably faster than the MooTools engine, especially when it’s used as the engine in MooTools.

I’ll finish with a portion of one of the comments I made this morning:

Our frameworks actually don’t have any other benchmarking method that I know of - selectors are the only thing. There’s no benchmark for CPU usage during animation or whatever.

But now the improvements are small. The initial speed-ups were orders of magnitude. We’re now squabbling over milliseconds. In a real world app, we’re talking about something like, maybe 50ms or so difference in performance for all the selectors running on a page. That’s what I mean by it not being a priority. Yes, maybe we could make MooTools perform 50ms faster, or a 100ms, but at this point, it’s not where we feel the biggest return on our investment lies.

After looking at the results of this test I think it’s reasonable to focus on speeding up how MooTools extends Elements, but not the selector engine. Expect to see improvements on that front in MooTools 1.3.

Some Input From The jQuery Team

Before I posted the above info I sent it off to Rey Bango. Aside from wanting to get advice from someone with a mustache, I thought it would be good to speak to someone on the jQuery team, and he fit both bills. Rey’s input to this post (which I haven’t altered other than to add this footnote) was the following:

“Our frameworks actually don’t have any other benchmarking method that I know of - selectors are the only thing. There’s no benchmark for CPU usage during animation or whatever.”

We have tests for DOM manipulation and event delgation which [are included in this set of tests as documented here].

I played with these tests which John has said “Sizzle is optimized to work best on the selectors that developers actually use - and the test represents that.” My issue with the test is that it is very repetitive - it tests against the same type of selectors repeatedly (#page, #masthead, #mastheadhd - essentially all just $ calls in MooTools - you wouldn’t use $$ for them - and .mastheadbd, .first, and .on which are all basically the same kind of selector - on a single css class name). John has created a version of SlickSpeed that repeats the types of selectors his framework is optimized for. Of course it performs faster. His argument that “MooTools optimizes for the wrong selectors” is, I think, presumptive. It would instead be accurate to say that it is not optimized for jQuery development.

The other point Rey made to me was he agrees with me that “we’re reaching the threshold of what can be achieved”. There are other things that need to be optimized, but at this point we’re trying to squeeze blood from a stone in many respects. In MooTools, we need to work on our post-processing to speed that up in IE. There are other things like DOM creation and manipulation that we need to optimize.

Included in the tests that Rey sent over were tests for DOM injection and event delegation, but I’m not convinced that the suite is real apples to apples. For instance, MooTools delegation doesn’t exist in the framework (I have a beta version in my own scripts, but that’s not official). The tests compare delegation in jQuery to MooTools standard event management which isn’t optimized for that sort of thing.

I’m sure that as such a test suite matures and is contributed to by other frameworks such as ours that we’ll see optimization race forward as we have with selectors. As it is, I’m still pretty convinced that Sizzle isn’t any faster than MooTools’ selector engine.

This entry was posted on Thursday, February 12th, 2009 at 3:25 pm and is filed under 3rd Party Libraries. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.