There’s been a lot of churn on selector engines in the last six months or so. The jQuery guys released a stand alone library (Sizzle) which was adopted by a lot of other frameworks (it’s now a part of the Dojo Foundation) including Prototype, Dojo, jQuery (obviously) and others.

There was a lot of hullabaloo when MooTools decided not to use it. One of the arguments that Valerio made in his post about that was this:

Let’s face it: every selector engine, every part of our libraries has benefited from the others. Where they diverge is not an indicator of which framework is superior to another. Rather, they are differences in philosophy. If everyone were to use the same, shared codebase, these awesome open source contributions and general advancements will stop, and users wouldn’t be able to choose the approach which works best for them. I don’t want to see that happening.

That notion – that if all the frameworks adopted the same selector engine that both innovation will suffer – was refuted generally by the advocates of everyone using Sizzle. John Resig left a long and thoughtful comment on my post about Sizzle that included this statements:

With so many minds at the table (all the major libraries) it’ll mean that this library will be in front of the majority of JavaScript users, in one way or another. It will be compelled to become faster.

…

Frankly, not using Sizzle will mean that MooTools will always be playing a game of catch-up. … Right now jQuery, Prototype, Dojo, and YUI are all looking at using the library – that only leaves one odd library out. I’m not attempting to put undue pressure on your team – it’s absolutely your decision – but you’ll definitely be in a position, if all the libraries use Sizzle, of constantly trying to catch-up to what is implemented in the de facto implementation.

I refuted this statement and so did members of the MooTools communities (and others). John’s argument that because MooTools is the odd one out that we’ll always be playing catch up seemed especially off the mark, because it implies that Sizzle will, by virtue of having so many people using it and contributing to it, always step ahead of MooTools, and that all MooTools would hope to do is catch up – that is, implement changes to match Sizzle’s progress rather than out pace it. In reality, that’s not how technology moves forward; competing parties take turns passing each other and drive the baseline forward.

Well, today it looks like it’s Sizzle’s turn to play catch up.

Introducing Sly

MooTools contributor Harald Kirschner today released his own selector engine, Sly. This is a stand alone library (much like Sizzle) that is only 3K gzipped (8K minified without gzip). This is on par with Sizzle (which claims that gzipped is 4K). It offers the same sort of flexibility and support for custom selectors plus all the CSS3 selectors. Form the announcement article on Harald’s site.

If you’re a code geek like me, you might want skip the details and go directly to the source, the speed tests or the specs.

Sly awesomeness:

• Pure and powerful JavaScript matching algorithm for fast and accurate queries
• Extra optimizations for frequently used selectors and latest browser features
• Works uniformly in DOM documents, fragments or XML documents
• Utility methods for matching and filtering of elements
• Standalone selector parser to produce JavaScript Object representations
• Customizable pseudo-classes, attribute operators and combinators
• Just 3 kB! (minified and gzipped, 8 kB without gzip)
• No dependencies on third-party JS libraries, but developers can override internal methods (like getAttribute) for seamless integration.
• Code follows the MooTools philosophy, respecting strict standards, throwing no warnings and using meaningful variable names

But what you probably care about most is the performance:

But enough babbling: I know you just want speed and validity results. The following results were measured using a list of frequently used selectors, searching on a copy of yahoo.com, running in slickspeed.

(See also the DOM fragment speed graph) If you are interested, run the various speed tests on your own system and post your results.

The adapted version of slickspeed also supports other test cases, like querying on a DOM fragment or an XML document. They make sure that Sly and other engines work the same in all environments.

Wait, But This Isn’t MooTools?

That’s right. This is a project Harald was working on and finished and released. This isn’t the official MooTools selector engine and won’t be. MooTools has it’s own new selector engine that is in development. I won’t go into why MooTools isn’t working on a different one (I’ll leave that to its authors to discuss when they are ready to), but the basic point is valid here: having more than one of these things is good competition and it moves the base line forward. Harald’s work has fed directly into the MooTools engine and, perhaps, will also influence Sizzle, who knows.

Why It Kinda Doesn’t Matter

I’ve been saying for a while now that the speed of selector engines is just not an issue any more. The speed of the current selector engines of jQuery, MooTools, and Sly look pretty different on a graph next to each other, but they are all blazing fast. Compared to what they were like a year ago, all of them are miles ahead now and at this point we’re quibbling over milliseconds. There are other things we need to optimize and selector engines are, at this point, plenty fast for nearly all tasks. That’s not to say that there isn’t always room for improvement, but at this point it’s more about features and flexibility than it is speed, and to a great extent all these engines are extensible, flexible, and powerful.

I’m happy to see people out there still cracking this problem open to look at it from fresh perspectives and I’m sure it’s gratifying to write something that performs better than the competition. Congrats to Harald for a fine job well done.

At Cloudera (where I work now) we’ve been hard at work for the last month on the product we’re launching today. (Side note: I’m not going to link you to it because you’ll just go there and play with it and screw up all our numbers when we see all these JavaScript people show up and click the forward, back, forward, submit, add, delete, etc. buttons just to see what they do and we’ll be wondering what we’ve done wrong to make the interface so confusing…)

Anyway, today’s launch is basically a 6 step form. It’s built on Django’s form wizard which, in many respects, does all the work for you. Making the experience interactive, helpful, and even kind of fun to use is my task, and it’s made me consider the cost of adding such (arguably non-critical) features.

Let’s face it. As much as we like to talk about how JavaScript has finally kind of arrived (what with all the “Web 2.0″ commentary) and, well, it really has kind of arrived when you think about what you can do with the technology, but as much as we tout that, it’s still expensive.

Developing a simple 6 step form takes probably, oh, a day to do with a framework with Django. Less even. A few hours. If you were practiced at it and knew exactly what you wanted, maybe not even an hour. But then we start thinking about Ajax, client side form validation, tool tips, interactive components that help you construct more complex data sets that otherwise might be a pain to do by hand (like, say, typing in a date), etc. etc. And the next thing you know, there’s 90K of JavaScript on the page. Oh, and it doesn’t work quite right in Chrome or Safari 4 because those browsers aren’t in your testing suite, and then there’s IE6, which you’d rather just not support.

It seems that, just as we’ve gotten to the point where we can just bang out something as simple as a 5 step form, we’ve managed to complicate it ten times over so it still costs a ton of time and money to make it work. Is it worth it?

I’d like to think so. I like having rich interfaces that look nice. I want a web app that I build to feel like it’s going the extra mile to make the user’s life a little easier and the task in front of them a little fun. Playful even.

I’ll give you another example. I made a login box for this app. It’s a username/password box floating dead center in the page. It fetches the server response to the user’s login attempt via ajax and if the user types in the wrong credentials, it shakes at them and puts a little red “!” next to the password field. It’s not that fancy, but it makes everyone who sees it want to type in a bad password. But doing this meant a bunch of photoshop and css to make it look slick, a new extension to MooTools’ Fx that took a bit of effort, plus a bit of hacking in the Django authentication code which isn’t quite designed for this style authentication.

It’s made me think more about my programming to patterns concepts. It’s more important than ever to focus on inheritance and derived functionality. Building a base layer of functionality and abstraction makes it possible to create a system of rich interfaces. The cost up front is somewhat daunting, because you basically have to approach everything in a generic way. Sure, you’re working on the login box, but to just make the thing jiggle you have to revisit the authentication model of your app and author a new Fx plugin which involves figuring out the math for a sinusoidal transition… Ack!

But it’s worth it when you get further into it. It’s what makes Apple products so rewarding. All you have to do is look at all the phones that came before the iPhone and know that there’s value in going that extra mile. Functionality paves the roads and makes the engines work, but the details in the design make the difference between a Ferrari and a Hyundai.

The task that we have before us, I think, is to figure out how to make the system that we use to add those fine details as cost effective as Django, Ruby on Rails, and other such frameworks have made the MVC pattern. For MooTools I can see that pathway (and to be fair, we’re behind other frameworks on this score – but MooTools More is the first step on that path), but ultimately it requires a lot of people contributing who are solving very different problems to create such a system. It’s always cheaper to do something for one time use if you are never going to reuse it.

So the task I have before me, personally, is to abstract as much of my own work away from my current goals so that others can use it. And your task is to do the same. And our task as a group is to contribute our use cases, knowledge and effort into finding the commonalities between these things and make life a little better for each other. If we spent the next 2 months, 6 months, and year doing that, we’d be in such good shape. In a way that’s what we’ve been doing with MooTools this whole time (and the same can be said of all the frameworks, which borrow from each other constantly). So when I write about why you should release your code this is one of the bigger reasons. Yeah, there are a lot of shorter, tactile reasons for doing it despite the effort it takes. But the big reason is that we’re all busy reinventing the wheel. We should be doing that, but after we’ve all invented the wheel, we should compare notes, and collaborate. Then the next time we need a wheel there’s nothing to think about, and the only thing left to do is to see what cool things we’ve all done with them. Ferraris indeed.

As I outlined the other day, I have too many things on my plate right now. I’m making some (awesome) progress on MooTools More and it’s various tools and code and hope to launch the release candidate soon (soon!).

One of the things I’ve done as MooTools More has gotten most of my attention is ignore the Clientcide codebase. Part of this is because it’s changing so much – a lot of it is moving into More. This means that I have two copies of the same code, and the version in More might be very different.

Consequently I have a lot of unaddressed bugs in the Clientcide code (20 bugs, as of right now).

So here’s what I’d like to ask: would anyone out there be interested in helping me address some of these? I hate to have problems like those outlined in the bugs holding up the work of others, but I simply don’t have the bandwidth to work on them for another week or three. Over time, a lot of this code will move into MooTools More where the development team there will be able to help out with bugs, but in the mean time, it’s all on me.

If you have an interest in helping out with these bugs, please ping me (leave a comment or send me an email).