Sly, the Latest CSS Selector Engine
There’s been a lot of churn on selector engines in the last six months or so. The jQuery guys released a stand alone library (Sizzle) which was adopted by a lot of other frameworks (it’s now a part of the Dojo Foundation) including Prototype, Dojo, jQuery (obviously) and others.
There was a lot of hullabaloo when MooTools decided not to use it. One of the arguments that Valerio made in his post about that was this:
Let’s face it: every selector engine, every part of our libraries has benefited from the others. Where they diverge is not an indicator of which framework is superior to another. Rather, they are differences in philosophy. If everyone were to use the same, shared codebase, these awesome open source contributions and general advancements will stop, and users wouldn’t be able to choose the approach which works best for them. I don’t want to see that happening.
That notion – that if all the frameworks adopted the same selector engine that both innovation will suffer – was refuted generally by the advocates of everyone using Sizzle. John Resig left a long and thoughtful comment on my post about Sizzle that included this statements:
With so many minds at the table (all the major libraries) it’ll mean that this library will be in front of the majority of JavaScript users, in one way or another. It will be compelled to become faster.
…
Frankly, not using Sizzle will mean that MooTools will always be playing a game of catch-up. … Right now jQuery, Prototype, Dojo, and YUI are all looking at using the library – that only leaves one odd library out. I’m not attempting to put undue pressure on your team – it’s absolutely your decision – but you’ll definitely be in a position, if all the libraries use Sizzle, of constantly trying to catch-up to what is implemented in the de facto implementation.
I refuted this statement and so did members of the MooTools communities (and others). John’s argument that because MooTools is the odd one out that we’ll always be playing catch up seemed especially off the mark, because it implies that Sizzle will, by virtue of having so many people using it and contributing to it, always step ahead of MooTools, and that all MooTools would hope to do is catch up – that is, implement changes to match Sizzle’s progress rather than out pace it. In reality, that’s not how technology moves forward; competing parties take turns passing each other and drive the baseline forward.
Well, today it looks like it’s Sizzle’s turn to play catch up.
Introducing Sly
MooTools contributor Harald Kirschner today released his own selector engine, Sly. This is a stand alone library (much like Sizzle) that is only 3K gzipped (8K minified without gzip). This is on par with Sizzle (which claims that gzipped is 4K). It offers the same sort of flexibility and support for custom selectors plus all the CSS3 selectors. Form the announcement article on Harald’s site.
If you’re a code geek like me, you might want skip the details and go directly to the source, the speed tests or the specs.
Sly awesomeness:
- Pure and powerful JavaScript matching algorithm for fast and accurate queries
- Extra optimizations for frequently used selectors and latest browser features
- Works uniformly in
DOMdocuments, fragments orXMLdocuments- Utility methods for matching and filtering of elements
- Standalone selector parser to produce JavaScript
Objectrepresentations- Customizable pseudo-classes, attribute operators and combinators
- Just 3 kB! (minified and gzipped, 8 kB without gzip)
- No dependencies on third-party JS libraries, but developers can override internal methods (like
getAttribute) for seamless integration.- Code follows the MooTools philosophy, respecting strict standards, throwing no warnings and using meaningful variable names
But what you probably care about most is the performance:
But enough babbling: I know you just want speed and validity results. The following results were measured using a list of frequently used selectors, searching on a copy of yahoo.com, running in slickspeed.
(See also the DOM fragment speed graph) If you are interested, run the various speed tests on your own system and post your results.
The adapted version of slickspeed also supports other test cases, like querying on a DOM fragment or an XML document. They make sure that Sly and other engines work the same in all environments.
Wait, But This Isn’t MooTools?
That’s right. This is a project Harald was working on and finished and released. This isn’t the official MooTools selector engine and won’t be. MooTools has it’s own new selector engine that is in development. I won’t go into why MooTools isn’t working on a different one (I’ll leave that to its authors to discuss when they are ready to), but the basic point is valid here: having more than one of these things is good competition and it moves the base line forward. Harald’s work has fed directly into the MooTools engine and, perhaps, will also influence Sizzle, who knows.
Why It Kinda Doesn’t Matter
I’ve been saying for a while now that the speed of selector engines is just not an issue any more. The speed of the current selector engines of jQuery, MooTools, and Sly look pretty different on a graph next to each other, but they are all blazing fast. Compared to what they were like a year ago, all of them are miles ahead now and at this point we’re quibbling over milliseconds. There are other things we need to optimize and selector engines are, at this point, plenty fast for nearly all tasks. That’s not to say that there isn’t always room for improvement, but at this point it’s more about features and flexibility than it is speed, and to a great extent all these engines are extensible, flexible, and powerful.
I’m happy to see people out there still cracking this problem open to look at it from fresh perspectives and I’m sure it’s gratifying to write something that performs better than the competition. Congrats to Harald for a fine job well done.
Follow @clientcide on twitter to get notified of new posts.
To follow me personally on twitter, follow @anutron.
March 25th, 2009 at 5:55 pm
Wow, looks impressive. I’m gonna go run off to github and check out the source.
March 25th, 2009 at 9:04 pm
YASS is much faster than Sizzle (and have almost the same architecture as Sly). But it was released in 12/2008.
http://yass.webo.in/
I’ve tried to put Sly and YASS face to face but can’t setup SlickSpeed correctly. Can you describe how elements should be iterated or what are final settings in config.ini?
Raw CSS3 branch of YASS is 5434 bytes minified and only 1941 bytes gzipped. Full repository is located here
http://code.google.com/p/yeasss/
March 26th, 2009 at 4:45 am
Looks promising. And I can’t wait to see what MooTools does to their selector engine. Hopefully it will step-up the competition.
March 26th, 2009 at 4:47 am
Another nice attempt to a selectors engine, I like the syntax and the numbers are really interesting but that’s all. This is an half compliant CSS3 selector engine.
I don’t see the point of comparing Sly with other selector engines, not yet at least.
From a cursory test I did it is missing/failing the most obvious features of CSS3 selectors like “> div”, “+ div”, all the “of-type” selectors, :root, :target, “document order” is not taken in consideration in comma separated selector and do not have a minimal distinction in attributes case-sensitivity, all pseudos-class like :visited, :link, :selected return the same wrong result set.
A simple “input[type=TEXT]” is enough to bend both Sly and Slick/Mootools and similar “loose” selector engines. Without proper checks/specs I will be surprised if it wasn’t fast…
Not for production yet, still I would see how this improves in the coming months.
March 26th, 2009 at 5:13 am
@Diego Perini: Thanks for your review and that (a bit sloppy) cursory test. Here my comments:
None of the engines take document order in consideration, except querySelectorAll is used. Ordered results would be a huge impact on speed and is rarely needed. I’ll implement it as optional feature.
Also the combinators you mention are all implemented (including the according specs and several tests in the suite. Try other selector lists.), the system is even customisable including a script with additional combinators (inspired by MooTools slick). The CSS3 selectors in Sly core are simply stripped to the useful parts. Additional pseudo classes can also be added, there can be Sly.FullBlownCSS3.js file, with all the (often useless) missing selectors, but I doubt that people would enjoy the additional bytes for the sake of completeness.
"input[type=TEXT]"fails correctly (ref: w3 css3: [att=val] Represents an element with the att attribute whose value is exactly “val”.) and I don’t know a selector engine that has a case insensitive attribute selector. Using a case insensitive would be very unexpected (too “loose”).March 26th, 2009 at 6:05 am
Harald,
I reiterate my compliments for your work, hope you take the above as useful suggestions and you can improve those parts in your project.
Several libraries return elements in “document order” as of today, Sizzle, Base2 and my own NWMatcher, it is a CSS3 specification requirement and I believe others are implementing that too…the slowdown is not considerable, the outcome is really to be considered, since now we can build TOC’s out of our pages headers and extract text from pages without messing up their order.
input[type=TEXT] seems not to work on Sly, nor on Slick/Mootools while it works in querySelectorAll() and in my NWMatcher…as said these are cursory tests I haven’t reviewed everything yet. It is just an example on how libraries can skip specs.
The “DOCTYPE” should make a difference in how case-sensitivity is treated, there are substantial difference about that between HTML, XHTML and XML.
These are the relevant parts in the specs related to “case-senitivity”:
http://www.w3.org/TR/html4/types.html#h-6.1 (for HTML)
http://www.w3.org/TR/xhtml1/#h-4.11 (for XHTML)
Hope you agree at least about these specifications.
March 26th, 2009 at 6:51 am
Updates: optional ordered result via Sly.search(what, *true*) and accompanying specs: http://digitarald.de/repos/sly/Specs/index.html?rerun=Sly.search%20Fragment
More CSS3: http://github.com/digitarald/sly/blob/master/Sly.W3.js
About the specifications: Accuracy is a major factor, but adding support for rare edge cases would require a lot of good will. XML and XHTML are case sensitive, developers understand attr=value as “real” equality selector, so it will work for 99.9% of all cases.
p.s. You should probably tell that you have your own selector engine too when you write a review about another one ;-)
March 26th, 2009 at 7:10 am
“Well, today it looks like it’s Sizzle’s turn to play catch up.”
Why are new selectors always compared to Sizzle, it’s neither the fastest nor the most CSS3 compliant. Base2, Ext, DOMAssistant, even our Selector does a better job.
@Harald Kirschner:
Not considering most of the CSS3 selectors when writing the engine will make it hard to optimize their queries. Boating the code even more in the end.
March 26th, 2009 at 7:21 am
Hey…
didn’t want to hijack too much this thread and still be useful with a head up.
“case-sensitivity” is something we will have to fight with for a while…we have to do with already existing code, nowadays skilled peoples will be able to avoid these pitfalls in their new projects by knowing the problem beforehand, I hope libraries are not designed just for those peoples.
Haven’t put Sly on my “match()” benchmarking yet but will do soon. The real interesting thing we need is the “match()” method to be fast not only the “select()” method, the latter method will be phased out in favor of “match()” many times in future.
This is the part where libraries are currently failing by not implementing a bottom-up element matcher: They just implement the “select()” ($/$$/search) method which forces us to go through all elements in the page several times.
The “match()” method is the basic requirement to boost event delegation, it is not feasible doing that the other way around, I mean do a “select()” and then go through the result set to see if the “element” is in there. This is a real slowdown in event driven applications.
Will setup some benchmark and post result to show what I mean.
March 26th, 2009 at 7:46 am
@Hendrik: “Not considering most of the CSS3 selectors” is slightly overstated. Sly supports about all CSS3 selectors, only the pseudo classes in the core are subset of the most useful cases. This is definitely an advantage, since Sly (and other engines) provide also an API to implement custom combinators, attribute selectors and pseudo selectors on the fly (see Sly.W3.js, Sly.Custom.js on github).
These customizations and allowing useful non-valid queries like Sly.search(“> div”, element) are more important for me than having a perfect clone of querySelectorAll.
March 26th, 2009 at 7:48 am
@Harald Kirschner:
Diego mentioned his engine in his second response.
I know you just updated Sly, but I put together a little test page to illustrate the points Diego first made: http://dl.getdropbox.com/u/513327/JavaScript/sly/index.html
March 26th, 2009 at 7:50 am
@Harald Kirschner: having different results returned for different browsers is a pretty big issue. I don’t see returning proper results as a nicety. You can have customizations and proper matching.
March 26th, 2009 at 7:54 am
@Diego: You’ll be surprised by Sly.match, I designed it with event delegation in mind. It currently only matches against the first found selector, combinators are not taken into account. I’ll see if I get feedback how much devs would need it (then it would also support combinators in :not like “:not(div > a)”)
March 26th, 2009 at 8:09 am
@JDD: I partially agree about the sorting, I’d add an permanent option for people who don’t see ordered results as a nicety, too.
March 26th, 2009 at 8:16 am
@Harald: Allowing custom selector to be added through external files is great, as Sly does. But to get optimal performance you have to solve that, the CSS3 pseudo selectors anyway, could traverse/filter DOM nodes very differently. Some are document unique (#id, :active, :focus, etc.), some dependant on siblings (-of-type).
March 26th, 2009 at 8:47 am
@Harald,
I see you got the point about the importance of the “match()” method.
I had a test with Sly.match() in my benchmarking environment but I couldn’t make it work, a simple “ul#nav li a” doesn’t seems to work for me. I have not tried most complex :not() because from what I understand your match() only does simple selectors like “div#foo.bar[attr=val]” without considering combinators or comma separated selectors. In these test my “match()” is normally 8 to 20 times faster of compared to other implementations (1000% – 2000% faster even on Safari/Chrome).
In previous thread I heard somebody saying that speed enhancements were not possible anymore now that all these CSS3 engines are out and fine tuned for event delegation… Well, let’s hope this can change their mind.
The “document order” you added is good, you should make it the default behavior, and to avoid slowdown every query maybe just do that reordering only if the selector is a comma separated group. This will ensure cross-browser functionality of selector.
I am happy if some of these talks will turn profitable for your Sly puppy.
March 26th, 2009 at 10:34 am
I think one point trying to be made about all using the same engine was that all of this work going into Sly could have been put into a selector engine that every library uses instead.
Of course, I think one learns a lot implementing a selector engine on their own that could not be learned as easily by starting out modifying their own. It would be great if the individual methods of implementation of the different selector engines could be separated out and discussed for improvement.
It almost seems that implementing a good selector engine is a good journeyman/master javascript project that earns your way into the top tier of javascript programmers. I know that I am impressed by everyone who implements their own complete selector engine, and I consider myself a good javascript programmer.
Kudos to Harald.
March 26th, 2009 at 10:44 am
Ha! Yeah, I wouldn’t want to author one myself, that’s for sure. As to your comment about pushing work into Sizzle, that would first require that you agree with Sizzle’s design. Given that there are so many frameworks out there, I think we can agree that there are a lot of smart people who have different ideas about how to accomplish the same goal. This is a good thing; none of the frameworks would be as good as they are if we didn’t borrow ideas from each other. The same is true of any aspect of any of these libraries.
March 26th, 2009 at 11:39 am
Well, the next areas that seem open to competition are events or DOM manipulation. Where are the stand alone easily integratable libraries for these? :D
March 26th, 2009 at 12:19 pm
@Josh, I’m not looking for a standalone library for these things. MooTools makes use of it’s own design patterns to implement its functionality. Removing that dependency adds file size and headaches for its developers.
March 26th, 2009 at 1:05 pm
Josh,
for an integrable stand-alone cross-browser event handling library look at my NWEvents.
It handles/emulates “capturing/bubbling” on IE and is able to manage handlers, listeners, delegates and custom events. It is the perfect companion of my NWMatcher selector engine.
March 26th, 2009 at 2:25 pm
Could there be a less fitting place to punt your own selector and event engines then on the release article of someone else’s? How very dare you… >:P
March 26th, 2009 at 2:52 pm
[defense]In his defense, I did ask… though I should have marked it up as rhetorical and humorous.[/defense]
March 26th, 2009 at 3:11 pm
Chris,
sorry if I offended you or anybody else, the same…I was distracted by a question (maybe not pertinent, but just maybe).
March 26th, 2009 at 3:19 pm
I took a quick look at NWEvents and it looked very useful, I love to play more with bubbling and relaying events.
I gotta add that Diego tried the Sly match functionality and indeed was *surprised* by the speed (though it only supports single selectors and no combinators, but I can life with that for noe ;) )
March 26th, 2009 at 4:34 pm
Holy cra…. Harald, how did you get the selector for an id in Safari to be twice as fast as the native implementation of getElementById?
Sly(‘anId’) is twice as fast as document.getElementById(‘anId’) in Safari 3.2.1
I’m averaging .0006 for Sly and .0012 for getElement.
It’s even more impressive in Opera. .0007 vs .0047
Nice.
March 26th, 2009 at 5:15 pm
Ah, are you caching results? How do you detect node changes/updates?
March 26th, 2009 at 6:00 pm
No worries, I do not cache results, only computed match algorithms ;-). Safari simply uses querySelectorAll, try your test with that one.
March 26th, 2009 at 11:02 pm
Of course, just the muted catch biorhythms… ha, how silly of me. So… what do you mean by “computed match algorithm?”
March 27th, 2009 at 6:14 pm
Oh, I thought that you read the code, because you saw the cache property :). I’m also not sure if I yet found the best name for that “thing”. The method (algorithm) to match single found element against a single selector is created (computed) only one time and reused later on.
March 28th, 2009 at 11:32 am
I took an initial peek at the code, but thought there was caching only because it was so much faster then the native functions. getElementById was faster then getSelector or getSelectorAll in my tests. and Sly takes .0005 milliseconds to retrieve results on average when looped 10000 times, whereas the native javascript takes .015 or more, so I figured there has to be some kind of caching going on. I’m looking forward to reading your next blog post on how you got things to go so fast.
March 30th, 2009 at 9:40 am
Ha, what a rube! That would be me. I was testing Sly(selector) against the other engines and native code. I wondered why everything was only taking .00053 milliseconds. Retesting everything with Sly.search(selector) now.
April 3rd, 2009 at 2:15 am
Why don’t MooTools use Sizzle? Because it would then be one step closer to becoming something it’s not. If everyone uses Sizzle, then it’ll never have to play catch-up with a better future selector engine!