I don’t mean to dismiss your contribution to our community. Benchmarking suites are very useful in driving innovation and I’m glad to see you taking it on. That said, I didn’t spend a ton of time with your test environment and all I meant to highlight was that I, personally, don’t know how much to trust the results as I haven’t investigated the methodologies closely. For instance, the fact that you can’t run MooTools and Prototype together tells me that you aren’t putting these things in Iframes (as Slickspeed does) which means that they might in some ways poison each others’ results. That’s very concerning to me.

My issue with including Resig’s code in the example is that 1) if I remember correctly it’s also very similar to Dean’s work (which is not a bad thing) but that 2) Resig never really put it to use. It’s an unfair comparison to say that the inheritance system is faster or slower than any other when you are testing more than that. MooTools does other things when you instantiate a class including providing hooks for mutations and customizations. So you aren’t really testing inheritance methods here so much as implementations that enable inheritance as well as other things. JavaScript has more than one way to facilitate inheritance – as your tests illustrate – but you aren’t testing that. You’re testing the programming patterns in various frameworks that happen to make use of inheritance as part of that pattern. But not Resig’s code, which was mostly an academic pursuit.

Still, I find this topic immensely interesting and informative. No doubt the Prototype guys and others will learn from these findings and improve their speed which is great for their communities. I don’t mean to cast dispersions on your work or your findings, but in communicating my own thoughts, I stand my statement that they are questionable – as in, I have questions. Unfortunately, it’s not very fair that I don’t have time to find answers myself. My apologies if you found this unfounded or unfair. I hope you keep this kind of work up as it helps everyone in the long run, questions or not.

I’m a little concerned by the tone of your post however. I fail to see how the results of this exercise are questionable. Yes, Resig’s code is best treated as experimental, but I make no bones about that. This was never intended, nor presented, as a head-to-head comparison of libraries. It was simply an examination of performance for a few, notable inheritance implementations. Resig provided a useful and informative example of what a streamlined implementation by someone well-versed in writing nuanced code might do. Had I drawn conclusions on how awesome (or shitty) MooTools or other libraries were based on that then that would be questionable. But the raw numbers I preset, and the one conclusion I come to (“Prototype is slow”) seem pretty irrefutable.

What it is more concerning to me is the assertion that JSLitmus is unreliable. As the author of JSLitmus I like to think I’ve created a tool that people will find useful and valuable. I am “appropriately” concerned by the report from WebReflection that he is having issues with it, but I have so far been unable to reproduce the problem he describes. Until I have more information from him, or corroboration from at least one other person experiencing the same issue, I’d prefer not to sound the “JSLitmus sucks!” alarm. It would be really helpful to me if you could at least give the tool a try and share your first-hand experiences with your readers rather than fanning the flames. I’m hopeful that rather than implying it is unreliable, you might be a little impressed at the value it provides in it’s fairly small form-factor.

Finally, I agree that a SlickSpeed-like test for JavaScript performance would be useful. It’s why I created JSLitmus. I felt it was important for people to be able to create interesting tests, and even more important for them to be able to quickly and easily share those tests so they could, oh I dunno, verify the reports and conclusions came to. I’m pretty confident that JSLitmus is about as close to “reliable” as you can get with a pure javascript implementation for a testing tool, so it pains me to see people implying it can’t be trusted.