On Github’s Pull Requests and Other Things
I love github. At the risk of sounding trite, I’ll say that they are the “2.0″ of open source project hosting. Sourceforge, Google Code, etc, have played a big role in open source development, but their tools pale in comparison to Git itself much less the beauty that is Github. Github is fun to use, nice to look at, and has served the MooTools well since we moved it there.
One of the thing that’s so encouraging about it is the pace of the development. New features seem to crop up every couple of weeks and while they may not be perfect, they’re almost always a good start. Take Github Issues for instance. It leaves a lot to be desired, but if you have a small project, it gets the job done. MooTool uses Lighthouse and Apache uses JIRA and Github’s Issues can’t yet compete with these tools. It works fine until you have > 100 bugs and > 30 people working on the project. But I thoroughly believe it’ll catch up eventually because the people at Github have proven that they’re talented and dedicated to quality. A lot of their features are in the beta-ish phase, it seems, and given the quality of those features at this early stage, I’m mostly content.
So when they recently released their new “Pull Requests 2.0” I was a bit excited and a bit wary. I’ve been trying to get the MooTools dev team to use Reviewboard for a couple of months now and not getting any traction for various reasons. One is that we’ve been trying to consolidate our community around Github (our Lighthouse account pre-dates Github’s Issues feature). With the revamped Pull Request feature I felt like I could finally drop that argument because it looked like everything we needed.
Like their other features, it works pretty damn well, but is missing a few key ingredients that I find myself yearning for from other tools that its meant to replace. I still have faith that the Github team will eventually get it polished, but of all the features they’ve rolled out, this one – code review – is the one I think they need to get all the way right. Code review, and easy contributions, are the key to open source. If it’s hard to review someone’s pull request, you’re either not going to pull it, or pull it without doing a good review. So I give this feedback with as much love as I can; this is literally meant as constructive criticism. Here’s what I think they need to do to make it perfect:
- Inline Comments: When you get a pull request you can comment in the “review” view, but not on a specific line. To comment on a line, you must find the commit that has that line change and comment there. For individuals who send a pull request with 10 or 20 commits, this is impossible. You *must* be able to make a comment on a specific line of code in the pull request’s review view.
- Pagination and Deep Linking: Currently, Github loads request views all at once and switch between them w/ JavaScript. This makes very large reviews impossible (it actually limits the view to a finite number of commits – 200 I think?) and very slow to load. Further, there’s no pagination so you have to scroll and scroll and scroll. This isn’t so bad, but it also makes linking to a specific line in the review hard, because the code view is hidden when you load the page, making any #anchor in that view unavailable to the browser. This ties into the comments issue, because I need to be able to link to a page and have it jump to the comment in question (as Github does when you comment on a commit) to allow people to reply to it.
Thats it. Those two things. They aren’t small things, but they aren’t big things. I’ve been leaving comments on their various blog posts around code review for a while now, and they always reply that they are aware of these things and will get to it. I have complete faith that they will. Their work to date has inspired great confidence. Of all the things on their long laundry list, I hope that these two changes are very near the top.
Follow @clientcide on twitter to get notified of new posts.
To follow me personally on twitter, follow @anutron.
September 21st, 2010 at 7:49 am
Hey, thanks for writing up your experience.
We’re definitely planning to add the ability to comment on the pull request diff directly. It’s tricky because you can push additional commits to the pull request branch, so we have to account for the diff moving around.
As for performance, I’d be interested to see the requests that were slow loading for you and in what browsers. The full response comes off the server in under 100ms on average — even for requests with large diffs / lots of commits — but, I’m interested in understanding client-side performance a bit better. We’re doing quite a lot in JS and it’s fairly CPU intensive for large requests. We’ve had a few other reports of slowness.
Lastly, I’ve fixed the issue with linking to files and diff lines. I’m not sure how that bug made it this far. Give it a try and let me know if it’s still a problem for you.
Thanks again!
September 21st, 2010 at 8:31 am
Hey Ryan,
I’ll reiterate how much I effing love Github. You guys are killing it.
There’s one other thing that I kind of left out of my previous comments and that’s diff comparisons. In Reviewboard, if someone updates the diff, I can view what changed between diff1 and diff2. I realize you guys are just diffing against the branch/hash, but consider the following:
1) I file a pull request
2) You leave some comments
3) I commit new changes based on your comments
4) The only way to tell you that there’s new stuff is if I comment (that’s ok I suppose)
5) You revisit to review my changes, but the compare view doesn’t give me a way to see them. Worse, if I used squash or –amend to compact my commits (which I should) you can’t see what changed. I think you should consider storing the diff states perhaps at each comment or something.
Anyway, as for large commits, the issue is when you have a big library update. Go compare MooTools 1.1 to 1.2 or 1.2 to 1.3. These are somewhat crazy things to do because the library changes dramatically and so they aren’t very practical. But sometimes you need that. Your diff view doesn’t paginate, so if I throw a project that has, say, an entire copy of jQuery and a few dozen plugins in it at your layout, it’s just going to crawl to render it. You’d be better to paginate (w/o ajax).
I understand that comments are tricky to do the way you’re implementing them, but I think you need to consider that reviewing a group of commits is a very different act than reviewing someone’s commits as they push them to their fork. When someone pushes a request out, that request is itself an entity (you have comments on those objects, certainly). Seeing the inline comments from before are nice, but maybe these are displayed differently (or even perhaps not at all). In theory, by the time it’s gotten to the pull request the inline comments have been addressed. Maybe just an icon that there ARE comments on that line on the commit would be enough. Either way, if there aren’t inline comments on the pull requests, it makes it very difficult to accept these things unless they are perfect.
Like I said though, I have faith in Github. I know it’ll work out.