Diary Of An x264 Developer

10/23/2011 (12:09 pm)

The neutering of Google Code-In 2011

Filed under: development,GCI,google,x264 ::

Posting this from the Google Summer of Code Mentor Summit, at a session about Google Code-In!

Google Code-In is the most innovative open-source program I’ve ever seen.  It provided a way for students who had never done open source — or never even done programming — to get involved in open source work.   It made it easy for people who weren’t sure of their ability, who didn’t know whether they could do open source, to get involved and realize that yes, they too could do amazing work — whether code useful to millions of people, documentation to make the code useful, translations to make it accessible, and more.  Hundreds of students had a great experience, learned new things, and many stayed around in open source projects afterwards because they enjoyed it so much!

x264 benefitted greatly from Google Code-In.  Most of the high bit depth assembly code was written through GCI — literally man-weeks of work by an professional developer, done by high-schoolers who had never written assembly before!  Furthermore, we got loads of bugs fixed in ffmpeg/libav, a regression test tool, and more.  And best of all, we gained a new developer: Daniel Kang, who is now a student at MIT, an x264 and libav developer, and has gotten paid work applying the skills he learned in Google Code-In!

Some students in GCI complained about the system being “unfair”.  Task difficulties were inconsistent and there were many ways to game the system to get lots of points.  Some people complained about Daniel — he was completing a staggering number of tasks, so they must be too easy.  Yet many of the other students considered these tasks too hard.  I mean, I’m asking high school students to write hundreds of lines of complicated assembly code in one of the world’s most complicated instruction sets, and optimize it to meet extremely strict code-review standards!  Of course, there may have been valid complaints about other projects: I did hear from many students talking about gaming the system and finding the easiest, most “profitable” tasks.  Though, with the payout capped at $500, the only prize for gaming the system is a high rank on the points list.

According to people at the session, in an effort to make GCI more “fair”, Google has decided to change the system.  There are two big changes they’re making.

Firstly, Google is requiring projects to submit tasks on only two dates: the start, and the halfway point.  But in Google Code-In, we certainly had no idea at the start what types of tasks would be the most popular — or new ideas that came up over time.  Often students would come up with ideas for tasks, which we could then add!  A waterfall-style plan-everything-in-advance model does not work for real-world coding.  The halfway point addition may solve this somewhat, but this is still going to dramatically reduce the number of ideas that can be proposed as tasks.

Secondly, Google is requiring projects to submit at least 5 tasks of each category just to apply.  Quality assurance, translation, documentation, coding, outreach, training, user interface, and research.  For large projects like Gnome, this is easy: they can certainly come up with 5 for each on such a large, general project.  But often for a small, focused project, some of these are completely irrelevant.  This rules out a huge number of smaller projects that just don’t have relevant work in all these categories.  x264 may be saved here: as we work under the Videolan umbrella, we’ll likely be able to fudge enough tasks from Videolan to cover the gaps.  But for hundreds of other organizations, they are going to be out of luck.  It would make more sense to require, say, 5 out of 8 of the categories, to allow some flexibility, while still encouraging interesting non-coding tasks.

For example, what’s “user interface” for a software library with a stable API, say, a libc?  Can you make 5 tasks out of it that are actually useful?

If x264 applied on its own, could you come up with 5 real, meaningful tasks in each category for it?  It might be possible, but it’d require a lot of stretching.

How many smaller or more-focused projects do you think are going to give up and not apply because of this?

Is GCI supposed to be something for everyone, or just or Gnome, KDE, and other megaprojects?

One Response to “The neutering of Google Code-In 2011”

  1. saintdev Says:

    Sounds like they’re trying to overcompensate for what were, in the end, minor issues with last years GCI. In the process completely neutering the entire program.

Leave a Reply