For years I’ve dealt with all sorts of horrific situations when dealing with open source. Like software modules written by different teams on a badly managed commercial project, different open source projects tend to defensively program around each others’ flaws rather than actually submitting patches to fix them. There are even entire projects built around providing API wrappers that simplify usage and fix bugs present in the original library.
In many cases people don’t even submit bug reports. Sometimes they outright patch each others’ libraries–and don’t submit the patches back to the original project. At best this leads to tons of bugs and security vulnerabilities being overlooked in the original project. At worst this leads to situations like the Debian OpenSSL fiasco, in which the people patching the code don’t know enough about it to safely work with it (and don’t even talk to the people who do).
But enough ranting–let me talk about a success story.
Some of you may know of the recent drama over BFS (Brain Fuck Scheduler) written by Con Kolivas. Its primary purpose was to reduce latency for ordinary desktop applications (potentially at the cost of absolute throughput). Unsurprisingly, someone soon tested x264 with BFS–and the results were absurd. BFS trashed CFS, the existing kernel scheduler, by enormous margins–up to 80%. Something was up with these results: if a scheduler A can get 80% better performance than scheduler B on a load as simple as x264, scheduler B must be seriously bugged. This theory was further bolstered by the fact that BFS is a very simple scheduler while CFS is a very complex one; one of the heuristics in CFS could be causing problems for x264.
So I tentatively submitted a test case to the Linux kernel mailing list. I didn’t know what to expect; maybe more flames carrying over from the BFS debate? Instead, I got “Thanks a bunch for the nice repeatable testcase!” This is one of the few times I’ve seen this outside of what I attempt to do with x264: a developer happy to see someone report a bug with his code and apparently eager to jump to fixing it. Though it certainly sounded good so far, but would anything result from this?
Answer: yes: up to a 70% increase in performance, committed the next day. But the kernel devs weren’t done yet: a quick grep of Linux kernel mails over the next weeks showed x264 popping up in quite a few scheduler benchmarks: they had added it as a regular test case. And just recently we got another 10% performance.
The morals of the story?
1. Talk to upstream. They know more about it than you do, full stop. Don’t blindly complain about problems with X or try to fix it yourself: talk to the people who know what they’re doing. Of course, if that fails, feel free to do it yourself: there are plenty of projects notorious for completely ignoring serious bug reports for years (e.g. GCC).
2. If you are upstream, listen to bug reports. “Patches welcome” is only a reasonable doctrine for feature requests, not for bug reports. A sufficiently good test case for producing a bug should always result in an investigation into the problem by real developers. I try to make this my doctrine at all times–if anyone reports anything weird with x264, at an absolute minimum I want to know why said weird behavior is occurring. A large number of bug fixes (and also some algorithmic changes, such as with VBV) result from user issue reports.
3. If you want x264 to run a lot faster, upgrade your kernel to tip, or at least upgrade on the next release. You’ll get an enormous benefit with 4 or more cores.