Diary Of An x264 Developer

10/23/2011 (12:09 pm)

The neutering of Google Code-In 2011

Filed under: development,GCI,google,x264 ::

Posting this from the Google Summer of Code Mentor Summit, at a session about Google Code-In!

Google Code-In is the most innovative open-source program I’ve ever seen.  It provided a way for students who had never done open source — or never even done programming — to get involved in open source work.   It made it easy for people who weren’t sure of their ability, who didn’t know whether they could do open source, to get involved and realize that yes, they too could do amazing work — whether code useful to millions of people, documentation to make the code useful, translations to make it accessible, and more.  Hundreds of students had a great experience, learned new things, and many stayed around in open source projects afterwards because they enjoyed it so much!

x264 benefitted greatly from Google Code-In.  Most of the high bit depth assembly code was written through GCI — literally man-weeks of work by an professional developer, done by high-schoolers who had never written assembly before!  Furthermore, we got loads of bugs fixed in ffmpeg/libav, a regression test tool, and more.  And best of all, we gained a new developer: Daniel Kang, who is now a student at MIT, an x264 and libav developer, and has gotten paid work applying the skills he learned in Google Code-In!

Some students in GCI complained about the system being “unfair”.  Task difficulties were inconsistent and there were many ways to game the system to get lots of points.  Some people complained about Daniel — he was completing a staggering number of tasks, so they must be too easy.  Yet many of the other students considered these tasks too hard.  I mean, I’m asking high school students to write hundreds of lines of complicated assembly code in one of the world’s most complicated instruction sets, and optimize it to meet extremely strict code-review standards!  Of course, there may have been valid complaints about other projects: I did hear from many students talking about gaming the system and finding the easiest, most “profitable” tasks.  Though, with the payout capped at $500, the only prize for gaming the system is a high rank on the points list.

According to people at the session, in an effort to make GCI more “fair”, Google has decided to change the system.  There are two big changes they’re making.

Firstly, Google is requiring projects to submit tasks on only two dates: the start, and the halfway point.  But in Google Code-In, we certainly had no idea at the start what types of tasks would be the most popular — or new ideas that came up over time.  Often students would come up with ideas for tasks, which we could then add!  A waterfall-style plan-everything-in-advance model does not work for real-world coding.  The halfway point addition may solve this somewhat, but this is still going to dramatically reduce the number of ideas that can be proposed as tasks.

Secondly, Google is requiring projects to submit at least 5 tasks of each category just to apply.  Quality assurance, translation, documentation, coding, outreach, training, user interface, and research.  For large projects like Gnome, this is easy: they can certainly come up with 5 for each on such a large, general project.  But often for a small, focused project, some of these are completely irrelevant.  This rules out a huge number of smaller projects that just don’t have relevant work in all these categories.  x264 may be saved here: as we work under the Videolan umbrella, we’ll likely be able to fudge enough tasks from Videolan to cover the gaps.  But for hundreds of other organizations, they are going to be out of luck.  It would make more sense to require, say, 5 out of 8 of the categories, to allow some flexibility, while still encouraging interesting non-coding tasks.

For example, what’s “user interface” for a software library with a stable API, say, a libc?  Can you make 5 tasks out of it that are actually useful?

If x264 applied on its own, could you come up with 5 real, meaningful tasks in each category for it?  It might be possible, but it’d require a lot of stretching.

How many smaller or more-focused projects do you think are going to give up and not apply because of this?

Is GCI supposed to be something for everyone, or just or Gnome, KDE, and other megaprojects?

04/02/2011 (11:33 pm)

You should apply for x264 Google Summer of Code

Filed under: development,GSOC,x264 ::

Want to do some fun open source work and get paid?  You should apply for GSOC.  Check out our ideas page and the official Google page.

(And yes, I’ll get around to approving the queued comments and writing more real posts.  Eventually!  I promise!)

12/05/2010 (3:35 pm)

Direct from the Blu-ray disc

Filed under: blu-ray,x264 ::

A MediaInfo from the Warner Brothers’ Blu-ray “The Town“:

General
Complete name : 00020.m2ts
Format : BDAV
Format/Info : Advanced Video Codec
File size : 528 KiB
Duration : 900ms
Overall bit rate : 4 745 Kbps
Maximum Overall bit rate : 15.0 Mbps
Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.0
Format settings, CABAC : Yes
Format settings, ReFrames : 3 frames
Codec ID : 27
Duration : 1s 1ms
Bit rate mode : Variable
Bit rate : 5 000 Kbps
Maximum bit rate : 24.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.101
Stream size : 611 KiB
Writing library : x264 core 104 r1683 62997d6

(Yes, it’s just a menu. But good things start small!)

11/25/2010 (8:35 pm)

Announcing TMPGEnc 4: now with x264!

Filed under: commercial,japan,licensing,x264 ::

A few months ago, we announced a commercial licensing program so that even companies unable to use GPL software in their products have a chance to use the open source x264 instead of proprietary alternatives.  The system worked on two basic concepts.  First, all licensees would still be required to give their changes to x264 back to us: x264 must forever remain free, with no useful contributions kept hidden from the community.  Second, all the profits would go directly back to x264, primarily to the developers who’ve made the most significant contributions to x264 over the years, but also to funding future development, bounties for new features, as well as contributing to other related projects (e.g. Videolan and ffmpeg).

Over the past couple of months, we’ve gotten an enormous response; over 40 companies have inquired about licensing, with more contacting us every day.  Due to the sheer volume of interest, we’ve partnered with CoreCodec, the creators of the free Matroska container format and developers of CoreAVC, to make x264 as widely available as possible in the world of commercial software as it is in the world of open source.  All of this is already filtering back to benefiting x264 users, with many bugs being reported by commercial licensees as well as some code contributed.

Today, we announce the first commercial consumer encoding software to switch to x264: Pegasys Inc.’s TMPGEnc.  Expect many more to follow: with x264 now available commercially as well as freely, there are few excuses left to use any other H.264 encoder.  Vendors of overpriced, underpowered proprietary competitors should begin looking for new jobs.

(Pegasys press release: English, Japanese)

11/25/2010 (1:30 pm)

Patent skullduggery: Tandberg rips off x264 algorithm

Filed under: patents,ripoffs,x264 ::

Update: Tandberg claims they came up with the algorithm independently: to be fair, I can actually believe this to some extent, as I think the algorithm is way too obvious to be patented.  Of course, they also claim that the algorithm isn’t actually identical, since they don’t want to lose their patent application.

I still don’t trust them, but it’s possible it’s merely bad research (and thus being unaware of prior art) as opposed to anything malicious.  Furthermore, word from within their office suggests they’re quite possibly being honest: supposedly the development team does not read x264 code at all.  So this might just all be very bad luck.

Regardless, the patent is still complete tripe, and should never have been filed.

Most importantly, stop harassing the guy whose name is on the patent (Lars): he’s just a programmer, not the management or lawyers responsible for filing the patent.  This is stupid and unnecessary.  I’ve removed the original post because of this; it can be found here for those who want to read it.

Read More…

10/17/2010 (6:50 pm)

How to contribute to open source, for companies

Filed under: development,open source,x264 ::

I have seen many nigh-incomprehensible attempts by companies to contribute to open source projects, including x264.  Developers are often simply boggled, wondering why the companies seem incapable of proper communication.  The companies assume the developers are being unreceptive, while the developers assume the companies are being incompetent, idiotic, or malicious.  Most of this seems to boil down to a basic lack of understanding of how open source works, resulting in a wide variety of misunderstandings.  Accordingly, this post will cover the dos and don’ts of corporate contribution to open source.

Read More…

05/25/2010 (11:01 pm)

Anatomy of an optimization: H.264 deblocking

Filed under: assembly,development,H.264,speed,x264 ::

As mentioned in the previous post, H.264 has an adaptive deblocking filter.  But what exactly does that mean — and more importantly, what does it mean for performance?  And how can we make it as fast as possible?  In this post I’ll try to answer these questions, particularly in relation to my recent deblocking optimizations in x264.

H.264′s deblocking filter has two steps: strength calculation and the actual filter.  The first step calculates the parameters for the second step.  The filter runs on all the edges in each macroblock.  That’s 4 vertical edges of length 16 pixels and 4 horizontal edges of length 16 pixels.  The vertical edges are filtered first, from left to right, then the horizontal edges, from top to bottom (order matters!).  The leftmost edge is the one between the current macroblock and the left macroblock, while the topmost edge is the one between the current macroblock and the top macroblock.

Here’s the formula for the strength calculation in progressive mode. The highest strength that applies is always selected.

If we’re on the edge between an intra macroblock and any other macroblock: Strength 4
If we’re on an internal edge of an intra macroblock: Strength 3
If either side of a 4-pixel-long edge has residual data: Strength 2
If the motion vectors on opposite sides of a 4-pixel-long edge are at least a pixel apart (in either x or y direction) or the reference frames aren’t the same: Strength 1
Otherwise: Strength 0 (no deblocking)

These values are then thrown into a lookup table depending on the quantizer: higher quantizers have stronger deblocking.  Then the actual filter is run with the appropriate parameters.  Note that Strength 4 is actually a special deblocking mode that performs a much stronger filter and affects more pixels.

Read More…

04/25/2010 (11:01 am)

Announcing the first free software Blu-ray encoder

Filed under: blu-ray,x264 ::

For many years it has been possible to make your own DVDs with free software tools.  Over the course of the past decade, DVD creation evolved from the exclusive domain of the media publishing companies to something basically anyone could do on their home computer.

But Blu-ray has yet to get that treatment.  Despite the “format war” between Blu-ray and HD DVD ending over two years ago, free software has lagged behind.  “Professional” tools for Blu-ray video encoding can cost as much as $100,000 and are often utter garbage.  Here are two actual screenshots from real Blu-rays: I wish I was making this up.

But today, things change.  Today we take the first step towards a free software Blu-ray creation toolkit.

Thanks to tireless work by Kieran Kunyha, Alex Giladi, Lamont Alston, and the Doom9 crowd, x264 can now produce Blu-ray-compliant video.  Extra special thanks to The Criterion Collection for sponsoring the final compliance test to confirm x264′s Blu-ray compliance.

With x264′s powerful compression, as demonstrated by the incredibly popular BD-Rebuilder Blu-ray backup software, it’s quite possible to author Blu-ray disks on DVD9s (dual-layer DVDs) or even DVD5s (single-layer DVDs) with a reasonable level of quality.  With a free software encoder and less need for an expensive Blu-ray burner, we are one step closer to putting HD optical media creation in the hands of the everyday user.

To celebrate this achievement, we are making available for download a demo Blu-ray encoded with x264, containing entirely free content!

Read More…

03/18/2010 (10:29 pm)

Announcing x264 Summer of Code 2010!

Filed under: development,google,GSOC,x264 ::

With the announcement of Google Summer of Code 2010 and the acceptance of our umbrella organization, Videolan, we are proud to announce the third x264 Summer of Code!  After two years of progressively increasing success, we expect this year to be better than ever.  Last year’s successes include ARM support and weighted P-frame prediction.  This year we have a wide variety of projects of varying difficulty, including some old ones and a host of new tasks.  The qualification tasks are tough, so if you want to get involved, the sooner the better!

Interested in getting started?  Check out the wiki page, hop on #x264 on Freenode IRC, and say hi to the gang!  No prior experience or knowledge in video compression necessary: just dedication and the willingness to ask questions and experiment until you figure things out.

02/22/2010 (3:05 pm)

Flash, Google, VP8, and the future of internet video

Filed under: google,H.264,HTML5,Theora,VP8,x264 ::

This is going to be a much longer post than usual, as it’s going to cover a lot of ground.

The internet has been filled for quite some time with an enormous number of blog posts complaining about how Flash sucks–so much that it’s sounding as if the entire internet is crying wolf.  But, of course, despite the incessant complaining, they’re right: Flash has terrible performance on anything other than Windows x86 and Adobe doesn’t seem to care at all.  But rather than repeat this ad nauseum, let’s be a bit more intellectual and try to figure out what happened.

Flash became popular because of its power and flexibility.  At the time it was the only option for animated vector graphics and interactive content (stuff like VRML hardly counts).  Furthermore, before Flash, the primary video options were Windows Media, Real, and Quicktime: all of which were proprietary, had no free software encoders or decoders, and (except for Windows Media) required the user to install a clunky external application, not merely a plugin.  Given all this, it’s clear why Flash won: it supported open multimedia formats like H.263 and MP3, used an ultra-simple container format that anyone could write (FLV), and worked far more easily and reliably than any alternative.

Thus, Adobe (actually, at the time, Macromedia) got their 98% install base.  And with that, they began to become complacent.  Any suggestion of a competitor was immediately shrugged off; how could anyone possibly compete with Adobe, given their install base?  It’d be insane, nobody would be able to do it.  They committed the cardinal sin of software development: believing that a competitor being better is excusable.  At x264, if we find a competitor that does something better, we immediately look into trying to put ourselves back on top.  This is why x264 is the best video encoder in the world.  But at Adobe, this attitude clearly faded after they became the monopoly.  This is the true danger of monopolies: they stymie development because the monpolist has no incentive to improve their product.

In short, they drank their own Kool-aid.  But they were wrong about a few critical points.

Read More…

Next Page »