Diary Of An x264 Developer

Want to do some fun open source work and get paid?  You should apply for GSOC.  Check out our ideas page and the official Google page.

(And yes, I’ll get around to approving the queued comments and writing more real posts.  Eventually!  I promise!)

A MediaInfo from the Warner Brothers’ Blu-ray “The Town“:

(Yes, it’s just a menu. But good things start small!)

A few months ago, we announced a commercial licensing program so that even companies unable to use GPL software in their products have a chance to use the open source x264 instead of proprietary alternatives.  The system worked on two basic concepts.  First, all licensees would still be required to give their changes to x264 back to us: x264 must forever remain free, with no useful contributions kept hidden from the community.  Second, all the profits would go directly back to x264, primarily to the developers who’ve made the most significant contributions to x264 over the years, but also to funding future development, bounties for new features, as well as contributing to other related projects (e.g. Videolan and ffmpeg).

Over the past couple of months, we’ve gotten an enormous response; over 40 companies have inquired about licensing, with more contacting us every day.  Due to the sheer volume of interest, we’ve partnered with CoreCodec, the creators of the free Matroska container format and developers of CoreAVC, to make x264 as widely available as possible in the world of commercial software as it is in the world of open source.  All of this is already filtering back to benefiting x264 users, with many bugs being reported by commercial licensees as well as some code contributed.

Today, we announce the first commercial consumer encoding software to switch to x264: Pegasys Inc.’s TMPGEnc.  Expect many more to follow: with x264 now available commercially as well as freely, there are few excuses left to use any other H.264 encoder.  Vendors of overpriced, underpowered proprietary competitors should begin looking for new jobs.

(Pegasys press release: English, Japanese)

Update: Tandberg claims they came up with the algorithm independently: to be fair, I can actually believe this to some extent, as I think the algorithm is way too obvious to be patented.  Of course, they also claim that the algorithm isn’t actually identical, since they don’t want to lose their patent application.

I still don’t trust them, but it’s possible it’s merely bad research (and thus being unaware of prior art) as opposed to anything malicious.  Furthermore, word from within their office suggests they’re quite possibly being honest: supposedly the development team does not read x264 code at all.  So this might just all be very bad luck.

Regardless, the patent is still complete tripe, and should never have been filed.

Most importantly, stop harassing the guy whose name is on the patent (Lars): he’s just a programmer, not the management or lawyers responsible for filing the patent.  This is stupid and unnecessary.  I’ve removed the original post because of this; it can be found here for those who want to read it.

Read More…

I have seen many nigh-incomprehensible attempts by companies to contribute to open source projects, including x264.  Developers are often simply boggled, wondering why the companies seem incapable of proper communication.  The companies assume the developers are being unreceptive, while the developers assume the companies are being incompetent, idiotic, or malicious.  Most of this seems to boil down to a basic lack of understanding of how open source works, resulting in a wide variety of misunderstandings.  Accordingly, this post will cover the dos and don’ts of corporate contribution to open source.

Read More…

As mentioned in the previous post, H.264 has an adaptive deblocking filter.  But what exactly does that mean — and more importantly, what does it mean for performance?  And how can we make it as fast as possible?  In this post I’ll try to answer these questions, particularly in relation to my recent deblocking optimizations in x264.

H.264′s deblocking filter has two steps: strength calculation and the actual filter.  The first step calculates the parameters for the second step.  The filter runs on all the edges in each macroblock.  That’s 4 vertical edges of length 16 pixels and 4 horizontal edges of length 16 pixels.  The vertical edges are filtered first, from left to right, then the horizontal edges, from top to bottom (order matters!).  The leftmost edge is the one between the current macroblock and the left macroblock, while the topmost edge is the one between the current macroblock and the top macroblock.

Here’s the formula for the strength calculation in progressive mode. The highest strength that applies is always selected.

If we’re on the edge between an intra macroblock and any other macroblock: Strength 4
If we’re on an internal edge of an intra macroblock: Strength 3
If either side of a 4-pixel-long edge has residual data: Strength 2
If the motion vectors on opposite sides of a 4-pixel-long edge are at least a pixel apart (in either x or y direction) or the reference frames aren’t the same: Strength 1
Otherwise: Strength 0 (no deblocking)

These values are then thrown into a lookup table depending on the quantizer: higher quantizers have stronger deblocking.  Then the actual filter is run with the appropriate parameters.  Note that Strength 4 is actually a special deblocking mode that performs a much stronger filter and affects more pixels.

Read More…

For many years it has been possible to make your own DVDs with free software tools.  Over the course of the past decade, DVD creation evolved from the exclusive domain of the media publishing companies to something basically anyone could do on their home computer.

But Blu-ray has yet to get that treatment.  Despite the “format war” between Blu-ray and HD DVD ending over two years ago, free software has lagged behind.  “Professional” tools for Blu-ray video encoding can cost as much as $100,000 and are often utter garbage.  Here are two actual screenshots from real Blu-rays: I wish I was making this up.

But today, things change.  Today we take the first step towards a free software Blu-ray creation toolkit.

Thanks to tireless work by Kieran Kunyha, Alex Giladi, Lamont Alston, and the Doom9 crowd, x264 can now produce Blu-ray-compliant video.  Extra special thanks to The Criterion Collection for sponsoring the final compliance test to confirm x264′s Blu-ray compliance.

With x264′s powerful compression, as demonstrated by the incredibly popular BD-Rebuilder Blu-ray backup software, it’s quite possible to author Blu-ray disks on DVD9s (dual-layer DVDs) or even DVD5s (single-layer DVDs) with a reasonable level of quality.  With a free software encoder and less need for an expensive Blu-ray burner, we are one step closer to putting HD optical media creation in the hands of the everyday user.

To celebrate this achievement, we are making available for download a demo Blu-ray encoded with x264, containing entirely free content!

Read More…

With the announcement of Google Summer of Code 2010 and the acceptance of our umbrella organization, Videolan, we are proud to announce the third x264 Summer of Code!  After two years of progressively increasing success, we expect this year to be better than ever.  Last year’s successes include ARM support and weighted P-frame prediction.  This year we have a wide variety of projects of varying difficulty, including some old ones and a host of new tasks.  The qualification tasks are tough, so if you want to get involved, the sooner the better!

Interested in getting started?  Check out the wiki page, hop on #x264 on Freenode IRC, and say hi to the gang!  No prior experience or knowledge in video compression necessary: just dedication and the willingness to ask questions and experiment until you figure things out.

This is going to be a much longer post than usual, as it’s going to cover a lot of ground.

The internet has been filled for quite some time with an enormous number of blog posts complaining about how Flash sucks–so much that it’s sounding as if the entire internet is crying wolf.  But, of course, despite the incessant complaining, they’re right: Flash has terrible performance on anything other than Windows x86 and Adobe doesn’t seem to care at all.  But rather than repeat this ad nauseum, let’s be a bit more intellectual and try to figure out what happened.

Flash became popular because of its power and flexibility.  At the time it was the only option for animated vector graphics and interactive content (stuff like VRML hardly counts).  Furthermore, before Flash, the primary video options were Windows Media, Real, and Quicktime: all of which were proprietary, had no free software encoders or decoders, and (except for Windows Media) required the user to install a clunky external application, not merely a plugin.  Given all this, it’s clear why Flash won: it supported open multimedia formats like H.263 and MP3, used an ultra-simple container format that anyone could write (FLV), and worked far more easily and reliably than any alternative.

Thus, Adobe (actually, at the time, Macromedia) got their 98% install base.  And with that, they began to become complacent.  Any suggestion of a competitor was immediately shrugged off; how could anyone possibly compete with Adobe, given their install base?  It’d be insane, nobody would be able to do it.  They committed the cardinal sin of software development: believing that a competitor being better is excusable.  At x264, if we find a competitor that does something better, we immediately look into trying to put ourselves back on top.  This is why x264 is the best video encoder in the world.  But at Adobe, this attitude clearly faded after they became the monopoly.  This is the true danger of monopolies: they stymie development because the monpolist has no incentive to improve their product.

In short, they drank their own Kool-aid.  But they were wrong about a few critical points.

Read More…

You’re running a video chat program on a relatively weak upstream connection.  Someone else opens a video chat program on the same connection and your available bandwidth immediately drops.  What do you do?

You’re running a streaming video server that sends live video to an iPhone.  The client moves into an area of weaker reception and the stream begins to break up.  What do you do?

You’re running a streaming video server and it has currently maxed out your connection with the current viewers, but you want another person to be able to connect.   You’d rather not restart the whole server though.  What do you do?

Read More…