Diary Of An x264 Developer

You’re running a video chat program on a relatively weak upstream connection.  Someone else opens a video chat program on the same connection and your available bandwidth immediately drops.  What do you do?

You’re running a streaming video server that sends live video to an iPhone.  The client moves into an area of weaker reception and the stream begins to break up.  What do you do?

You’re running a streaming video server and it has currently maxed out your connection with the current viewers, but you want another person to be able to connect.   You’d rather not restart the whole server though.  What do you do?

Read More…

As seen in the previous post, a whole category of x264 improvements has now been completed and committed.  So, many have asked–what’s next?  Many major features have, from the perspective of users, seemingly come out of nowhere, so hopefully this post will give a better idea as to what’s coming next.

Big thanks to everyone who’s been helping with a lot of the changes outside of the core encoder.  This, by the way, is one of the easiest ways to get involved in x264 development; while learning how the encoder internals work is not necessarily that difficult, understanding enough to contribute useful changes often takes a lot of effort, especially since by and large the existing developers have already eliminated most of the low-hanging fruit.  By comparison, one can work on x264cli or parts of the libx264 outside the analysis sections with significantly less domain knowledge.  This doesn’t mean the work is any less difficult–only that it has a lower barrier to entry.

For most specific examples given below, I’ve put down an estimated time scale as an exercise in project estimation.  This is no guarantee as to when it will be done; just a wild guess by me.  Though it might serve as a personal motivator for the tasks that I’ve assigned to myself.  Don’t harass any of the other developers based on my bad guesses

Do also note that even though projects have time scales doesn’t even necessarily mean that they will be finished at all: not everything that we plan ends up happening.  Many features end up sitting on the TODO list for months or even years before someone decides that it’s important enough to implement.  If your company is particularly interested in one of these features, I might be able to offer you a contract to make sure it gets done much sooner and in a way that best fits your use-case; contact me via email.

Read More…

x264 has long held the crown as one of the best, if not the best, general-purpose H.264 video encoder.  With state-of-the-art psy optimizations and powerful internal algorithms, its quality and performance in “normal” situations is mostly unrivaled.

But there are many very important use-cases where this simply isn’t good enough.  All the quality and performance in the world does nothing if x264 can’t meet other requirements necessary for a given business.  Which brings us to today’s topic: low-latency streaming.

The encoding familiar to most users has effectively “infinite” latency: the output file is not needed by the user until the entire encode is completed.  This allows algorithms such as 2-pass encoding, which require that the entire input be processed before even a single frame of the final output is available.  This of course becomes infeasible for any sort of live streaming, in which the viewer must see the video some predictable amount of time after it reaches the encoder.  Which brings us to our first platform: broadcast television.

Read More…

Note: I originally posted this a day earlier, but quickly retracted it when it was pointed out that I made a rather egregious error in the ffmpeg tests’ settings, so I re-did those and added a few more codecs.

Encoder comparisons are a dime a dozen these days, but there’s one thing I’ve almost never seen tested: cartoon sources.  Animated material has totally different characteristics from film and presents a whole separate set of encoding challenges.

First, we’ll start with what makes such video easy to compress.  Animation is mostly static; backgrounds are completely static with characters placed in front of them.  The characters themselves are mostly static as well; modern animation is usually at a significantly lower framerate than the actual video.  Furthermore, characters may stand still with only their mouths moving, dramatically reducing complexity.  Finally, animation is usually very clean, without any film grain.  All of this combines to seemingly make animation compression a very simple task.

Read More…

“There is nothing like looking, if you want to find something… You certainly usually find something, if you look, but it is not always quite the something you were after.”

– J.R.R Tolkien

About a year and a half ago, I had an idea: what if we made a graph of how each block of the video referenced other blocks temporally and used this graph to increase quality on blocks which are referenced a lot and lower it on those which are referenced less?  Clearly this would greatly improve average quality… but when I thought through it, the problem became messier and messier.  I decided to put it off to later. I ended up making it a Google Summer of Code project for 2008, but that student disappeared after a few weeks of relative non-work and a hardly-working initial patch.  I mostly forgot about it; it was in the same category as explicit weighted prediction and MBAFF: messy things that might help, but I didn’t want to do.  This idea in particular got filed away under the name “MB-tree.”

Read More…