Diary Of An x264 Developer

11/15/2009 (5:15 am)

The spec-violation hall of shame

People generally make the assumption that when they make an MP3 file, an MP3 player will play it.  When they save a PNG file, a browser will display it.  And when they save a PDF, any PDF application will view it.

And so we assume that H.264 decoders abide by the specification.  If they claim to support Main Profile, they’ll play back Main Profile.  There’s a whole set of rather exhaustive official test cases to validate compliance, which would hopefully ensure that all decoders would work with even the most bizarre combinations of features that fall under the subset of what they claim to support.

Among other things, this makes development dramatically easier: imagine how ridiculous it would be to have to test every change on dozens upon dozens of decoders to make sure that they all still worked.  Instead, we can just test against the official reference to ensure that our streams are still valid.

A week ago Dylan finished up the weighted P-frame prediction Google Summer of Code project.  It passed the JM verification test and it worked with libavcodec too.  We also tested a few other decoders to make sure; all were fine.

And then we committed it.

We immediately got reports of artifacting in a wide variety of decoders, despite all of them claiming to support the feature.  As such, here is a hall of shame of products that are distributed with broken decoders that do not abide by the spec.

Hopefully by making the world’s most popular H.264 encoder generate such problematic bitstreams by default, we will force the makers of these decoders to fix their software.  I’ll update the list as the decoders are fixed (already contacted DivX [Mainconcept owners] and Adobe).

NB: this is not a new feature; it’s been around for 6 years without any relevant change in the specification, so nobody has any excuse.

Adobe Flash Player [fixed in 10.1, thanks Adobe!]

Apple TV

Archos players (being worked on as we speak)

CoreCodec CoreAVC [fixed in 2.0]

MediaTek chipsets, in Oppo and some LG and Phillips Blu-ray players [reported to Oppo, fixed on March 29th]

Want to see if your decoder is broken?  Download this sample and watch it, especially frames 310-330.  If you notice obvious artifacting, compare it to the output of a working decoder (e.g. libavcodec) for reference.  If you get issues and your decoder isn’t on the list, post your results in the comments.

Edit/Note: It seems that earlier we were testing with a stream that was actually invalid; the reference decoder didn’t complain, but after putting it through a stream validator, it found an obvious issue (which we fixed in x264 r1342).  As such, the list is a lot shorter now.

21 Responses to “The spec-violation hall of shame”

  1. Mathias Says:

    Corecodec will not fix the issue, they will make you buy the next version. (Its like saying you can fix the decoding issues on ATI cards with buying a NVIDIA card.)

  2. Alex Says:

    In AAC land there are a variety of features that don’t work on decoders that claim to support MPEG-4 AAC-LC/the AAC Profile. Features include the channel coupling element, DRC, the 960 frame length, and some don’t even support PNS.

  3. JB Says:

    Your examples are a bit funny – LAME has a –strictly-enforce-iso option to solve problems with some MP3 decoders, and also had to change some behavior to please FhG. MS had some bugs on their PNG decoder well into the XP era, many deflate decoders fail with certain corner cases in the huffman trees – and good luck trying to decode a 16-bit PNG on all exotic platforms. Ans PDF… well, it has eight versions and it supports embedded video, Flash and JavaScript, so you can certainly make it so only Acrobat 9 will display it.

    But I digress.

    I see “Adobe Flash Player” in the list of broken decoders. I recall seeing it work before with this feature, any idea of what exactly fails?

  4. Dark Shikari Says:

    @JB

    Indeed, it’s true that they’re not the best examples–though in this case, we’re not changing the profile to add obscure features, so it isn’t like 16-bit PNG or the sort.

    You can probably divide the failures into a few categories: the Apple TV completely doesn’t support explicit weighted prediction at all, probably due to Apple simply not caring. Flash seems to have a subtle bug in reference frame handling–it makes assumptions that don’t always hold. CoreAVC was explicitly designed to not handle this case, in the hopes that nobody would ever use it… ;)

    Flash seems to have some minor artifacting issues in a few cases, as does Mainconcept’s native decoder (which is what Flash uses). I’ve reported the issue to contacts at both Adobe and DivX (owners of Mainconcept).

  5. Multimedia Mike Says:

    Good news from the Flash Player side: This problem is fixed in the Flash Player 10.1 beta which is available for download now:

    http://labs.adobe.com/downloads/flashplayer10.html

  6. Guillaume Poirier Says:

    Hey Mike!
    How come there’s no AMD64 version of 10.1 version? Too bad!

  7. Shevach Riabtsev Says:

    I play-back the test stream on ZR39 chip (ZORAN)and no visual artefacts observed.

  8. mitocho Says:

    How the hell do you know so much about coding? Your profile page says you’re a 3rd year student at HMU. I’m hoping that was written like 6 years ago. I’m about to graduate from the University of Southern California with a degree in Comp. Eng. / Comp. Sci. , and I feel woefully inadequate.

    On a side note, if you know any books that would help get me up to snuff on this sort of stuff, I’d really appreciate it.

    Sorry for highjacking your post…. I just figured you’d be more likely to respond to you most recent post.

  9. Dark Shikari Says:

    @mitocho

    No, it’s up-to-date. I’m currently working my way through mountains of final papers.

    You don’t need to respond to the latest post; te comments interface internal to the blog sorts by date, not by post.

    This page has a link to a mildly useful textbook on video compression technology.

    Also, I’m certainly not much of a guru at coding in general; I only really know C and assembly well, and Java/C++/Python/Scheme/everything-else-they-forced-us-to-learn-in-class relatively minimally.

  10. illusion Says:

    WD TV HD gen.1 (the one without a network port) seems to have the problem.

  11. Ganesh Says:

    Jason,

    On the subject of spec violation by decoders, I would like your comments on the decoding ability of some Sigma chipsets (those used in the WDTV / WDTV Live / Popcorn Hour media streamers) with respect to the two sample streams here:

    Panasonic Demo – Summer in Alaska
    Panasonic Demo – Nature

    [

    Both can be downloaded from this page :

    http://www.demo-world.eu/trailers/high-definition-trailers.php

    ]

    I personally tried both of them on my WDTV Live, and there seems to be quite a number of dropped frames / stuttering in general, and this is accentuated in some particular places (Nature clip : 1:46 – 2:00 ; Summer in Alaska : around 2:05)

    These 2 clips are Blu-Ray spec compliant. I didn’t find anything amiss in the Mediainfo reports. I definitely know it is not a frame rate issue since other 29.97 fps files play fine on the same player.

    Do you have an idea of what might be going wrong with the deocding process of these files?

    If you are interested in exploring further, I can try to make available a direct link to at least one of the files in a email to you.

    Thanks!

    PS: I was made aware of the issue with these 2 files from a post on AVS Forums.

  12. Dark Shikari Says:

    @Ganesh

    Not sure. I recall that Sigma chips tend to be absurdly picky: we had a case at Avail Media where one of them failed because a motion vector was a few pixels longer than the 255.75 max allowed for interlaced footage. A lot of them may only do Level 4.0, not Level 4.1.

  13. Shiv Kumar Says:

    As a software engineer I can appreciate the need to accomplish the implementation that meets the specs, however, from a main stream/consumer perspective one has to look at things from a more pragmatic approach. For example:

    1. Benefit versus pain. Does the typical viewer notice/appreciate the “improvement” if her decoder supports this feature or is it just that she notices it when it do not support this feature, in turn causing more grief than good.

    2. As a publisher is it worth it to encode videos using this new capability? Again, will the viewer notice/appreciate the difference/improvement? If not, then am I setting myself up for more grief by way of support issues and having to answer a whole host of user complaints etc.?

    I mean it great that decoder implementers are being forced to comply but can we have the option to turn off this capability in the encoder so we can decide when to and when not to use it?

  14. Superna Says:

    Hi,

    Could you make the sample stream available in raw h264 bytestream with the source yuv4mpeg frames for comparison ?

    Thanks

  15. Rolf Says:

    Forcing software developers to fix their software is one thing but how do you expect MediaTek to fix their already deployed hardware chips?
    Making an incompatible option default while knowing that it breaks certain devices in an unfixable way is very bad style in my opinion. The customers are the victims here and I for one really hope this will mean x264 will get forked and/or replaced by solutions with more sensible defaults.

  16. nigel Says:

    would anyone know if the problem is with the MediaTek chip in the LG players can this be fixed with a firmware update, or is it hard-wired?

  17. Dark Shikari Says:

    @Rolf

    I talked with someone at a Blu-ray device company before I committed this and ensured that they got MediaTek on the issue before it was even out. They believe they can fix it in software.

    Also, there are dozens and dozens of devices that are not compatible with x264 on default settings. Worse, many of these devices are mutually exclusive and CANNOT be simultaneously supported by default. How would one “fork” x264 to “remove incompatible options”? It’s not even physically possible.

  18. Stewman Says:

    I understand that LG just released a firmware update to fix the problem. I believe they use the same MediaTek chip as the Oppo BDP-83. Any word from Oppo about this?

  19. AvalonX Says:

    Add the JVC XV-BP1 to the list. I have been emailing them like crazy for a fix.

  20. vega1970 Says:

    Oppo released a firmware update for the BDP-83 on March 29th. I haven’t tested it yet, but the guys on avsforum say that it corrects the weighted p-frame prediction issue.

  21. Michael Miller Says:

    This seems like a good place to ask my question, if there’s a better venue, please let me know. We have a MediaPointe system which captures s-video and DVI inputs into mp4 format. After editing one of these videos with QuickTimeX and saving to an mov format, it plays fine from the local hard drive but when served from our Darwin Streaming Server, there is some smearing of the video in the upper right and the audio is stuttered to the point of being useless. It reminds me of how scanning professional photographs gives a smeared result, but we shouldn’t have any DRM issues since we created the content. You can see the video here: http://dss-vm.ncsa.illinois.edu/numerical_libraries.mov Any ideas what’s causing this? Why would this occur when streaming from a server and not from the local HD? Any pointers are greatly appreciated.

    Michael

Leave a Reply