Submitted by Mozilla, sticking to its ideals of the open web, decided long ago that support for the patent-encumbered H264 codec would not be included in any of its products. Not only is H264 wholly incompatible with the open web and Free software, it is also incredibly expensive. Mozilla could use one of the open source implementations, but those are not licensed, and the MPEG-LA has been quite clear in that it will sue those who encode or decode H264 content without a license. Software patents, however, are only valid in some parts of the world, so an enterprising developer has started a project that was sure to come eventually: Firefox builds with H264 support.
The project, dubbed Wild Fox, is in its infancy, and has not yet released any code. “The Firefox project has opted to exclude certain features due to software patents, patents which are only valid in a small number of countries, including the USA and South-Korea,” writes project lead Maya ‘Elledan’ Posch, “This means that decisions have been made due to patents which do not apply in most parts of the world. The Wild Fox project aims to rectify this by releasing builds with these features included, builds which will of course only be available to those not in software patent-encumbered countries.”
While I despise the idea of H264, it only makes sense for someone to start a project like this. The idea is to either make use of libavcodec from ffmpeg, or to go the Gstreamer route; this hasn’t been decided yet. The project is looking for developers.
“As I am just a single person, help is required to set up this project successfully and to release stable builds for as long as is required,” Posch writes, “If you are familiar with the Mozilla source and/or C++ plus have at least passing familiarity with (multimedia) libraries, you are more than welcome to join the project.”
Yes, this is what supporting H264 leads to. Due to the anti-open nature of Apple, Microsoft, Google, and its supporters, forks are necessary for users to be able to experience the whole web. To make matters worse, users in the United States will be opening themselves up to lawsuits by using these forks, since they are not licensed from the MPEG-LA. This organisation has clearly stated it will sue unlicensed users (and is clearly not afraid to do so), so beware of what you’re using.
Firefox with H.264 is a cool idea, but you don’t have to include any decoders. Just use the decoders built in to the OS (OSX or Windows). After all, Apple and MS have already paid the royalties, hence it is perfectly legal to use the built in functionality.
I have no idea about Windows (last time I was forced to program for windows, MFC was the big thing), but its pretty easy to use the built in frameworks in OSX and just send them an mp4 stream, and I assume its probably also pretty easy to do something similar in Windows with direct show.
To re-iterate: If you use the built in functionality of the OS, you are not violating any patent!, you re-use existing functionality, less code to maintain.
Edited 2010-05-16 13:24 UTC
Eh, no. More code to maintain.
Code to make use of DirectShow (XP/Vista/7), but since DirectShow is currently being phased out, you need code to support its successor too (Media Foundation). Then you need code to support QuickTime. Then you need code to support Gstreamer (and hope the user has H264 codecs installed on his system, not a given in Linux land).
So, more code.
I still content that using a set of existing decoders is a lot simpler than writing your own.
Even if you use some existing cross platform decoding library, you still need to tie the out to the windowing system, which is different on OSX, Windows, and X11.
Fine, direct show is being phased out, like I said, I have not been forced to use Windows in a long time, but I’m still assuming that MS gives you a pretty easy way of decoding video. I do know thats it pretty easy to decode a video stream with either the CoreAnimation or QuickTime frameworks, its about 20 – 30 LOC (lines of code) to decode with QuickTime.
Its probably more effort to tie the 3rd party decoder to the native windowing system than it is to use the native decoder. You are also probably shortchanging your users by using a 3rd party decoder as the build in decoders are pretty dammed optimized by Apple and MS. You also have to build, package and deliver any 3rd party libs. On the linux side, just use a package manager to make sure that the existing decoders are installed.
You still contend.
I think he is content with what he wrote.
Easier then (in Linux land, at least) to use the decoder embedded into the video card. Two advantages: (1) it is legal on Linux (since the video card is paid for, everyone has an implied license to use it), and (2) you get hardware-accelerated decoding.
Mozilla itself could ship (as open source) a browser that used video decoders embedded within the system’s video card hardware (if there is any). No need for forks and clones.
Downside: Doing this just encourages h.264 on the web, which is not in the best interests of the vast, vast majority of people.
Edited 2010-05-16 13:54 UTC
As far as I know there *is* no ‘video decoder’ embedded in video cards. There is dedicated signal processing hardware that does things video codecs need to do quickly (like DFTs, colour conversion etc.). But there’s nothing that takes H.264 as an input and gives you raw video as an output.
Here are the specs for my video card:
http://www.amd.com/us/products/desktop/graphics/ati-radeon-hd-4000/…
There is also quite a long list of other functions related to hardware video acceleration, although these are not specific to h.624 I would think.
In any event, the video card hardware covers the patented functions of H.264. There can be no patents applicable to getting the video data stream in to and out from these hardware functions, because those processes are in no way novel or inventive.
Since I paid for the video card, I have an implied license to use all of the functions of the card listed in the specifications (link above), even if the OS that I run happens to be Linux.
http://en.wikipedia.org/wiki/Implied_license
http://en.wikipedia.org/wiki/Exhaustion_doctrine
Therefore, I am licensed to use UVD 2 (which is described as a 2nd generation Unified Video Decoder enabling hardware decode acceleration of H.264, VC-1 and MPEG-2) on my ATI card (even if I run Linux as my OS). Therefore, also, it is legal for Mozilla to ship an open source browser that used UVD 2 (if it was present) to decode h.264 videos.
Edited 2010-05-16 23:24 UTC
Does Hardware Accelerated Decoding (or “hardware decode acceleration”) === Hardware Decoding? I know that the terms are often used synonymously, but are they the same thing. Surely your graphics card still requires some software which understands h.264, to tell the GPU what to do.
The graphics card functions are accessed via an Application Programming Interface … aka an API.
Software which calls that API would be termed a “driver”.
Video player software which could play H.264 videos would call the driver, which in turn would use the video card API to pass on the video data to the video card to be decoded by the card’s decoder functions (collectively known as UVD).
The decoder functions are on the video card. They are not executed by any of the higher layers of software in this stack.
My Linux system could have a player, say Firefox, or VLC, or SMPlayer, which would call the driver, xf86-video-ati, which in turn would call the card’s hardware video acceleration API, XvMC, which would then access the actual decoder embedded on the card. None of the software layers would actually have a H.264 encoder themselves.
http://en.wikipedia.org/wiki/XvMC
See? The video decoding is offloaded to the GPU video-hardware.
Therefore, the higher layers of software do not violate any H.264 patent.
Therefore, since I have purchased the video card, I have an implied license to use the patented decoder functions embedded within the video card, and I do not require any further license to use the rest of the stack (the higher layers), since these are not patented. Therefore, I can use all of this on my Linux system perfectly legally, since I have paid for the video card.
Edited 2010-05-18 14:45 UTC
For a counter example of why Acceleration != Full-Implementation see 3D accelerators. They contain neither OpenGL or DirectX, but rather functions which can accelerate their common operations.
It is difficult to say from the outside whether a particular card supports full decompression of a codec, however for ATI I believe this is the case.
XvMC doesn’t support full decoding of h264 as far as I know, only selected parts of the pipeline. I can’t say whether this is a fundamental incompatibility, or just not implemented yet. Some proprietary implementations , like VDPAU from NVidia (which works quite well for me), support the full pipeline, I believe.
The layers you describe exist in theory but probably aren’t actually being called/used in the way you describe. Firstly XvMC likely calls the xorg driver, secondly, I think the XvMC/xorg-driver layers do not support h264. I’m not sure what that means for licensing…
I think you might find that there are a great number of patents involved in h.264, but all of them relate only to one or other of the low-level functions. (Here, I use the classic meaning of “function”, which is a mathematical operation which takes one or more parameters as inputs, and produces one or mor values as output. Which, BTW, is why software patents are silly in the first place, because mathematics should not be patentable subject matter). Anyway, there is no patent which covers h.264 “overall”.
The card is advertised (by ATI) as including the following functionality within the hardware.
http://www.amd.com/us/products/desktop/graphics/ati-radeon-hd-4000/…
This is a long list.
When I buy the card, I get an implied license to use all of those functions.
http://en.wikipedia.org/wiki/Implied_license
There is no doubt at all that when I purchased my ATI HD4350 video card, I have purchased a physical embodiment of all of the functions listed in the specifications as linked above.
Within the list it includes specifically hardware decoding of h.264. I have a full and reasonable expectation, when I bought my ATI HD4350 card, that included within that purchase was a license to use the h.264 decoder which is described as being included within it.
Any lawyer representing MPEG LA is going to have a very, very difficult task trying to claim that I must pay again for a licenase for the same functions in my Linux OS, because of this:
http://en.wikipedia.org/wiki/Exhaustion_doctrine
Having been effectively sold one license for H.264 decoding, I don’t need to buy any other. To imply anything else would be misrepresentation at least, or outright fraud even.
Now, the really, really interesting thing is this: why exactly are people like you so keen to try to imply that this is not the case?
Hmmmmm?
Edited 2010-05-18 23:35 UTC
Sure, but the primitive functions have to be composed in the correct manner to make the more complicated ones. For example, you can provide matrix multiplication primitives, which then are used to do all sorts of interesting things (rotations, some basic cryptography, statistics, etc). The primitive operations could be accelerated, while the upper-level ones (patented ones, possibly) are not. However, I don’t believe this is the case for modern video cards, in most cases they would have a full decoder onboard.
I was merely responding to the idea that one may always construe ‘acceleration’ as ‘full-implementation’.
Wow, you’re paranoid. It was just a question, I’m no lawyer.
Neither am I a lawyer, but I do know when companies are trying to rip me off, and I do know a little about what rights I should have as a consumer.
Perhaps I am a bit paranoid … you copped that response because it is an absolute favourite notion of some supporters of commercial software on OSAlert that Linux users have no legal way to render h.264 video. They desperately want h.264 to be illegal on Linux systems, whilst at the same time they try to insist that h.264 can be hardware-accelrated but Theora cannot (which is not true).
Little do they realise that their two favourite themes in respect of h.264 are actually at odds with one another.
If h.264 over the web is to be hardware-accelerated when viewing the video in browsers, then Linux can do that just as legally, and with the same performance, as commercial OSes can, because Linux users have also paid for their video card hardware, and they are therefore licensed to use it.
If video over the web is to be software-rendered when viewing the video in browsers, then Theora is by far the better choice because it is significantly less demanding on the CPU.
Either way, h.264 web video (or Theora web video) just isn’t going to be a lock-out for Linux or a lock-in of commercial OSes and browsers.
This is especially so when you look at the constantly-improving performance of Theora, in particular at low bit-rates. Here is the latest update:
http://people.xiph.org/~xiphmont/demo/theora/demo9.html
Here is the history:
http://people.xiph.org/~xiphmont/demo/
Edited 2010-05-19 06:48 UTC
Eh no. It’s less code to make use of existing media framework and codecs rather than write your own. Unless you’re a fan of NIH.
It^aEURTMs not as simple as that. Outside rendering, such as with Flash and QuickTime do not act as a native element that you can apply CSS effects to nor layer correctly. HTML5 video acts the same as any image element. To do this, the video decoding has to be part of the browser engine, and this gets particularly tricky around colourspace conversion. HTML5 also provides the ability to get the current frame of a video playing. This sort of thing could prove to be really tricky to do when another thread of another system is doing the video rendering (even in the GPU).
To have any hope of this working cross platform in a reasonable time frame (and not potentially requiring changes to upstream projects) they will have to build the decoder in.
This is not true. It is possible to use 2D compositing or 3D transforms on video rendered by native OS libraries and doing so will take best advantage of the hardware in the system. It might not be trivial to do this in Firefox’s existing rendering pipeline but if so this is only a reflection of where Firefox falls short in using built-in hardware acceleration for graphics.
Flash and QuickTime are separate matters as HTML5 video is not constrained by the rules of NPAPI.
This is very interesting.
If this is the case, then from a technical standpoint alone, because it is far less demanding of CPU for decoding, Theora should be the only codec for HTML5.
Only if you use the CPU for decoding. But H264 already is widely decoded by specialized hardware chips, both on PC video cards and on mobile devices. When that is the case H264 decoding is done with near zero CPU load, freeing the CPU for other tasks.
That’s one of the problems that proponents of Theora and open standards will have to overcome: members of MPEG-LA are the ones that get to dictate which devices get hardware video decoding, and for which codec.
Because of this, Theora is a no go on mobile devices; decoding it in software is simply not practical in the near future. It is acceptable on PC’s because the average consumer PC has enough juice nowadays to cope with video software decoding and keep the UI responsive at the same time.
Err, you didn’t read the post to which I was replying, did you. That post claimed this:
So, when you say that “H264 already is widely decoded by specialized hardware chips” the actual point to which I was replying (to whit, the bit about the video decoding having to be part of the browser engine) went wooooosh right over your head, didn’t it?
http://people.xiph.org/~j/bzr/theora-fpga/doc/leon3_integration/
http://www.bitblit.org/gsoc/g3dvl/index.shtml
http://wss.co.uk/pinknoise/theorarm/
Your merely asserting something doesn’t make it so.
Edited 2010-05-17 13:56 UTC
Not true. That code already exists for GStreamer (Songbird uses it, for example).
A Wild Fox release would actually be relatively simple to do: Grab the existing GStreamer patch from https://bugzilla.mozilla.org/show_bug.cgi?id=422540 (developed by Mozilla itself, I may add) and distribute it with GStreamer + GStreamer’s DirectShow/QuickTime plugins, and you’re done.
…and not using the native functionality is good how, exactly? I rather like video not using a ton of my CPU, and that requires taking advantage of various fully or partially platform-tied features, right now.
If that’s not going to get used, then there’s little benefit to using any standard MPEG codecs, and low-power platforms get screwed (in the case of the Atom, you get screwed anyway, but you at least have options to reduce the pain, like ION).
The interface to a video player is an order of magnitude less work then maintaining the player itself.
K
WebKit IS LGPL, so just look how they do it, and copy it. Note, WebKit uses CoreAnimation on OSX, and some DirectX library on Windows to display video.
Safari uses QuickTime on both Windows and Mac. Chrome has the decoders built directly in.
And put the add-on(s) outside US and check with a lawyer the details…
There has been all this banter in the last few months about h.264 vs Theora vs… Instead of arguing the technical merits of h.264 vs. Theora, which is the discussion we should be having, we are arguing the legal issues.
The only reason we argue the legal issues that countries whose governments (regardless of party) are bought and paid for by multinational corporations have this asinine concept of software patents, whereby any abstract, vague concept can be patented.
The issue we need to be having is how do those who live under such corrupt governments go about changing things. How do we really go about eliminating software patents.
I think the best approach is to form a patent troll that sues for shutting companies down, rather than suing for money.
That is, patent trolls are usually in it for hit and run profits. A patent troll that tried to shut down HTC, Microsoft, and Apple all in one fell swoop… that would trigger patent reform, and fast.
When users have to pay for your politycal agenda this is what it happens.
I say kudos to this project.
Edited 2010-05-16 15:29 UTC
I say kudos to this project.
Yeah, good project. Too bad it is still illegal to use in the USA, Japan, Down Under.
This is a common misconception. Using it is not illegal. Distributing unlicensed MPEG codecs from inside the US etc. is illegal under under the condition that more than 100,000 copies are distributed per year. (Distributing less than 100,000 is free according to the MPEG-LA’s AVC licensing website.)
So … include a torrent program in firefox and distribute it from there each user would distribute less than 100 copies give or take.
Simply host the server outside the US. Mandriva are doing the same and no one ever sued them for it.
True, though best to keep in mind that MPEG-LA has threatened to go after anyone they find using an unlicensed decoder or encoder. That in and of itself doesn’t make it illegal to use (that’d be for the courts to decide) but no doubt the legal fees to defend against the MPEG-LA would bankrupt your typical US citizen.
That’s a claim I’ve read often, but the actual MPEG-LA website says otherwise. In fact, the MPEG-LA does not even have a licensing program for users. They only have one for distributors and even distributors don’t have to pay anything as long as they don’t have more than 100,000 customers per year.
Giving a free ride for anyone who distributes 99,999 copies per year, but suing individual users who don’t have a license, because they can’t even obtain one? That does not make sense.
Such a case won’t even be accepted by courts.
Edited 2010-05-16 20:27 UTC
Source? They’ve made it clear that anyone using an unlicensed implementation is breaking the law, and that they reserve the right to sue in that case.
But that doesn’t matter. They have no obligation to provide you the means to license it, and can still sue anyone who uses it without a license.
Probably not, but look at the MPAA and RIAA. They went after individual users to try to create a public example to deter others. Sometimes it’s easier to go after the little users than the larger groups distributing, which can always just move their servers somewhere else.
Now that’s just silly, of course it would. You’re trying to look at this using common sense, but that’s something that has no place in a court of law.
You’ve got to be kidding me. The MPEG-LA website is the source. If you’re incapable to visit the site and browse through the AVC licensing terms, you have to learn to use the internet first before you post here.
You asked me for a source, you don’t provide one for your claim? Hilarious.
Show me an actual press release by the MPEG-LA that states that non-distributing home users are about to be sued by the MPEG-LA and show me how that would be compatible the the licensing terms.
You’re joking, right?
AVC licensing by the MPEG-LA solely centers around distribution: distribution of content and distribution of codecs.
Both kinds of distribution have the same terms: Free of charge under 100,000 copies per year.
Let’s say I wanna give my 5 friends videos of my family vacation. When I do it, I’m still way below the 100,000 limit.
No. They have nothing to do with this topic.
Copyrights and patents are even covered by entirely different laws.
Neither MPAA nor RIAA have actual licensing terms posted on their webites that grant free distribution of less than 100,000 copies per year.
Don’t confuse your state of mind with mine.
Edited 2010-05-17 00:11 UTC
http://lwn.net/Articles/371751/
In response to your specific question, under the Licenses royalties are paid on all MPEG-4 Visual/AVC products of like functionality, and the Licenses do not make any distinction for products offered for free (whether open source or otherwise). But, I do note that the Licenses addresses this issue by including annual minimum thresholds below which no royalties are payable in order to encourage adoption and minimize the impact on lower volume users. In addition, the Licenses also include maximum annual royalty caps to provide more cost predictability for larger volume users.
I would also like to mention that while our Licenses are not concluded by End Users, anyone in the product chain has liability if an end product is unlicensed. Therefore, a royalty paid for an end product by the end product supplier would render the product licensed in the hands of the End User, but where a royalty has not been paid, such a product remains unlicensed and any downstream users/distributors would have liability.
My bold. This was discussed quite broadly at the time:
http://blog.christophersmart.com/2010/02/01/mpeg-la-confirms-h-264-…
http://weblogs.mozillazine.org/roc/archives/2010/01/h264_licensing….
http://www.osnews.com/story/22828/MPEG-LA_Will_Not_Change_h264_Lice…
Fortunately, there is a H.264 decoder embodied in the video card of a good number of systems in use today. Given that people have purchased their video cards, and that the supplier of the video card has paid the license fee, then end users have a license to use that h.264 decoder (embedded within their video card) even if they happen to run an open source OS.
Edited 2010-05-17 00:37 UTC
However, according to Allen Harkness, Director, Global Licensing, MPEG LA:
http://lwn.net/Articles/371751/
“Therefore, a royalty paid for an end product by the end product supplier would render the product licensed in the hands of the End User, but where a royalty has not been paid, such a product remains unlicensed and any downstream users/distributors would have liability.”
This means that if the (perhaps Chinese) OEM of the digital camera or mobile smartphone which you used to take your family videos did not pay for a license, then you are liable.
It also means that if any of your friends watch your family video with decoder software where the author of the player has not paid a license (such as ffmpeg/gstreamer, which may be embedded say in an android mobile phone) then your friends are liable.
This type of liability, where the persons (end users) who end up being liable have absolutely no way to check if they are licensed or not, means that H.264 is in reality totally unsuitable for the very type of activity which you give as an example.
Edited 2010-05-17 03:11 UTC
OK, here you are: http://lwn.net/Articles/371751/
The important bit is at the end:
Care to show me your source now? That seems pretty darn ironclad to me.
Yeah, but how did you give it to them? I hope you didn’t put it up on a public website, because anyone could get to that and easily bump it over the limit. I guess we’re back to using snail mail and sending people DVDs. And if you break that distribution rule, then everyone else who took advantage also suddenly broke the rule, without even realizing it. So if you’re an innocent user, would you really trust putting your legal liability in the hands of whoever that distributer is?
They stand in as examples of large litigious groups trying to protect IP. Sounds a lot like the MPEG-LA to me.
Of course copyright law is different than patents. You’re completely missing my point.
Dude, there’s no reason to get nasty here. Do you really think a judge is going to throw a case out just because it’s against a small end user rather than a larger target? Seriously?
Edited 2010-05-17 03:16 UTC
If GStreamer or the native platform player is used then there would be no issues with legality. Ubuntu paid the licensing fees for h.264, both OSX and Win7 have h codec included in the OS on top of all the hardware that has licenses as well. They should be covered depending on how they implement it.
Ubuntu paid the h.264 licensing fees only for its enterprise customers.
Whenever I try and play html5(theora) videos on FF, it is like watching a slideshow. Same computer, same videos with chromium is fine.
The issue that really gets to me, like it does a lot of people, is the patents. It’s really evil, spiteful and anti-consumer what these companies are doing. Even going as far as to attack the open community developed alternatives.
I don’t blame Mozilla for wanting to settle this before everyone standardizes on H.264 for web video. Apparently It’s a really nasty issue that should’ve been dealt with on video cameras long ago, but almost no one knew/knows about it’s legal traps.
Clearly the only idealism showed here is from pro-monopoly economists and policy makers. It can clearly be shown here that monopolies aren’t spurning innovation, at least not the kind people want.
Why are we just talking about this online, on pages people can safely ignore? People use to riot and march in the streets, not they’ll “send an angry email – that’ll show ’em!”. I’ve been reading all these stories and getting pretty tired of the same story again and again; get the story out to major news organisations, camp outside W3C offices with placards… make a fuss!
We keep talking about this stuff as an inevitability – well it will be if we keep up this low standard of protest! My apologies for those who may have been to major news organisations and have been camping out at W3C, etc. If you are, then great, BUT I’VE NOT HEARD OF YOU OR SEEN YOU. If that’s the case, make yourselves known.
Thank god the EU voted overwhelmingly against software patents ever though MS lobbed hard for them…
In my country (the UK) I will legally be allowed to use this version, in fact most of the world will be able to.
Shame that it will illegal in the USA ‘land of the free’ …
Sorry. I wanted to post this reply somewhere else, but you froze the thread too early.
Certainly an excellent quick fix to the problem of trivial patents is to make patent applications prohibitively expensive to increase selectivity. But why should small companies have to be affected more than large ones? Why can’t fees be proportional to the income of the corporation, or something like that? It’s not a rhetorical question, really, why?
I understand that it’s in their best interest to play the naive and helpless observers of kindergarten egalitarianism, but how come open source zealots and always-holier-than-politicians journalists don’t promote this idea either? Is the world really so insane?
I just want to hear somebody’s opinion.
yes.
An even better solution is to adopt the European Patent Convention, and simply exclude certain things as being unsuitable subject matter for patents:
http://en.wikipedia.org/wiki/Patentable_subject_matter#European_Pat…
My bold.
This approach pretty much solves the problem of patent trolls straight away.
For example: H.264 would not be patentable as a software codec because it would a program for a computer, and it would also not be patentable as a hardware-embedded decoder on video cards because it is a mathematical method.
Problem solved.
Edited 2010-05-19 02:49 UTC