If You [Can't] Build It, They Will [Still] Come
Slashdot ran a Your Rights Online post a couple months ago posing the question “Do Build Environments Give Companies an End Run Around the GPL?”
The upshot: are companies1 basing their device firmwares on Linux breaking the GPL by posting only their source code, but omitting details2 regarding the environment required to to build that firmware, much less flash a device with these customized firmware.
It’s certainly an interesting question.
Ask any developer and they probably wouldn’t consider the build environment as at all related to GPL-compliance requirements. That’s likely because the vast majority of open source software builds on any standard GNU/Linux machine3; the context of “GPL-compliance” is version 2 of the GPL, released in 1991, when Linux-using embedded devices wasn’t on anyone’s radar.
But as embedded and mobile consumer electronics companies have leveraged the wealth of open source software to bring products to market quickly, this has become a very real issue, and the keepers of the GPL, the Free Software Foundation, have realized that could be a problem.
One of the main issues is “tivoization“, named after the obvious reference after they disallowed the execution of firmware containing modifications on their hardware. Such behavior is specifically restricted in version 3 of the GPL.
In the latest version of the GPL, this behavior was specifically called out as a allowable use of the end user and/or developer.
This problem doesn’t just affect embedded devices.
A common starting point for an open source hacker poking at these types of products it to try to reproduce what a company ships to its users4. But these days, that may not even be possible in software.
Mozilla Corporation, for example, now builds some of its builds with profile guided optimizations. This requires building Firefox, running it in an instrumented fashion, and using that data to guide the optimization of hot spots.
To provide a good, real world set of runtime data, Mozilla tests with a static page set of about 200 websites on the web. But it can’t release this archived content—now used in the build process—due to copyright restrictions.
So, even though open source developers are building Firefox on Win32, it won’t match what Mozilla Corporation actually ships to millions of users5.
To be sure, the issue of build systems, specifically, involves a very narrow interpretation of the GPL. Even in the cases the GPLv3 specifically addresses, there isn’t broad agreement among open source developers that such use should be disallowed7.
It may be annoying to open source hackers to be unable to fully experiment with custom firmwares their shiny new embedded, consumer product whiz-bang device.
It may take more time to reverse engineer how to flash these devices8 to get firmwares on them.
This sort of “freedom” may require a tradeoff in terms of enjoyment or functionality of these devices as manufacturers move more functionality into service offerings in the cloud, they can refuse access by these custom firmwares10.
But up to now, users have voted with their dollars and download clicks: they don’t care.
Most open source developers apparently don’t either.
_______________
1 who have the resources, but not the resolve
2 Either images or documentation
3 The venerable ./configure && make && make install triad
4 The so-called identity proof
5 To be fair, I cite Firefox as an example of this problem because it’s one I’m familiar with; Mozilla requires probably one of the most complex open source build environments around, and they’ve done a good job of, especially, documenting their build environments to the extent possible6 and making it easier to build on Win32.
6 Something I have no qualms asserting a large chunk of responsibility for having made happen
7 Linus Torvalds, for one, has stated that he doesn’t consider such restrictions on the use of the hardware to be a problem
8 And make no mistake, there are open source hackers out there, that totally get off on that sort of stuff, so they will find a way to do it9
9 Even if it nets them a few bricked devices
10 As Sony has done with its “Other Operating System” feature, and now routinely does with the Playstation Network and PS3 updates
My background in commercial/proprietary software required me to often have to setup a CD or harddrive with *everything* required to build/bundle/deploy a system because that was the requirement for escrow. This often required me to have to worry about test files, docs and other non-build items as they were involved in the process. I bring this up because maybe that’s the level of detail that should be required here – not only the makefile but also the helper scripts that automate the process.
Surely one can commit a representation of the optimizations used in each particular build? That wouldn’t seem to be copyrighted by anyone except Mozilla. On the same lines, someone commented about a leak of the Cisco IOS code a few years ago that it wouldn’t be much use without the build infrastructure anyway.
@Bear:
Even here, I think Mozilla does a pretty good job; the Build Mercurial repository, as you know, contains a lot of this stuff. But even still, that’s not enough, and that’s really the point of this post.
I suppose a better metric would be “reproducibility,” e.g. “With the tools I’m given, can I reproduce exactly what was sent to users?” If I can’t, then I’ve failed that test, and for what it’s worth, Firefox on Win32 fails that test.
If we use this metric, it’s also easier to see why it’s sort of an axis which open source people don’t even think: there is no “official, commercial source” of, say, a GCC distribution. Linux distributions, which package GCC, are the closest thing to such releases, and they generally (thinking of Debian, most notable) do have build farms with enough auditing/documentation and scaffolding (.spec files, etc.) to reproduce the build exactly.
@Matt:
Great to hear from you!
That’s an interesting question; I suppose the answer would be predicated on whether cl has a redistributable format of the representations of its optimizations; I would bet they don’t, but I’d love to be proven wrong, not only because it would solve this problem, but because it would be interesting to see what the representation was.
Even if it existed, it does provide yet-another hurdle for open source developers; Mozilla’s modifications to the Microsoft C Runtime to support jemalloc give a good example of “It’s possible to do this, but really difficult.” As an example, see the patch you have to apply to the MSCRT sources, which (of course) they can’t distribute: http://mxr.mozilla.org/mozilla1.9.2/source/memory/jemalloc/crtvc8sp1.diff
I suspect compiler optimization representations would be equally difficult/annoying.
FWIW, you’re wrong about the PGO builds. The profiling run is done on a small set of pages, all of which are in the build/pgo directory. Most of the win from PGO is just from starting and stopping the browser anyway, adding more pageloads didn’t really move the needle.
@Ted:
Thanks for the correction; I do appreciate it.
Was this always the case? I thought the PGO runs used to be all of the Talos page set? Did that change at some point because it wasn’t, as you noted, a real win?
The argument could be made (although, even I will admit that it’s stretching the original point) that not having the Talos pageset available for (non-MoCo1) developers, still presents a similar burden, but since that would be most useful for getting stuff into the tree, it’s certainly less relevant to Joe-random hacker, or the example the original Slashdot poster gave.
As I noted, I wanted to cite another example that wasn’t just firmwares and embedded devices; as Bear notes, it seems to be a common problem. Having said that, I hope you read footnote 5, and remember the post I did on MozillaBuild; Mozilla, on the whole, is one of the better projects in this area, especially given that Win32 is (arguably) its main platform, and building there in the “bad ol’ days” was excruciating.
I can [and as I feel it necessary to note, have no problem doing so
] take some credit for that, but certainly you, bsmedberg, and various members of the Mozilla Corporation build team can as well.
__________________
1 I believe; while I was researching this, I found a wiki page referencing a file to obtain, but no link or further instructions on how to obtain it that I could see