Build System Improvements By the Wall Clock Numbers
 It’s like a Turbo button… only faster!12 |
In between the constant grind of getting releases out the door, it’s often a struggle to find time to make improvements that are imminently noticeable.
That’s not to say that us build engineers are sitting in a room, typing make && cp over and over again and staring at the screen until it finishes.2
On the contrary, improvements to our build and release infrastructure are made constantly.
But they often involve making our lives easier: a script that handles errors more appropriately, or hooking up two pieces of the puzzle together, so we can go to bed at 2 am, instead of 4 am. These are not readily visible to the end-user or our customers, the developers3.
So, given that, you’ll have to forgive me for being a bit squee4 about the following:
svn up -r 13127 | grep ^Updated && time make -f songbird.mk > /dev/null 2>&1 && make -f songbird.mk clobber > /dev/null 2>&1 && svn up -r 13128 | grep ^Updated && time make -f songbird.mk > /dev/null 2>&1 Updated to revision 13127. real 3m45.008s user 3m7.301s sys 0m26.064s Updated to revision 13128. real 2m40.991s user 2m22.129s sys 0m17.253s
Yes, that really is a 29% improvement in wall-clock build time.
On repeated trials, it drops to about 26%. I’m guessing this has to do with caching the source files.
And these results are from Linux; since the core problem7 had to do with excessive use of $(shell ...), and therefore fork(), the results should be even slightly better on Win32.8 I didn’t test Mac, but one of our Mac gurus twittered the Turbo before I mentioned it to anyone.
Lessons learned:
- make‘s $(shell ...) construct is very useful. It’s also extremely dangerous, in part because it’s so convenient, easy to use, and… useful. If you can do anything else to avoid it, do that.
- If you’re assigning the results of a $(shell ...) call into a variable, you probably want :=, not =. And if you really want =, see LL#1.
- Win32 MSYS fork()ing performance has a notoriously bad reputation… but it’s not like fork(2) is free on Linux or Mac. I’m surprised at the savings on Linux and the small delta between Linux and Win32.
- Running make SHELL="/bin/sh +x"9 on your project might prove very interesting to you. Are you really executing that much extra fluff in a subshell? Yes. Yes you are.10
- Make has a surprising number of useful functions; if you’re spending any time doing build system stuff, read up on them. I myself was guilty of doing ifeq (exists,$(shell test -e $(FILE) && echo exists) before I learned about $(wildcard ...). The former spawns three processes. The latter: zero.
This is just one of the few makeovers11 that I’m working on; there’s something in the works for the installer, and I’m working on some pretty fundamental changes to the build system itself.
(I call that project Build SystemNG, or BSNG for short.)
I’d be lying if I said I didn’t enjoy working on something that people notice… either because it’s noticeably faster or completely broken.
One of the two.
__________________________________
1the famed “rebuild the airplane while it’s in flight”-problem, which I’ve decided is probably the most asinine analogy I’ve ever heard. So I’m going to stop using it.
2 As some might have you believe
3 Except, maybe, if you’re paying attention to the amount of scotch consumed
4 I keep thinking everyone knows what this means, but I keep running into people who don’t. Even people who I would think would… don’t. So here’s a convenia-link5 for it to Urban Dictionary, in case you don’t.
6 Like a perma-link, only less permanent, and more convenient
7 Bug 15529 has all the details
8 I only had a VM to test those; I don’t have the numbers in front of me, but as I remember, they were somewhere between 30 and 32%.
9 A benchmarking hint from Mecklenberg’s O’Reilly GMake book, which I recommend
10Over half a million of them, actually… which is why tips-and-tricks are always helpful: everyone wants faster builds.
11 Hahahahhaa… make over? gmake? Get it?
12 Photo courtesy D.G.S..
Nice catch! bsmedberg has been ferreting some things like this out of our build system (as noted in that bug you linked). Ideally we’ll get to a place where we can replace GNU make with pymake, and just skip process execution for anything that’s not a compiler or linker. The vast majority of the shell commands that we execute are trivially reimplemented in Python, so that should be a fairly easy way to make things faster.
Yeah, I read with some interest about the pymake stuff.
I’ll be interested to see where it leads; I have mixed feelings about it in general (immaterial of what seems to be the fundamental goal of the project, which seems sound), but, as I noted, “everyone wants faster builds.”