I’ve been trying to come up with what to say after that sentence for couple of days now, and after all that time, all I got is “Well, at least you’ll be spared from those peskyplanepostsonPlanet right?”
Except, even that’s not quite true, since I’m planning to continue in my role as the release module owner and am hoping to play an active part in its development.
So that doesn’t work.
…
Well… I’ll keep thinking about what should come after that sentence.
Until then, it definitely needs to be said that it’s been a pleasure and a privilege working with everyone at the Corporation.
I’ll still be hanging out on IRC, if you need to find me… just holler, and you’ll get me.1
______________________ 1 Or Reed… one of us…2 2 You had to know I was gonna do some footnotes here…
It occurred to me the other day that I haven’t blogged about my IFR training in forevar.
I think that’s more a function of I-stopped-doing-IFR-training-because-I-only-have-so-much-flying-monies- and-the-summer-weather-has-been-just-too-damn-beautiful-to-be-spending-it- under-a-training-hood than… y’know, any other factor.
In late August, I flew myself to Portland’s1Hillsboro airport.
It’s a six hour flight with a fuel stop, and cost over $1,0002. Sure Alaska airlines would’ve gotten me there in ninety minutes at a quarter of the cost, but it wouldn’t have been nearly as fun.
I wouldn’t trade the experience for anything.
I learned more about myself, as a pilot, a software engineer, and an individual, than I have in… quite awhile. Of course, being aloft at 10,000 feet, flying along the Northern Californian coast, and then inland across western Oregon’s beautiful, seemingly endless deep-green forests has a tendency to… be conducive to that kind of contemplative thought.
I highly recommend it if you’ve never had the opportunity.
I flew up (and back) alone3, but since it was my first flight out of both the state and beyond the boundaries of a Californian ARTCC—Helloooo Seattle Center!—my friends demanded that I have a camera (video and still), a voice recorder to grab all the interesting interaction with air traffic control, and that I file4 flight plans, so they could “fly along.” I got some incrediblepictures and some great footage.
I finally sat down this weekend to digitize some of the video.
This was on the way up to Portland, landing at KCEC, my fuel-stop airport. The video camera was placed on the dash of the plane, not held, so the camera depicts accurate movement of the aircraft itself.
Oh, and for those that don’t readily do the conversion from knots to mph5, the (cross)winds were eighteen mph, gusting to about thirty mph6.
Every time I watch this video, I have to remind myself that the pilot is, in fact, me.
Sure, it’s not my prettiest approach ever7… and I’ve certainly had smoother landings… but when push came to shove, it’s a pretty concrete reminder that I do, in fact, possess the ability to handle the situation to a successful outcome8.
Of course, any situation can become overwhelming… in which cases, material like this video is useful in helping to figure out where the lines between a “fix it” and a go-around are.
Sometimes, in the heat of the moment, when the approach gets bumpy, it’s easy to forget that one possesses those skills… and those options.
But there’s nothing better than a beautiful, long flight with only you, your chariotplane, and the endless blue sky… to remind you that you do… and if you use them, it’ll all be fine.
_______________________________ 1 Oregon, not Maine 2 Yet more proof that you don’t take up flying because it’s faster or cheaper 3 Which means I was stuck doing my own drink and snack service… 4and close 5 Like *cough* me… 6 Astute viewers will be probably be able to see the points at which I caught one of those nice, thirty mph gusts… 7Especially in retrospect. 8A testament to my instructor, to be sure.
For quite some time, we’ve been wanting to reorganize how bits and builds find their way out to the world.
It’s a project we’ve affectionately referred to as “stage migration,” and we’re finally at a point where we’re ready to make changes that will hopefully improve security and auditability, reliability, and expandability.
The primary goals of this move are to support:
Ensuring that builds get scanned before they’re available for download
Audit access to the FTP tree and move to LDAP authentication for easier management
Move to expandable storage, backed by a highly-reliable storage platform
Combine ftp.m.o and archive.m.o, to make it easier to find builds
To address these goals, this week, the Build & IT teams will be reconfiguring the way the servers that make up ftp.m.o are set up.
The main changes that you’ll notice are:
archive.m.o and ftp.m.o will be merged into one site, which will respond to both addresses. (So, no need to switch between them if you’re regression testing!)
Directories with many builds, like firefox/nightly/, will be rearranged to allow much quicker access than the current archive.m.o. We’ll be creating YYYY/MM subdirectories, and moving deliverables into those directories. For example, the builds from March of last year would be found in firefox/nightly/2006/03/, and in that directory, you would find directories with builds like 2006-03-01-04-mozilla1.8 and 2006-03-01-04-trunk. We will be doing this for all builds older than a month.
All files will be virus scanned before becoming available. We currently virus scan all builds, but depending on a number of factors, it was possible for unscanned-builds to appear on the FTP site for a window of time; we’ve removed this window.
Interactive shell accounts on the FTP farm will be replaced with sftp-only accounts.
We will be doing most of this work during an outage window starting on Thursday, 27 September at 10 pm to 2 am. The tree will be closed during this time, because we’ll be teaching Tinderbox about the new FTP structure. Updates might be wonky on Friday, as well.
We will be moving the archived builds around earlier in the week, on Wednesday during the European day (“Go, go, gadget cf!”
We’re doing our best to minimize adverse changes but because of the above, it’s possible that your account could become disabled, or data in your home directory on the current FTP machine becomes unavailable. If that happens, please email build@mozilla.org, and let us know, so we can fix it.
With last week’s release of Thunderbird 2.0.0.6, we’ve finished a stint of security fire drills.
My friends and family have come to learn what that means1, but I often get asked what it feels like to be in firedrill mode, especially on a [consumer-oriented-yet-open-source] project like Firefox. I’ve been trying to come up with a good way to explain it, and I think I may have finally found something to help.
The idea came to me when I noticed that a scene from a particular movie kept flashing through my head, very distinctly and vividly, while working on various release activities during this last drill.
It’s from a 70′s movie called The China Syndrome2, which I found a conveniently-You-Tubed3 clip of. You’ll want to let the full clip load, since the part that flashes through my head most commonly in the last thirty seconds or so, starting at about 3:35 in.
After watching this, a friend asked “So… you’re comparing firedrill-mode to a nuclear meltdown?”4 We shared a laugh about that, because it wasn’t the analogy I was intending at all.5 What the scene does capture pretty perfectly is the sense of urgency and the mode the characters go into when “the alarms go off.”
During a firedrill, there’s bits and pieces of information coming in from all sources and angles; there’s a level of coordination involved in terms of getting issues looked at and prepping for builds; there are usually always external people watching [from above] what we’re doing and how we’re doing it; sometimes, we have to address problems creatively, even though “the book says you can’t do it”; there can often be conflicting information that makes making a decision difficult. Many of the roles, reactions, and interactions of the characters in the scene are reminiscent of how we get a firedrill out the door.
I actually hadn’t noticed that until I watched the clip again, since I originally identified with the last thirty seconds6 and hence why it tended to flash through my head at weird hours while starting at Tinderboxen prompts.
Every component, everyone one of them, is tested, and then it’s tested again. Every critical weld is radiographed. Every single thing is checked, double-checked, it’s rechecked. Everything. … In any thing that man ever does, there’s some element of risk, right? That’s why we have what we call “defense in depth.” Now that means, backup systems to backup systems. To. Backup. Systems. … The system works. Even with a faulty relay, even with a stuck valve, that system works.
I’m sure QA can certainly relate to the amount of testing we do to ensure we ship a quality product… and a number of those tests mesh with verification and testing we do on the release side. In our case, it’s more “even with an invalid automatic update snippet, even with a bogus locale repack, we catch the error and correct them before the release.”
We learn something new from every firedrill we do.
And as a result, we’ve got a lot of newer initiatives all over the project to make firedrills more smoother and faster, from automated regression testing in Tinderboxen to automated smoketests and BFTs to automated release deliverable verification. These are all going to improve on Firefox 2′s already-pretty-awesome patch-delivery time.
The analogy I make here illustrates how stressful it can be for that time we’re getting it out. All the stuff I mentioned above will reduce that stress, but I think at least some of it is self-induced8: we take this stuff and the way we respond to and engage in them very seriously, and I know I’m not the only one: everyone, from the developers focusing on the fix to the release managers coordinating everything to the QA engineers verifying the fix and looking for regressions, to the build engineers pushing the bits out experience this responsibility together. (In fact, as I watch this clip again, I identify with Lemmon’s character at the end, but in the beginning of the clip, when he gets up from his desk as the alarms initially go off, an image of dveditz pops into my head for some reason…
It’s part of the commitment to security and, more generally, doing right by our users that not only defines Mozilla and our community, but that I know we all personally hold dear.
So maybe no matter how good we get at this, there will always be little alarm klaxons going off in my head when I hear the callemail come in.
But… maybe… just a little bit of that… is a healthy thing.
_______________________ 1 “Preed’s gonna ‘go missing’ for awhile… don’t ask…” 2 If you haven’t seen The China Syndrome, I highly recommend it… if
for no other reason than getting to see a young pre-activist Jane Fonda and 70s-haired Michael Douglas. And of course, any movie with Jack Lemmon is A++++ WOULD WATCH AGAIN. 3 Oh how I love thee, copyright infringementfair use! 4 I don’t even want to think about the Slashdot headliner on that one… 5 Maybe I didn’t make the connection because—not to spoil the movie if you haven’t seen it—but there is no meltdown in the movie 6 We’ve striven for correctness on the common cases with the release automation, and in firedrill-mode, we can be faced with situations that aren’t entirely common… so there’s been some level of back-to-non-automated/manual release work. Performing certain staps manually has resulted in (somewhat familiar) typos, errors, and general suboptimality7 that you generally expect with a human—me—typing lots of commands in a magical, ordered incantation. When that happens, and it gets caught by QA, my reaction is… well… it’s why I keep seeing this scene in my head. 7 Is that even a word? 8 At least, I know for me it is… and yes, it’s a bad habit.
After having been conned convinced to buy an iPhone by ss on opening night, I finally bit the bullet and activated it a couple of days later (using a Mac… sigh), and have been using it for a couple of weeks now.
Admittedly, I’ve had more than a few “iPhone moments.” I think my favorite was when I was heading home from an early-evening appointment and a hankering for Chinese food snuck up (you know how these things are). There’s a place I always go to, so I Google-directioned them on my iPhone; the entry included their phone number. I called, placed my order, and by the time I got over there, which was on my way home anyway, my food was ready to go.
Welcome to iLife 2.0!
With that… iPros
[Continued...]
While doing a Bay Tour this weekend, I had an interesting first-time-aviation-experience ™.
First-time-aviation-experiences ™ tend to cause much after-the-fact pondering, because they typically turn out one of two ways:
a. Everything goes okay, you make it back to the earth, and you have a war story to tell in the hanger next time when it’s IFR and rainy and everyone’s having a beer instead of flying.
2. It… uhhh… doesn’t go so okay… and for some value of “doesn’t matter anymore,” it… well… doesn’t matter anymore.
I’ve had a couple of “first-time-aviation” experiences (mostly as a student pilot) that [I now sorta laugh at, but] at the time, they were a big deal.
While on the tour this weekend, we hit some pretty bad turbulence around Mount San Bruno, near San Francisco International. It was so bad, in fact, that we dropped about 400 feet in ten seconds1. Arguably2, I did precisely the right thing, and sacrificed altitude for attitude control of the aircraft, but at the expense of… comfortable flying.
It turned out to be a problem for one of my passengers.3
In retrospect, while the experience wasn’t really all that big of a deal4,5, it did present a new situation which I haven’t had to deal with yet: “How do I get this plane on the ground, as soon as possible, in crowded airspace?” It turns out the exercise is useful for dealing with “other conditions”6 which pilots must be prepared at anytime to deal with.
To narrate this experience, I isolated7 an air traffic control clip from Saturday’s flight; it’s about three minutes long, and you can hear us—Cessna seven-five-one-sierra-papa—talking to San Francisco Tower and NORCAL Departure8: 2:40 96 kbps MP3.
The relevant portion starts about two minutes into the clip:
Me: Uhh, NORCAL, Cessna seven-five-one-sierra-papa, we need uhh… to do uhh… landing in Oakland, uhm so we’re turning towards Oakland, we don’t have the numbers.
NORCAL Departure: Cessna seven-five-one-sierra-papa, okay, is everything okay?
Me: Uhh, yah, we just uhh… have someone who’s not feeling great, so we’re gonna land in Oakland real quick before going back to Palo Alto.
NORCAL Departure: Okay, let me uhh… see if I can get direct for you. Air Canada five-eighty-one, fly heading three-four-zero, proceed direct Red Bluff when able.
Air Canada 581: Heading three-four-zero, Red Bluff when able, Air Canada five-eighty-one.
NORCAL Departure: seven-five-one-sierra-papa, you may proceed direct, altitude your discretion, contact Oakland Tower on one-one-eight-point-three, have a good day.
Me: Thanks for your help, direct Oakland, eighteen-three for the Tower, seven-five-one-sierra-papa.
What’s interesting about this interaction9 is that the air traffic controller basically cleared us directly under Oakland Airport’s departure path, so that we could get on the ground sooner (about half the time it would normally take). It required a measurable amount of extra work on her part to call Oakland Tower on the telephone and request—and receive—a clearance for us to fly directly to the airport. When we got there, we were put ahead of the line to land.
This is one element of our air traffic control system that just always amazes me and gives me warm fuzzies at the same time: you can be a two-seater 152, but the second you declare an emergency, they’ll push 747s full of people out of your way if you need them to.
I’ve heard it happen.
In many ways, it reminds me of one of the most important aspects, in my opinion, of the Mozilla Community which is so hard to convey to others, but which makes being part of and involved in our Community so rewarding.
________________________________ 1 Do the math on the acceleration. 2Justin is the only one arguing with me… 3 It should be made clear: the passenger was apologetic to a fault which, while endearing, was unnecessary. I tried to reassure them by recounting the story of life as a student pilot, where I continuously got sick every time we tried to do “ground reference maneuvers.” 4 The failure condition, while somewhat… uhh… disgusting… wasn’t life-threatening, really. 5 But I’m sure it was a big deal for the passenger in the back seat 6ENGINE FAILURE 7 Which is a fancy way of saying I removed all the other stuff so it looks like navigating San Francisco Airport’s airspace is really friggin’ easy. 8 For some reason, all my friends comment that I sound “butch”—their words, not mine— on the radio. My instructor claimed I sound like a “grizzled Eastern Airlines captain.” I really don’t see it. 9 I’ve italicized the relevant statements.
So… a few weeks ago, I finally broke down and got a Facebook account.
After having spent some time filling in all the little “tidbits about me,”1 and more or less trying to interact with… “it” over the past couple of weeks, I’ve decided I don’t really grok Facebook.
The first thing I noticed about Facebook is that it is very… SYN-ACK heavy.
[Continued...]
With a time-warpish flight, the Nomadic Build Engineer ’07 Tour has come to an end.
I think United is playing games with the flight times on this flight: they claim it’s about a five and a half hour flight on the plane, it’s published as a five hour, forty-five minute flight, we left JFK almost fifty minutes late, and got into San Francisco a half-hour early.
Yah, I don’t get it either.
I flew United home, so of course I listened to channel 9 (the ATC audio channel) and got to enjoy the beauty of the nation’s aviation transportation infrastructure again.
No matter how many times I fly commercially, I’m always amazed that “RBV J230 LARRI J230 AIR J80 EMPTY SPI J80 HLC HGO J197 DVC ILC RUMPS OAL MOD3” gets turned into:
***
We went plane-spotting at JFK for a few minutes right before I left; aircraft operations out in the east are, like most things, slightly different than out in the west. They cram the planes in closer together, and when they’re not running instrument approaches, they do weird things like have them turn a three or four mile final at only 2,000 feet.
We got to see a bunch of big planes skid the turn to one-three-left today; they tend to keep their speed up, because they all have traffic on their tails, so it’s very interesting to see them come out of the turn, drop everything in, and plant it on the runway.
Especially when the plane is something like a 747 or A340.
A 20 knot crosswind kept things interesting.
I was gonna grab some of the clips from air traffic control that kept things… bearable (but probably only for me… since I was seemingly the only one sitting in my seat, giggling) while we sat and waited for fifty minutes, but it turns out that JFK’s ATC Internet feed is multiplexed, so… the joke about the pilot asking “A few whats between friends?” isn’t as funny when the recording missed the tower controller saying “What’s a few digs between friends?”
Lamentably, it also only grabbed half of the conversation about the pilots “turning into pumpkins” in a half hour.
***
Oh, and to my New York hosts: thank you. This trip was fun and memorable… for more reasons than you know.
And Tony: I promise next time, we’ll let you make omelets.
