The Sober Build Engineer

Managing Mail Madness with Mutt

Monday, December 12th, 2011

A few weeks ago, I tweeted about a great blog post on a technique for managing the massive amounts of email many of us navigate daily.

I’d become frustrated with the state of my own inbox¹ and had been on the prowl for a new method to manage the email torrent.

Having used it for a little over a week now, I can say: it works wonders!

One caveat: the author uses Postbox, which has some core features that are the cornerstone of the technique.

Now, I’ve been a longtime proponent of Postbox, and I use it as my work client. But for a variety of historical reasons, my personal email-life is centered around the venerable Mutt mail client². So, the technique described in the article works perfectly if you use Postbox.

This is how I successfully re-created the work flow with Mutt.

Labeling

The technique relies heavily on the ability to label mail, as Gmail and Postbox both easily and handily support.

Mutt calls this “tagging,” and while there’s been support for reading the X-Label header since 1.4, support to edit labels isn’t in a currently-released Mutt.

The good news: there’s a patch to support label/tag editing.

The bad news: the patch is against a pretty old version of Mutt³.

I’m a Gentoo user, so I updated the patch⁴ to apply to [Gentoo's] 1.5.20^5,6 and also wrote an ebuild⁸ you can add into a local overlay to make it easier to install⁹.

After that, add the following to your muttrc to bind label-editing to keys and display labels:

# Add X-Label to your unignored headers
unignore From: To: Subject: Date: Cc: Reply-To: X-Label:
color header brightwhite default '^X-Label:'
macro index \Cy " ~y " "Limit view to label"

The patch binds the ‘y’ key to adding, deleting, or changing a message label in both Mutt’s message and index views; the above config binds Ctrl-y to limiting the view of the folder based on a specific label.

When editing labels, they are space-separated.

Search, Don’t Sort

The “Search, don’t sort”-mantra isn’t new; but when all you have is grep, you still tend to sort your email as an initial step to being able to easily find it later.

Enter mairix¹⁰.

Setup is relatively easy: configure it with an rc file, and run it. I run it from cron every four hours to keep the index fresh, and reset the entire index every month¹¹

Since mairix provides its results as an mbox or Maildir folder, I wrote a simple wrapper that only launches Mutt if there are results to my search and makes the folder read-only, to keep the conceptual model—these are search results, and I shouldn’t edit them—consistent.

As for sorting, I still do sort some email, mostly Twitter, Facebook, and other automated notifications, as well as mail from important people that tends to fit neatly into boundaries (my parents, certain friends, etc.)

Keeping Folders Tidy

Now that my mail is searchable, where we save it isn’t as important. Now I can move to a model of just saving everything to mutt’s default saved-messages and going from there.

Only problem is, won’t that folder balloon over time? Probably.

Cron comes to our rescue again; I have the following entry:

5 0 1 * * MAILBOX="/home/preed/mail/saved-messages-archive/saved-messages-$(/bin/date +\%Y-\%m)" && /usr/bin/find /home/preed/mail/ -maxdepth 1 -type l -name 'saved-messages' -exec rm {} \; && /bin/touch $MAILBOX && /bin/ln -s $MAILBOX /home/preed/mail/saved-messages

This makes “saved-messages” a symlink that automatically gets updated at 12:05 am every month to point to a folder with the year and month.

And mairix updates its index with these new folders the index gets reset every month.

I use this same technique for my sent-messages, spamassassin-populated spam folder, and deleted-messages folder; the last two are auto-deleted by cron six months later.

Filtering Assaults On Your Inbox

The last tenent of this email management technique is probably the most important: if you don’t want it in your inbox again, set up a filter immediately. Don’t wait.

I’d been getting spam mails from Ticketmaster forevar. I didn’t set up a filter for them until I started using this technique.
It’s a bit costly at first, but it pays off and quicker than you think.

To help with this, I set up an easy way to test procmail rules without running them on my live inbox.

I stole the scripts from this page and slightly modified them to fit paths on my system.

Inbox Zero?!!

After using this method for about a week, I was able to dump most of my inbox into a saved-messages-archive folder, and begin using the dated saved-messages folder.

I’ve been able to find any message I’ve needed to find, and my inbox now hovers around 50 messages.

Finally, people emailing me are getting responses on the order of 72 hours; before, if their message got lost in my inbox, it was hit and miss; I just restarted a four month old thread, because I’d lost track of it, and couldn’t tag it “todo.”

Have I reached Inbox Zero? No. But as the original post points out: “I don’t care about ‘Inbox Zero’ because there’ll be more [email] tomorrow as soon as the sun comes up.”

But using this technique, I’ve been able to turn my inbox into something manageable, without the fear¹² that I’ll lose a message.

All in all, I’ve been very impressed with how it’s been working so far. Hats off to Mr. Ignition for the inspiration!

_______________
¹ Over 2,000 messages in 75 megabytes before I started this project
² Having used them since I was 13, my customized Mutt key bindings, cribbed from Pine, are hopelessly burned into my muscle memory
³ It’s against 1.5.1; the current released version is 1.5.21
⁴ PGP sig
⁵ Which I’m using because of an attachment handling bug in 1.5.21
⁶ I also hacked up patches against pristine-1.5.20 (PGP sig) and pristine-1.5.21⁷ (PGP sig)
⁷ This patch doesn’t include the documentation changes; that’s left as an exercise for the reader
⁸ I just added one to the ebuild revision; I’d love to hear from a Gentoo-dev on whether this is the correct way to do this, because I’m betting it’s not…
⁹ There’s also a method to allow editing of labels using an external script, but I didn’t use this script, so I can’t vouch for how well it works
¹⁰ Which I have @scanlime to thank for turning me on to
¹¹ A full index of my 1.6 gigabytes of mail takes 97 seconds; reindexes only look at new messages, so it’s pretty low cost
¹² Which was probably largely irrational anyway

Tags: lessons, process, tips, tools
Posted in blahblahblah, personal, tipstricks | No Comments »

The Elevator Storyteller

Tuesday, August 30th, 2011

I recently read FastCompany’s interview with John Lilly.

I knew John from my time at Mozilla¹, so it was interesting to hear him talk about the developments after my tenure at the Corporation.

One thing he said stood out to me:

[W]hen I was at Mozilla the activity I did mostly was to tell the story—tell the story simply, understandably over and over and over again.

… I’d feel bad about telling the same story 50 times, so I started to change it and embellish it from time to time. What I realized is that if I told a different story to 50 different people, then suddenly the whole organization will be slightly out of alignment. And so, my job was to tell the story in a simple way that was repeatable and amplifiable, not to make it all diverge.

This was an interesting insight to me.

I have struggled² to explain precisely what I do and why it’s important.

I’ve had varying degrees of success over the years, and the success I have had has been inconsistent.

As I thought about the above excerpt, I realized what I may have been missing: a simple, coherent, engaging way to accurately, yet consistently and succinctly tie it all together: the so-called “elevator pitch.”

I was phone screening a candidate recently, and they likened their role as a release engineer to that of a mailman: “She doesn’t write the letter; she didn’t drop it in the mail; she didn’t make the envelope; she’s not going to read it, but she’s critical to moving the contents around and ensuring it gets delivered.”

While I liked the analogy, longtime readers know I prefer a different one: air traffic controllers.

I’m often asked why I continue to find this analogy so relevant.

• Air traffic control is operational in nature: functionally, it involves a lot of planning and talking, but it’s fundamentally an exercise in scaling process to keep people safe and keep things moving.
• There’s a level of detail orientation that many other professions simply do not require.
• It’s an open secret that air traffic control towers are often understaffed and under-resourced; this can turn situations that would be small problems into catastrophes, which leads me to…
• No one pays attention to air traffic controllers until catastrophic events occur, and then (and usually only then) are they under the spotlight.
• Air traffic control is a service-oriented role, but to maintain sanity and ensure consistent outcomes in a complex environment with many actors with competing requirements and goals, they must do so within a framework³.
• It’s well-known that it’s a high stress environment⁴.
• Controllers’ (advertised) responsibility is the safe, expeditious movement of traffic, but they must prioritize that necessity with procedure and air craft emergencies and deviations; good ones know when and how to safely issue shortcut instructions.
• On its face, controllers’ work seems simplistic: “Move the blips around, make sure they don’t hit each other and talk on the radio all day? A chimp (or maybe a computer) could do that!” This prompts some interesting perceptions about their role⁵.
• A pilot won’t realize why air traffic controllers are useful if they only fly on sunny days with no turbulence in the middle of nowhere, by themselves.
• Controllers can be one of your best friends when something goes wrong with the plane and a pilot finds themselves declaring an emergency.

Replace “controller” with “build engineer,” “traffic” with “release” or “check-in” and “plane” with “project” and you’ll start to see why I find the similarities so compelling, at least when release engineering is approached the way I approach⁶ it. Now, to turn it into a narrative like Pushing Tin.

Or maybe Top Gun.

_______________
¹ I have more-than-a-few-vivid memories of interactions with him, including when he interviewed me
² Especially back at Mozilla
³ And expect pilots to do the same
⁴ [Never ending] Firedrills anyone?
⁵ And in many ways, informs how they’re treated
⁶ Pun intended

Tags: aviation, lessons, process, release engineering, strategy
Posted in Preed on Build/Release Engineering, Releng Machinery | 2 Comments »

Hatin’ on the Critics

Tuesday, July 19th, 2011

Allow me say this as plainly as I can: all “haters” are critics¹, but not all critics are “haters.”

During a recent (mostly-online) discussion, one of the responses to the issue at hand that I (indirectly) ran across was “don’t be a hater yo.”

It’s a common response in tech circles these days, especially among open source projects and startups in pursuit of high visibility and using 3rd party developer-engagement to do so.

The statement is many things, but it is not, I repeat, not a substantive response to a reasoned question.

In fact, it’s even worse than a substance-lacking response: it frosts the non-response response with a thin, sugary layer of ad hominem frosting.

Every time you say it, you might as well be saying “I don’t want to be burdened by having to explain my position; I want to continue thinking what I think and I don’t want anyone or any thing² to challenge my opinion. So shut the fuck up before that happens³.”

Which is a totally fine response to give to someone who’s challenging you.

But don’t delude yourself that muttering something about “haters” is the same as making a cogent point or somehow adding substance to the conversation.

When you Godwin-ily decry “the haters,” it’s an indication that you’ve failed to do either, and you’re hoping no one else noticed.

But it turns out: we do.

_______________
¹ Mmmm… probably
² “Data!”
³ And people—”geeks,” actually—are confused/surprised/angry at the suggestion that there’s a new new geek anti-intellectualism growing?

Tags: lessons
Posted in Community, blahblahblah | 2 Comments »

Honesty’s Net Worth

Wednesday, July 6th, 2011

I recently told a friend of mine I had discovered James Altucher.

He said “Oh, he’s a great read! I love reading his stuff. He’s totally crazy.”

At the time, I didn’t understand. But after reading more of Altucher’s work, I realized what he was referring to: Altucher’s frankness¹.

As I’ve continued to enjoy his work, including his new book, a question started to burn in my mind, growing with each post and chapter.

Today’s Altucher-post answered it: Do You Have To Be Rich To Be Honest?

It’s a great read. You should go read it.

Right. Now.

I had been struggling with exactly this question for some time.

I learned The Hard Way^TM that so-called open door policies may not, in fact, actually be about the open and honest communication of ideas.

As a bright-eyed, bushy-tailed build engineer², I made the mistake of taking that particular managerial concept at face value and assumed that “Please post your feedback” actually meant “We want to know what everyone is thinking” not “Everyone needs to submit their agreement in writing, so it can be later referenced, in case there is any dissent.”

And so, like a dumbass, I gave my opinion. My full and honest opinion. Sincere, too. On any issue that was asked.

I assumed that’s what they wanted; they asked for it, after all.

Right?

Of course, since that’s not what the exercise was about, that failed.

In an ill-fated attempt to “fix” it, I tried espousing the official line.

That, too, failed: as Altucher points out, people can totally tell when you’re doing that. (It turns out: both the people telling you to repeat what they’ve said, and the people listening.)

That lesson really stung.

It was probably exacerbated by the fact that I didn’t get the right take away at the time; Altucher explained today what it actually is:

If you are going to lose a job by being more honest, should you stay in that job? My guess is you’ll be a lot happier in another job. Will you lose your girlfriend if you tell her the truth? My guess is you probably would be happier with someone different if she really would leave you upon hearing the truth. Or maybe you honestly don’t know her (or him) as well as you think you do.

He’s spot on, but like so many of his posts, I think what he’s really saying is “You need to have faith.” In this case, faith in the idea that being honest to yourself and to those around you will, over the long term, work out for the better³.

Having faith is really hard work⁴. Whenever Altucher discusses it, he never for a moment implies that it isn’t arduous and difficult.

But then… I guess so is lying constantly⁵.

_______________
¹ Which, if his frankness is any indication of how he lives his life, leads—predictably—to crazy stories
² Are build engineers ever bright-eyed or bushy-tailed?
³ Probably for everyone
⁴ A lesson I’ve only recently begun exploring
⁵ Or telling everyone “Yes sir, your idea is perfect; thank you sir.”

Tags: lessons, personal
Posted in Community, blahblahblah, personal | No Comments »

Considering Crucial Criticisms

Friday, July 1st, 2011

My communication was supposedly so poor, it necessitated “gifting” me this book…

One of the worst mistakes I have ever made was letting my manager-at-the-time convince me I was a poor communicator.

Quality communication is hard, certainly¹, and we could all be better at it.

But I’d been in the workforce for some time at this point, and my communication skills had never been criticized before².

At first, I found it a little surprising, but… maybe he was right? He even gave me a book, claiming that he’d given it to many people, and it had helped them with their poor communication skills.

Not wanting to rock the boat—I had only recently started reporting to him—I decided to really examine this whole “You say stuff that sense doesn’t make”-criticism I was receiving³: I really tried to pay attention what I said and how I said it.

I proofread things three or four times⁴. I rehearsed conversations in my head endlessly, right up to the point before they were to take place. I read about these “crucial conversations” I was having, how I was clearly and obviously (I was told) screwing them up, and how I could fix it.

I didn’t end up working with him very long: it turns out you can’t really work with someone you can’t communicate with.

This individual, on the other hand, may indeed have… an “issue” with his workplace communication skills.

I spent a ton of time reflecting on how I could have done better. I reviewed so many interactions in my head, they all started to blur together. But no matter what, my lacking communication skills were to blame, right?

Obviously.

After letting it simmer for a few months, I came to an interesting conclusion: he was totally, utterly, and completely wrong.

I was able to facilitate this conclusion, in part, because over ensuing months, I’d heard story after story from others about how every single conversation with this critic-of-mine was “like a teeth-pulling trip to the dentist.”⁵

The only thing more surprising than the number of confirmations of these experiences was the viscerally painful descriptions: dragging meetings off point, taking thirty minutes to say something that could have been said in two, or just plain not grokking the words that were coming out of people’s mouths.

It was a real wakeup call for me.

Clearly, not all criticism is bad.

But when someone feels it’s their place to criticize something so fundamental to who and what you are as your ability to string words into a coherent sentence or your style of putting words on a page⁶, that authority had better be derived from something more than the workplace-equivalent of a military dictator overthrowing some small South American country.

And when someone does give you “constructive criticism” or “helpful feedback” or downright orders you to change and shows you the proverbial stick they’ll beat you with until you do, that advice had better pass your personal smell test before you take it to heart.

If it doesn’t, beware. Only if you’re lucky, like I was, do you eventually realize that smell is your soul rotting.

Ignore that acrid odor at your own peril.

_______________
¹ A good one-and-a-half serious relationships of mine have fallen apart over that very issue…
² I’d been told I could be a bit… abrasive when I was first out of college, but… that’s style, not content
³ Repeatedly, pretty much every single day; this shortly becomes relevant to our story
⁴ Mostly for content and tone
⁵ Not my words
⁶ Or screen

Tags: lessons
Posted in blahblahblah, personal | 3 Comments »