We work with a small but quite tangled system. The system itself is only a few years old but contains many technologies and bears the scars of several smart but inexperienced people learning how to design as they went. I won’t articulate the details of the bug we were experiencing but I will say that it is something that recently appeared and was troublesome but non-critical.

The difference in our two points of view became apparently very quickly. My desired approach was to try to pin down the bug, deconstruct it to the point of understanding, and then apply a fix. My coworker’s approach was to make a well-educated guess about the cause of the bug and apply a refactoring to our codebase that would address the anticipated problem.

His reasoning was that our codebase was too complex to spend time trying to understand it in detail, and in the worst-case scenario we would still have completed a useful refactoring that improved the quality of our codebase. I found this a bit hopeless. Is software really so complex that we as programmers now have to act like doctors – diagnosticians – making reasoned guesses about the world around us? Is there no more engineering?

10 or 15 years ago I would have considered his approach ridiculous. Now I’m honestly not sure. Modern software systems contain so much abstraction and layering that it is really hard to judge the level of effort that will be involved in addressing any one problem.

So what option did we choose? We guessed and did the refactoring. I don’t think I would have agreed to that with just any pair partner, but this particular coworker is a very productive programmer. He had a clear path forward and a clear rationale. And one thing you don’t do with a productive programmer is get in their way.

Did it work? No. I almost want to say “OF COURSE NOT!” but that would be unfair. It could very easily have worked. What did eventually work? A pair sitting down and tracing through the application stack with gdb. And even that has only located the approximate location of the error. We still haven’t solved the problem yet. But we’re much closer.

I’ve worked in quite a few shops over the years and I doubt any of them – including the mega corps – have scored higher than a 7. This leads me to suspect that damned few organizations actually score 11 or better, which Joel considers the minimum acceptable score.

Joel’s high standards aside, the strength of the Joel Test isn’t in the questions, it’s in the areas the questions cover. With 12 quick questions you can get a rough idea of how an outsider would view your competence in many areas of software development. Very handy.

The downside of the Joel Test – and any metric of anything, ever – is that over time people align themselves to the metric, instead of the spirit of the metric. To put it another way, the Joel Test has some leaks.

Here’s my corollary to the Joel Test. Just like the original, each is a simple yes/no question. The difference is that for each “yes” on my list, you subtract one from your Joel Score.

1. Do you use more than one kind of version control system?
2. Is there anyone on your team who can’t build in one step?
3. Are specs developed without programmer input?
4. Do you write new code the day before a deadline?
5. Does your team regularly work more than 40 hour weeks?

How did you do? Did you go negative? I hope not, but I bet a lot of shops would.

Here’s my explanation for each item:

1. Do you use more than one kind of version control system?

Do you have several older projects in cvs but everything new is in Git? Why? Don’t get me wrong – there could be a damned good reason that is tied directly to your company’s bottom line. But I kind of doubt it.

Certainly your team is full of smart people that can manage more than one vcs. That isn’t the point. The point is that it is hard to have a unified approach to problems that everyone on the team uses when you’re all using different tools. This also creates a bus factor problem when Bob – the main dev on your legacy Zapruder project – goes on vacation. After all, he’s the last one on the team that uses VSS regularly and no one can figure out what the hell brand of crack he was smoking before he left. And those Zapruder people are getting pissed!

2. Is there anyone on your team who can’t build in one step?

This goes along with the idea of everyone using the same tools. Can everyone deploy the latest rev of the Zapruder project to the dev server with one click? Including off-site developers? Your team in India? That one guy in Fresno who works remotely? Your graphic designer who wants to update some templates and see the results?

3. Are specs developed without programmer input?

When you leave a programmer out of the specification process, you get airbags on guard rails.

4. Do you write new code the day before a deadline?

I maintain that at least 80% of deadlines are macho bullshit, usually generated upstream from the software development process. Even so, deadlines happen. What you don’t want to find yourself doing is writing code for new features the day before a deadline. That’s just not responsible development. Notice how I emphasize the word “new“. Certainly if you find a bug during your final testing, you’re going to fix it. Duh. But if you’re still cranking out features the day before a deadline, how in the hell are you going to test what you’ve written? The honest answer is that you’re not. Not in any meaningful way. You’re going to cross your fingers and hope.

5. Does your team regularly work more than 40 hour weeks?

This one is straight out of the Agile playbook and revolves around technical debt. It’s also a responsibility thing. People can only do so much work in a given amount of time. That’s just a fact of life, despite what the stereotypical programmer “caffeine == code” culture says. (More macho bullshit.) There will always be special times when more work is required, but those should be exceedingly rare.

These questions only scratch the surface, but hopefully they plug a few leaks. (How’s that for a mixed metaphor?)

So there I was, riding the crest of the bubble, with dreams of startups dancing in my head, when a private list of angel investors fell into my lap. What did I do?

I didn’t have a lot of ethical options. The biggest quasi-ethical play I could have made would have been to cold-call these people and solicit them for investments. Investments in what, I didn’t know. I had nothing concrete to invest in. I wasn’t sure what I wanted to do and had no partners. In short, I had nothing to offer and was months if not years away from even thinking about funding of any kind.

There are other people in this world who can just fire up the bullshit machine and start spewing until dollars start flowing. They work out the details later. I get the sense that many dotcoms were started that way, in fact. For better or worse, that just isn’t my style. Even if that was my style, that list wasn’t mine to use. What felt very exhilarating at first just seemed depressing after a few moments of thought. Yes, I had found a valuable hidden gem, but it wasn’t mine to keep.

So what did I do? I did exactly what @AdamD said in his comment on my previous post: I sent the author an email telling him about the problem. I explained how I found it and what he should do to fix it. I almost didn’t send that email though… I was worried that the author might claim that I had hacked his site or trespassed in some way. That isn’t as far fetched as you might think: I hear that eBay will claim a form of trespass if you scrape their website for data instead of using their API. And while legal matters are pretty murky on the Internet today, they were far murkier 9 years ago. So I waited for a response, unsure of what – if anything – would come back.

A few days later I got a reply from the author, who was happy I told him about the problem, but he wasn’t nearly as worried about it as I thought he might be. After all, having an angel list isn’t nearly as valuable as having actual relationships with those angels.

As a thank-you he sent me a t-shirt from his company which I wore often while playing hockey for many years after that.

That was one of the best decisions I’ve ever made. I spent a month playing roller hockey and shooting hoops in the breezy Northern California fall. When I wasn’t doing that I was plotting my next move, which I thought might be a startup.

During one of my frequent visits to the bookstore I found a book by a well known Silicon Valley marketing guru slash venture capitalist. This was one of those business books that is light on details but somehow manages to inspire and stimulate through anecdote and analogy. I enjoyed it quite a bit and read it quickly.

At the end of each chapter the book had exercises for the reader. One of the exercises was to do research on people by using the internet. The author suggested starting off by researching him. So that’s exactly what I did.

After a few minutes of “googling my yahoo” I discovered an oddball, partially completed area of the author’s website. The full url was kind of strange, as if the page was buried in an app, but the page I was seeing clearly wasn’t part of an app. If you build web apps long enough this kind of goofiness starts to catch your eye. Feeling a little nosy, I altered the url a bit to try to figure out what kind of app was supposed to be there. That’s when I found an ugly little page with only a few links. This was clearly not part of the main site. It looked to be part of an oddball personal organizer app which had been shoehorned into the website. The kind of thing that you don’t expect to see because you’re not supposed to see it without a password. Except there was no password! This was wide open to the world, but only if you knew exactly where to look. I am not a hacker. I just played with the URL a bit. (Also: I’m not a player. I just crush a lot.)

One of the links on this page said “Angels list”.

I believe “HOLY SHIT!” was the precise phrase that came out of my mouth. You know I clicked that link, right? And sure enough, there was a big old list of names and phone numbers. I had stumbled onto the Rolodex of a well-connected VC during the Bubble. That does not happen every day.

So what did I do? Well, I’ll tell you in my next post. In the meantime, let me ask you: What would you have done?

Seriously, other than “don’t do it!”, I have very little advice. I’m still knee-deep in painful lessons. Talk to me in a few years. Maybe.

I do have something to share with you today though. This video describes behavior that EVERY SINGLE CLIENT will pull at some point in your relationship. I’ve personally had every conversation in this video. If you’ve never had to negotiate with clients before, this video is practically psychic training. Hell, you might even want to share this video with clients just to clue them in. Or weed them out. Enjoy!

For a preview of my speaking style, check out this clip of Tim Calhoun:

So. Would you like to know the single most important thing there is to know about developing software?

Are you ready?

Can you handle it??? Here it is, BAM!!!:

Manage Expectations

[Matt drops mic and walks off the stage.]

Um… that’s it. That is the whole thing, right there.

If you’re disappointed right now it is probably because I didn’t manage your expectations very well.

It’s true though. Managing expectations is more important in software than gathering requirements, analyzing problems, writing gleaming code, or shipping a high quality piece of software out the door. Those are all very important things. They just aren’t THE most important thing.

For years I’ve said that given the right circumstances I could succeed as a programmer without ever writing a single line of code, just by managing expectations. A couple of years ago I saw a programmer prove this, albeit in a rather unfortunate way. This guy was on the job for three months and hadn’t written a single line of code. And then he got fired. But he wasn’t fired for his code. He was fired due to office politics and teamwork problems. In fact, it was only discovered that he hadn’t written any code the day after he left! The best part is that before he was fired, he was being considered for a promotion to team lead!!!

Did you see the video making the rounds last week about Susan Boyle, the frumpy 47 year old lady that blew away Simon Cowell with her singing? If you haven’t seen the video, Go take a look. (Ordinarily I would embed the video but the fine folks at Youtube and/or BBC have disabled embedding on this one.)

Yes, Susan sings well. Hardly the point though. The real point is that the producers are smart! They have totally and completely managed expectations. Think about the entire set up: The first thing you see is a middle aged frump shoving a snack into her mouth. Then she talks and every word is in a charming but distracting brrrrrogue. And then mix in some well-timed editing: Simon’s unamused reaction when she walks up. The extreme-eye-rolling girl in the audience. Raised eyebrows everywhere. Finally, toss on some ham from the lady herself and you have a perfect recipe for a managed expectation. No one is going to watch all of those social cues and genuinely expect the lady to be able to sing a note. So when she can sing and sing well, POW, they got you. TV producers FTW.

You may think all I am describing is manipulation. That is not the case. Managing expectations is about empathy as much as it is about control. If you promise a feature in a new release of software and it isn’t there, that is going to be very disappointing for your stakeholders. And if you promise to ship a new version in a month and it takes you six months, that might put your stakeholders out of business if the project is critical enough. On the other hand, if you promise something in six months and it takes you four, you’ve got a couple of months to catch any problems you may have missed and you can ship early.

It is far, far better to manage expectations – including your own – and produce average results, than it is to try to produce outstanding results without managing anyone’s expectations. And many times – when you manage expectations well – your “average results” turn out to be quite above average.

Photo courtesy of Sea Turtle

I didn’t come to programming from Computer Science. I was an Electrical Engineering major who switched over to Physics and then fell into programming as a hobby which turned into a career. When I first started programming I had a very low opinion of it. I see a lot of EE types with this attitude: software is easy, and what they do is the hard stuff. Even back then I could see that the field of programming was a huge mental quagmire and I swore that I would not get caught up in it. Software is just there to make the hardware do stuff. My attitude was that the “how” doesn’t matter so much as long as it works.

Naturally that’s not what happened. I got swept up in the industry trends. I became addicted to learning what was new and cool. I was an object zealot and fanboy. Then STL. Scripting languages. ORM. Patterns. TDD. Refactoring. The past few years it’s been functional and logic programming. And that’s all great. I’ve learned a lot and had a lot of fun. But sometimes I wonder if I had just kept my head down and coded more instead of trending more, where I would be. I’m not talking about ignorance here, just pragmatism. There is a balance between the two and I wonder how far off the mark I am, or if I’m off the mark at all.

What I’m growing increasingly intolerant of is zealotry and opinionism. Both of which are almost inseparable from computer programmers. The worst is when I feel it from within. When someone shit-talks the tools I use to do my job, I get a little tight in the chest. And why? Why the hell should I? If I’m getting my work done the best way I know how and the people who use my software are happy, then what is the problem? The truth is, there isn’t a problem.

Michelangelo is said to have seen sculpting not as chiseling out an image from stone, but as releasing the sculpture that was already in the stone. That is such a powerful idea because it means that there are an unlimited number of sculptures in that stone. No two sculptors would set the same sculpture free. Programming is like that. Software is so very abstract, and the limitations on what we do are relatively few compared to other disciplines. Anything that works and makes sense to the programmer is probably not wrong in a fundamental sense. Certainly issues of specific application, maintenance and communication can be discussed, but what I’m getting at is that my methodology or toolset is not dumb just because it isn’t the one you use.

“The Tao that can be told is not the eternal Tao.”

That’s a nice way of saying that you do not have all of the answers.

I think it’s a little harder to blow off C.A.R. Hoare. Thanks Tony!

Please charge me $5 a month.

At a local tech discussion here in Portland a couple of weeks ago, one of the attendees said that if Twitter started charging money, he’d move to identi.ca. His rationale was this: software is easy to change but hardware is hard to change.

That thought has stayed in my brain for two weeks now. Is software easy to change? No. Absolutely not. That is a flawed premise. If software were easier to change than hardware, Microsoft would have crapped out a long time ago. Software being hard to change is practically the backbone of the software industry. There are entire vertical markets whose existence is predicated upon the near-impossibility of changing software.

As a recent example of how hard it can be to change software, just look at the people who got screwed by the sudden closure of I Want Sandy and Stikkit. And that was just a pair of websites that were free and hadn’t been around forever.

Certainly I can point to the series of social networking sites that I’ve used and abandoned with great ease over the years: Friendster, MySpace, Ryze, LinkedIn, and Facebook. I’m sure I’m forgetting one or two. But none of those were sites that I actually relied on. I probably invested the most time in MySpace (many years ago), but even after many, many years on that site there is nothing there that I can’t do without. If MySpace burned down tonight it wouldn’t phase me at all.

If Twitter burned down tonight, I’d go nuts.

Twitter has this network of people that I’m plugged into. Unlike other social networks, I get genuine value every day from Twitter. Sure, I could try to replicate that network on indenti.ca. But you know what I’d end up with? A network of only the alpha-geeks who bothered to switch over to identi.ca! Well that’s fine, but only to a point. I like alpha-geeks as much as the next alpha-geek. But those aren’t the only people I want to be connected to. And I sure as hell don’t want to have to straddle services like I already do with IM.

I’m afraid that if Twitter doesn’t figure out a good (read: non-evil) way to make money, that they’ll either experience the ultimate Fail Whale (run out of cash and close) or turn to the Dark Side (and ruin Twitter). Either way, I’d be stuck without this cool network of people.

So, Twitter, if you’re listening (and hopefully you’re too busy improving your service to be reading crap like this), please charge me $5 a month. Trust me on this. You don’t need to do any market research here. I am a user asking you to charge me. That is a sales person’s wet dream!

$5 is the limit though – you can’t get away with $10 a month. That would piss off too many people and then they really would leave for indenti.ca and parts unknown. And no, you can’t charge micropayments per tweet. If you charge me 2-cents per tweet, I’ll probably leave too. Pay-per-tweet is a stupid idea and I wish people would quit suggesting it. You don’t want to punish the people who create value on Twitter. A flat-fee is simple and fair.

Certainly let people have a free account if they want to receive Tweets. And let them DM their friends from their free account. Why not? That would be great. But please charge those of us who want to talk. Get some cash in your door before you run out of whatever fickle VC steam you’ve got and we lose you entirely. In exchange, as a paid subscriber I will receive no ads in my Twitter stream. (Or anywhere else!) And you’ll make twit-spam nearly impossible, since other users won’t be able to talk to me without my permission. You’ll effectively cut off the MLM and Get-Rich-Quick idiots who want to put me in their downline or have me put water in my gas tank.

Then you can start adding some more features to the cool system you’ve created.

Thanks for listening,

@built