Read my latest article: 8 things I look for in a Ruby on Rails app (posted Thu, 06 Jul 2017 16:59:00 GMT)

Lessons through failure. Episode 1

Posted by Sat, 17 Jan 2009 22:34:00 GMT

I fucked up this last week.

On Monday, our primary contact for a large client sent over some last minute requirements and deadlines that were needed by end-of-day Wednesday. I didn’t have a lot of time to collect requirements and execute it without having to rearrange my priorities. But, I accepted the challenge.

The big change involved was that we were going to be supplied with a ton of data to be imported in to the database and approximately 20% of the data provided was new records, while the rest were duplicates. However, the other 80% wasn’t to be discarded as there were a few attributes that needed to be updated from the data file (which was supplied from the client’s parent company). In my haste to get the task done on time (didn’t get proper export file to be imported in our system until Wednesday morning)... I ended up running a few tests locally and pushed it out to production.

I managed to get the import file to run in production before leaving on Wednesday afternoon. The following morning, I came into the office to find out that my import process didn’t match up records properly and resulted in nearly all of the 80% side of that to be duplicated in the system. This resulted in lost productivity for our client, their vendors, and our team over a 12 hour period as people were confused about why reports were running weird, online transactions didn’t account for the duplicated, etc.

It took me most of Thursday and Friday to clean up the data that got skewed due to that oversight. Hi ho.

So, the take away from this? Sure, I could have blamed it on a lack of sufficient time to properly test things, but that’s bullshit. I should have had at least one other developer from our team review the problem and evaluate my proposed solution prior to me attempting to push into production.

Luckily, the client was happy that we were able to finish the last minute tasks, despite the unexpected headaches that cropped up.

If anything, I was just disappointed in myself, but Alex reminded me how important it was to fail early, fail often. It didn’t kill me (or anybody else for that matter), cost us the project, nor was it irreparable.

In the real world, deadlines and requirements change on a moments notice and it’s experiences like this that will make ourselves more confident that we can quickly respond to and execute.

What was your latest failure?

iPhone's Missing Feature

Posted by Thu, 03 Apr 2008 23:17:00 GMT

I love my iPhone. There… I said it.

There is one problem with it (and I don’t blame Apple for it). Multimedia Messaging is pretty much non-existent with non-iPhone friends & family. My family continues to send me pictures/videos via text message… because they can with their other friends and family. I find myself looking at the following message quite often.

Uploaded with plasq’s Skitch!

Then I go to AT&T’s AMAZING site…

viewmymessage is shit
Uploaded with plasq’s Skitch!

...and I get a broken image.

Great work AT&T! Apple… this is tainting my experience with the iPhone. Make them fix it!

Embracing Failure, part 1

Posted by Tue, 10 Apr 2007 20:38:00 GMT

I’m currently reading To Engineer is Human, by Henry Petroski and found the following applicable to software development and managing client and customer expectations.

“As much as it is human to make mistakes, it is also human to want to avoid them. Murphy’s Law, holding that anything that can go wrong will, is not a law of nature but a joke. All the light bulbs that last until we tire of the lamp, all the shoelaces that outlast their shoes, all the automobiles that give trouble-free service until they are traded in have the last laugh on Murphy. Just as he will not outlive his law, so nothing manufactured can be or is expected to last forever. Once we recognize this elementary fact, the possibility of a machine or a building being as near to perfect for its designed lifetime as its creators may strive to be for theirs is not only a realistic goal but also a reasonable expectation for consumers. It is only when we set ourselves such an unrealistic goal as buying a shoelace that will never break, inventing a perpetual motion machine, or building a vehicle that will never break down that we appear to be fools and not rational beings.”

I’m sure that most of us are guilty of having high expectations for products that we purchased. (why does my ipod screen scratch so easily when in my pocket?) We also set high expectations for the code that we develop, which is why we (hopefully) continue to refine our process. We’re bound to time and budget constraints, which often prevent us from testing every imaginable edge case. Given our constraints, problems are almost always going to arise. It’s no wonder that we see Test-Driven Development as an important part of a healthy development process. We want to catch our failures as early as possible.

Our clients often have high expectations and it’s almost always very reasonable. That’s not to say that some clients will not have highly irrational expectations. It’s our job to manage these expectations as best as possible.

Do we mislead our clients by convincing them that our TDD/BDD process is going to prevent any bugs from creeping from the woodwork after the development cycle is finished?

“I thought that we paid you to fully test the code?”

Really… is that even possible? Can we predict (and test) every possible interaction within an application? Highly unlikely.

What we can do is plan for and embrace failure. We can help our clients understand that almost every application needs to be maintained after it’s initial development cycle. Bugs are inevitable and there needs to be a clear process for handling them.

Perhaps I’m abusing the bug fixing process by calling it a failure… but I’ve also found that yes… many bugs are due to failure. Whether that be a failure to specify application behavior, a failure to understand the project goals, a failure in communication, ...or maybe a failure in our software architecture. We’re constantly failing.. and it’s okay!

IT’S OKAY TO FAIL! (some of the time…)

“No one wants to learn by mistakes, but we cannot learn enough from successes to go beyond the state of the art. Contrary to their popular characterization as intellectual conservatives, engineers are really among the avant-garde. They are constantly seeking to employ new concepts to reduce the weigh and thus the cost of their structures, and they are constantly striving to do more with less so the resulting structure represents an efficient use of materials. The engineer always believes he is trying something without error, but the truth of the matter is that each new structure can be a new trial. In the meantime the layman, whose spokesman is often a poet or writer, can be threatened by both the failures and the successes. Such is the nature not only of science and engineering, but of all human endeavors.”

As we’re creating these virtual structures… are we really taking the time to reflect on our failures? This is why some teams adopt practices like iteration retrospectives and post-mortems.

I’ll end this with a few questions, which I hope that you’ll share your experiences about…

  • In what ways is your team embracing the failures of your development projects?
  • How do you help manage your clients expectations… so that they too can plan for and embrace failure? Isn’t their new business venture on the web… likely to experience some failure?

We have so much to learn…