Update: The newest installment in this series covers the topic of clustering.

Three thoughts have encouraged me to contribute my perspective on developing reliable, scalable and secure software.

Thought #1

When I was in graduate school one of my professors told me “If someone comes to you claiming to know how to build reliable, scalable and secure software, run away.”

Thought #2

I’ve observed during my 10+ years in the software industry that there are any number of products on the market that claim to be platforms which magically allow software applications to scale.

Thought #3

A good friend of mine if taking a class teaching distributed computing concepts. He showed me his first assignment and asked “Is this course going to get harder? This stuff seems so easy.”

Having taken the same course myself (and having gotten an A) I know exactly why my friend had the perspective he did given that the course was actually quite a tough one. The techniques which make software reliable, scalable and secure seem simple on the surface. But when you take those ideas apart and look deeper to understand the foundations on which those techniques are based, things get very complicated very quickly.

My 10+ years of experience with software has led me to this thesis:

Scalability, reliability and security are indispensable, elusive and are in fact extremely difficult to achieve. They cannot be purchased in shrink-wrapped packaging and there are no one-size-fits-all or “magic bullet” techniques that can be applied to every problem to provide these properties. Each system must be looked at carefully and in the context of the other applications with which it interacts, the hardware and networks on which it runs and the use cases to which it will be subjected. Only after this careful analysis is underway can you begin to understand which of the many available techniques are appropriate for the system at hand.

So, in a series of upcoming posts I’ll be covering a number of techniques commonly discussed in the industry in simple terms as tools for achieving reliability, scalability or security. I’ll present the typical over-simplification and outline in detail some of the complexity that makes it so difficult to achieve something as essential and commonly discussed as building robust software.

Final Thought

I encourage anyone and everyone to read the upcoming posts in this series and to do so with their critical-thinking hat on. I also encourage all of my readers to share their experience by joining the discussion on each post. I’m always interested in refining the techniques I have at my disposal. Likewise, I always delight in a story about a system or situation that went horribly wrong. So please, join the discussion.

If you have any techniques that you specifically would like me to cover, please let me know either by commenting on this post or by sending me an email at blog@evanscode.com.

I look forward to hearing from all of you.

And now, onto the topics:

• Part One: Clustering