The Art of Large Systems

I was discussing art with my daughter, someone who is extremely talented at what most people consider art.  That is, being able to draw and paint things that look amazing and you know you could never do yourself, not in a million years.

In any case, she made the comment that all artists hate their work.  This is an intriguing statement because if it were true than there is no incentive to actually make art.  If you know you’ll hate it what’s the point.  With further questioning we clarified that the statement really means an artist is never happy with the outcome.  This is far more reasonable, nothing ever turns out like the perfect image you have in your head for how something should be.  Try as you might, you know if you could just figure out how to get there, the piece would be infinitely better.

Now this makes sense to those of us who work on large dynamic constantly changing systems.  Systems that are constantly under varying stressors such as high volumes of traffic appearing in ways that were not anticipated, frequent code releases that can cause unintended consequences, and the connection to numerous other systems that often go awry.

All of these things are done in code, and we all know that if we could just puzzle it out, there’s a better way to construct this system; something which is eminently simple.  Maybe we’re using Tinker Toys when we should be using Lincoln Logs.  There’s a seismic shift that can happen if we could just force our brains to make a jump, we can often feel it out there waiting to be discovered.   But the reality of having a job and a deadline kick in and we have to deliver something that works.  Perfection not achieved, again.

When writing code you get used to this feeling because you are delivering every day, often on a long timeline and things just have to get done, sometimes badly.  We’re not happy about it but the world moves on.

Looking back at the large systems you’ve built you can always point out the things you wish you could change.  Sometimes you get the opportunity to refactor them, possibly finding out the idealized new architecture actually was worse than the original system.  Sometimes your amazing ideas fall down when confronted with the complexity inherent in large systems.  However, sometimes the new system is fantastic, there’s just some things that could still be better…

It is hard to love the outcome, it’s the child that rebelled and ran away from home after stealing all your money and taking your car.

Creating large systems is essentially a form of art.  There’s no defined methods to ensure a positive outcome.  For works like, the work is in the public domain and constantly being judged by individuals and the media.  Some people love it, some hate it, but everyone has an opinion.  And, finally, you can sell it (well technically the company could sell it).