DevOps in the 1990s
Last week, I had the pleasure of attending LinuxCon/ContainerCon Europe 2016 in Berlin. Besides visiting a fascinating historical capital - there is great irony, and victory, in seeing "Ben-Gurion-Strasse" - or "Ben Gurion Street" - named after the founding Prime Minister of Israel in the erstwhile capital of the Third Reich. And while I had many a hesitation about visiting, the amount of awareness, monuments and memorials to the activities of the regime in the 1930s and 1940s was impressive.
Being a technologist with a love of operations and improving them, I had many conversations with very smart people people about operations and especially DevOps.
While engaged in one of these conversations, I suddenly had a realization: we were doing DevOps in 1995!
Morgan Stanley IT of the 1990s, for which I have many fond memories, and to which I am grateful (especially Rochelle who convinced me I would be insane to accept the other offers on the table... she was right), was a very cutting-edge, experimental place. We took risks, failed at some of them, and succeeded at others.
Our leaders rewarded risk-taking. Fail reasonably, get rewarded; succeed, get rewarded more. Do something really stupid, of course, let alone the same stupid thing twice, and you paid for it.
How did we do DevOps?
There are lots of definitions of DevOps. One of the premiere practitioners, Jez Humble, is quoted as saying it is:
a cross-disciplinary community of practice dedicated to the study of building, evolving and operating rapidly-changing resilient systems at scale.
Nowadays, that often involves several key elements, including:
- Developers taking ownership of the entire application lifecycle, right down to pushing the button to deploy to production;
- Automation to enable developers to deploy to production safely;
- Automated testing systems - continuous integration pipeline - to guarantee full testing before something goes out to production.
The key element here, though, is the culture. It is one where everyone, not just the operations teams, is responsible for live systems. Developers, the people who actually build the software systems, are the ones to make them live, and they are the ones to be alerted, whether at 2pm or 2am, when something goes wrong.
While we did not have all of the elements, we had two very important ones.
- Every developer was responsible for his or her application from building it right through to pushing to production and making it live.
- Automation to enable that process.
Twenty years ago, we built a system that told developers, "you own it, you push it, you fix it."
I wish I could tell you it was my brainchild, or that I was the chief architect. Unfortunately, I was neither. I just played my part. The real architect was Phil Moore.
Here was how it worked.
We had a single, global filesystem namespace. Anywhere you went in the world, if you were connected to the Morgan Stanley network, "/root/whatever" pointed to the exact same thing. And if you went to "/root/dist/something", well, that something was a local replica, right here in your city (or even building) of a read-only application.
What was a developer's workflow?
Developers built apps in "/root/dev/something", where something was unique to their app. They built it in a version directory, e.g. "/root/dev/something/1.2.3/".
When a developer was convinced it was ready, they'd run a command called "vms" (no relation to DEC VMS), that pushed that version 1.2.3 of something out... everywhere. Every single location in the world now had a local, replicated identical copy of something.
Here was the best part. When ready, the developer could run a different "vms" command to make version 1.2.3 live. From that moment (well, it took a few moments to propagate worldwide), anyone launching something would get that version. Not operations, not some release manager, but the developer who owned it.
Did it break stuff? All the time. I lost track of the number of things I broke myself (often saved by others). But we also delivered stuff at a pace unheard of at the time... and it showed.
We did DevOps at Morgan Stanley 15-20 years before it became popular. And I only realized that what we were doing was DevOps, way ahead of its time... last week!