Last week I was talking to someone about the sad realities of not having access to good backups. The example I talked about was the famous “NotPetya” attack on the worldwide shipping firm, Maersk. It’s fascinating reading (WIRED wrote a great story about it here)…the upshot was Maersk’s entire worldwide network was destroyed, necessitating a complete replacement/reload of the hard drive of EVERY PC in their organization. It’s a great front-row story of the incredible sacrifices Maersk’s tech team did in order to survive a multi-million calamity that was way beyond their Disaster Recovery planning.
Bad enough, yes — but every Active Directory domain controller was also lost, with no current backups. BUT: they found ONE surviving untouched domain controller in Ghana — which, thanks to Third World power brownouts, was offline when NotPetya wormed its way through the network. The domain controller hard drive was hand-carried, like a tech relay race, from Ghana to Maersk HQ in the United Kingdom, allowing staff to rebuild the entire network, and restore thousands of logins and machine accounts, in days rather than weeks or months.
We experienced our own, less successful version: when we were adopting our child from China back in 2005-2008, it was the waning years of Apple’s web ecosystem and iWeb integration. Our blog lasted for years and was very popular among our friends. We had tons of comments from well-wishers and folks going through international adoption. We even took a laptop to China in August 2006 in order to give everyone back home daily updates, and witness how our adoption trip morphed into just a family vacation with a new baby. When Apple’s iWeb environment crashed in late 2008 and got corrupted, we had a complete backup. Or so we thought: the posted comments were only on Apple’s web, and not backed up anywhere accessible to us.
The surviving website online stretches from 2008 to 2010, and the original (2005-2008) lives on only on our Mac laptop, so our daughter can someday read it. But the comments exist only in paper form — thanks to my Mother printing off a copy! In this case, analog is our only way to preserve that data — those comments and well-wishes.
Anyone working in technology for any length of time will have a failed backup story, with that awful “pit of their stomach” feeling that accompanies. To this day I always employ a “belt and suspenders” approach to backups: not just one, but two. And sometimes three. Because you never know what physical system or link will fail at the worst possible time. At the Tulsa Area United Way, our most critical and irreplaceable data is backed up locally, and to two separate cloud repositories, with different approaches to each backup. One is an image backup, one is a database backup, and one is a VM backup stored in Azure, ready to be flipped to ON if our building were uninhabitable. Oh, and a copy of our Active Directory stored in Azure AD.
Which makes me think: we humans spend a lot of time worrying about preserving things from catastrophe or fantasize about it (popular fiction, dystopian or otherwise). It’s not just the tech types and our Business Continuity Plans. From the Global Seed Bank at Svalbard to James Hilton’s “Lost Horizon,” we consider ways to protect ourselves as a species from our baser instincts.
The book “Lost Horizon” was one of my favorites in school. I first got hooked as a child by the horrible 1973 movie version, but the underlying concept stuck. A few years back my wife gave me a hardback copy as a birthday present, which I revisit from time to time. One of the concepts in Shangri-La was that the lamasery had made it their work to collect and spirit away the world’s treasures of art, culture, and civilization to their high mountain valley — the vision of Father Perrault was that after the world’s tribulation, the true embodiment of the Christian ideal would come to pass:
Look at the world today! Is there
anything more pitiful? What madness
there is, what blindness, what
unintelligent leadership! A
scurrying mass of bewildered
humanity crashing headlong against
each other, propelled by an orgy
of greed and brutality. The time
must come, my friend, when this
orgy will spend itself, when
brutality and the lust for power
must perish by its own sword.
Against that time is why I avoided
death and am here, and why you
were brought here. For when that
day comes, the world must begin to
look for a new life. And it is our
hope that they may find it here.
For here we shall be with their
books and their music and a way of
life based on one simple rule: Be
Kind.
I don’t know about you, but much like Conway in the book, I often feel the gentle yet constant tug towards a world that’s ruled by kindness. It’s long been a part of me and perhaps indirectly, led me to a career in technology.
A lot of the early-Internet euphoria of the 90s was grounded in the hope that connections through technology would create a more egalitarian world — where different views would be seen as admirable, and where people everywhere could work as equals. I’ve written elsewhere in this blog about the selflessness of famous developers writing software patches overnight for free, for the good of the Internet commonwealth. You may remember the hope (for some it still is) of people learning tech skills in the Third World and being able to provide for their families without having to move around the globe. A lot of the placelessness of corporations, freed from the tax strictures of home governments, stems from this period also — the natural result of money and capital wanting to be as free as photons moving down fiber optic cables.
But a high mountain valley where we can live away from the world’s problems is not real. We have to venture out daily, remembering to Be Kind. We have to do our best to protect what’s important and valuable from the rust and decay of time — all the while remembering to Be Kind. We have to protect our environment from our own baser instincts, for our children. Also by Being Kind.
When faced with cutthroat, dog-eat-dog messages of our world, it can be a hard lesson to internalize.
