Lost Horizons of Tech

Last week I was talking to someone about the sad realities of not having access to good backups. The example I talked about was the famous “NotPetya” attack on the worldwide shipping firm, Maersk. It’s fascinating reading (WIRED wrote a great story about it here)…the upshot was Maersk’s entire worldwide network was destroyed, necessitating a complete replacement/reload of the hard drive of EVERY PC in their organization. It’s a great front-row story of the incredible sacrifices Maersk’s tech team did in order to survive a multi-million calamity that was way beyond their Disaster Recovery planning.

Bad enough, yes — but every Active Directory domain controller was also lost, with no current backups. BUT: they found ONE surviving untouched domain controller in Ghana — which, thanks to Third World power brownouts, was offline when NotPetya wormed its way through the network. The domain controller hard drive was hand-carried, like a tech relay race, from Ghana to Maersk HQ in the United Kingdom, allowing staff to rebuild the entire network, and restore thousands of logins and machine accounts, in days rather than weeks or months.

We experienced our own, less successful version: when we were adopting our child from China back in 2005-2008, it was the waning years of Apple’s web ecosystem and iWeb integration. Our blog lasted for years and was very popular among our friends. We had tons of comments from well-wishers and folks going through international adoption. We even took a laptop to China in August 2006 in order to give everyone back home daily updates, and witness how our adoption trip morphed into just a family vacation with a new baby. When Apple’s iWeb environment crashed in late 2008 and got corrupted, we had a complete backup. Or so we thought: the posted comments were only on Apple’s web, and not backed up anywhere accessible to us.

The surviving website online stretches from 2008 to 2010, and the original (2005-2008) lives on only on our Mac laptop, so our daughter can someday read it. But the comments exist only in paper form — thanks to my Mother printing off a copy! In this case, analog is our only way to preserve that data — those comments and well-wishes.

Anyone working in technology for any length of time will have a failed backup story, with that awful “pit of their stomach” feeling that accompanies. To this day I always employ a “belt and suspenders” approach to backups: not just one, but two. And sometimes three. Because you never know what physical system or link will fail at the worst possible time. At the Tulsa Area United Way, our most critical and irreplaceable data is backed up locally, and to two separate cloud repositories, with different approaches to each backup. One is an image backup, one is a database backup, and one is a VM backup stored in Azure, ready to be flipped to ON if our building were uninhabitable. Oh, and a copy of our Active Directory stored in Azure AD.

Which makes me think: we humans spend a lot of time worrying about preserving things from catastrophe or fantasize about it (popular fiction, dystopian or otherwise). It’s not just the tech types and our Business Continuity Plans. From the Global Seed Bank at Svalbard to James Hilton’s “Lost Horizon,” we consider ways to protect ourselves as a species from our baser instincts.

The book “Lost Horizon” was one of my favorites in school. I first got hooked as a child by the horrible 1973 movie version, but the underlying concept stuck. A few years back my wife gave me a hardback copy as a birthday present, which I revisit from time to time. One of the concepts in Shangri-La was that the lamasery had made it their work to collect and spirit away the world’s treasures of art, culture, and civilization to their high mountain valley — the vision of Father Perrault was that after the world’s tribulation, the true embodiment of the Christian ideal would come to pass:

              Look at the world today! Is there 
                         anything more pitiful?  What madness 
                         there is, what blindness, what 
                         unintelligent leadership! A 
                         scurrying mass of bewildered 
                         humanity crashing headlong against 
                         each other, propelled by an orgy 
                         of greed and brutality. The time 
                         must come, my friend, when this 
                         orgy will spend itself, when 
                         brutality and the lust for power 
                         must perish by its own sword. 
                         Against that time is why I avoided 
                         death and am here, and why you 
                         were brought here. For when that 
                         day comes, the world must begin to 
                         look for a new life. And it is our 
                         hope that they may find it here. 
                         For here we shall be with their 
                         books and their music and a way of 
                         life based on one simple rule: Be 
                         Kind.

I don’t know about you, but much like Conway in the book, I often feel the gentle yet constant tug towards a world that’s ruled by kindness. It’s long been a part of me and perhaps indirectly, led me to a career in technology.

A lot of the early-Internet euphoria of the 90s was grounded in the hope that connections through technology would create a more egalitarian world — where different views would be seen as admirable, and where people everywhere could work as equals. I’ve written elsewhere in this blog about the selflessness of famous developers writing software patches overnight for free, for the good of the Internet commonwealth. You may remember the hope (for some it still is) of people learning tech skills in the Third World and being able to provide for their families without having to move around the globe. A lot of the placelessness of corporations, freed from the tax strictures of home governments, stems from this period also — the natural result of money and capital wanting to be as free as photons moving down fiber optic cables.

But a high mountain valley where we can live away from the world’s problems is not real. We have to venture out daily, remembering to Be Kind. We have to do our best to protect what’s important and valuable from the rust and decay of time — all the while remembering to Be Kind. We have to protect our environment from our own baser instincts, for our children. Also by Being Kind.

When faced with cutthroat, dog-eat-dog messages of our world, it can be a hard lesson to internalize.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.