Crash proofing your technology ecosystem: 7 ways to prevent single points of failure.
What would you do if a social media failure prevented you from logging into your work apps?
On October 4th, 2021, Instagram and Facebook had a significant worldwide outage. The easiest way to describe what happened to Facebook, Instagram, and What's App is if the internet was the world, and every webpage had a home address, then something went around and removed the address plate from their web page's house (Source).
I originally started this blog post regarding how nice it was to be unplugged from Facebook for a few hours. The proverbial "When the fishermen can't go out to sea, they mend their nets" post. However, as I started to think more about this, I realized what a more extensive problem this posed for everybody. How many logins do you have that require Facebook authentication (IE, you log in via Facebook)? In engineering, we call this a single point of failure in a system. That is, if this one critical component went down, the whole show would be down.
A single point of failure is a significant design flaw that is sometimes unavoidable. You can lose power in several electrical outlets in your own home and still charge your phone in the working ones. If the garbage truck severs your electrical lines outside, then you lose power to the whole house. The external power cable is a single point of failure.
Apple also has a single point of failure; itself. You are most likely locked into the Apple ecosystem of apps, downloads, texts, emails, and streaming services if you have an iPhone. Migrating from apple to android is a daunting task and something that both companies specifically make difficult. There is a financial burden of purchasing your apps all over again for a different platform that is part of the inherent migration problem. That's the point; these companies want to lock you into their platforms and make it difficult for you to migrate away.
I am asking you to think about what single points of failure exist in your technology setups. How can you mitigate your single point of failure? Let's look at some common things that we can do to mitigate some single points of failure.
1. One Company Ecosystems.
Evaluate your digital ecosystem. Are you locked into apple or android? What happens when you forget your Apple ID password and have to re-set it across every device? How can you diversify some devices? What devices are the best at their jobs? I, for example, think mac laptops have better reliability for live performance. However, my production computer is a PC. I use Ableton because it's cross-compatible and can work on Mac and PC. I can load up Ableton on a PC laptop and perform without much disruption if I need to.
2. Facebook Authentication.
Too many apps require Facebook authentication. Yes, it's easy because one click will allow you to log in. But what happens when Facebook fails? Upon reviewing my Facebook, I have 22 apps. My goal is to change those to email logins. Email logins are a bit more robust because it doesn't require the website to authenticate through Facebook. The site holds onto the login information in its internal database. I tend to opt for email logins and put the login and password in a spreadsheet.
3. Old Hardware.
Apple only supports new hardware for 5-7 years. After that, the hardware is no longer supported. If the equipment breaks, an Apple tech support person wouldn't even look at the machine. Apple would recommend you purchase a new device. How would you keep going with minimal interruption to your workflow? Do you have a backup machine? I try to replace my upgrade the hardware on my production PC about every two years. I might replace a video card in year 2, hard drive year 4, motherboard and RAM year 6, power supply year 8. It has been a successful rotation. Laptops, however, are a bit more daunting as hardware is less upgradeable. I also tend to keep my laptops in the Apple ecosphere. In my experience, Apple laptops do have extended lifetimes. They are well built and don't tend to fail as often. I have kept my laptops as-is for about 5-7 years.
4. Internet Outage.
We recently lost our home WIFI because a garbage truck cut the lines on the road. We live in a cellular dead zone, so my family relies on that internet fiber cable. We had to rely on our cellphone data which was severely lacking in terms of speed and reception. The slow data speeds turned out to be a challenge for today's internet-hungry devices. However, it was manageable to use cellular hotspots for most web browsing and some streaming. The biggest issue was Dropbox. Pulling files off of my Dropbox account was painfully slow for cellphone hotspots.
5. Power Loss.
What's your plan if you lose power? Are your computers on backup battery power? Is it essential to save your session and shut down the computer properly in the case of a power failure? My production PC connects to a UPS power supply that has a 5-minute battery. This UPS allows me enough time to save my session and shut down my computer safely.
6. Data Backup.
Power failures and internet failures happen a lot more frequently than you might think. What is your strategy for backing up your important data? I would suggest you have two forms of backup. The first is a cloud backup. Most of my documents and data files back up to a Dropbox account. Dropbox is great because it allows for multiple machine access. I use a Dropbox folder to transfer files from my PC to Mac. It's much easier than setting up an FTP between both machines. Secondly, I back up my whole computer on an external USB Hard Drive once a month. In the event of a crash, the most data I would lose is 30 days' worth. A USB hard drive allows minimum disruption and gives my files a physical backup if Dropbox doesn't work or my internet is down for an extended period.
7. Cable FailuresCables don't often fail. The connectors fail more than the cables, but failures depend on two factors.
The first is how many times the connector is used (plugged-in and unplugged). The connector itself often doesn't fail, but the strain relief and the cable around the connection is the most significant point of failure. Why? Because often, we pull the cable instead of the connector. The constant pulling on the cable leads to fraying around the connector itself.
Secondly, cables in high traffic areas will have more wear and tear. If you have the cord on the floor, you can crush the cable easily. A pinch in the cable will slow down electron transmission over time.
The best way to mitigate your cable failures is to have 1-2 spare cables per type. This strategy will allow you to swap out a cable quickly and keep on with your workflow.
Most cable failures happen where the connector and the cable meet.
In broadcast engineering, we design most systems with redundancy. Redundancy doesn't mean that we have two of something, but it does mean that some equipment has a dual function. Most good companies have a plan on how to work around each piece of equipment in case of failure. Redundancy requires a healthy working knowledge of your technology ecosystem. If you are unsure how a piece of equipment in your home network or WIFI, entertainment center, or other parts of your home technology setup works, now would be a good time to brush up on your working knowledge. Good working knowledge will help you troubleshoot failures more efficiently.
Dr. Mike Testa
