Welcome to All Cloud, No Cattle Weekly #21.
Tech
Thundering herds, noisy neighbours, and retry storms
Mads Hartmann:
I love the names that people have come up with over the years. Some of them describe observed patterns, as Lorin Hochstein so eloquently put it “Operators give names to recurring patterns of system behavior that they observe” (tweet), others describe techniques used to mitigate these observed patterns.
Mads has put together a great list of operational patterns, their causes, and some mitigation tactics.
The Downtime Project
…a podcast where the hosts dissect incident reports. You son of a bitch, I’m in.
You might as well timestamp it
Jerod Santo:
Storing timestamps instead of booleans, however, is one of those things I can go out on a limb and say it doesn’t really depend all that much. You might as well timestamp it. There are plenty of times in my career when I’ve stored a boolean and later wished I’d had a timestamp. There are zero times when I’ve stored a timestamp and regretted that decision.
I had to think about this a bit, but he’s right. Every time.
How to Connect to Private EC2 Instances without an AWS Bastion Host
Chris Blackden:
But wait, do you actually need a bastion host? Nope. In fact, you can use AWS Systems Manager (SSM) to take the place of a bastion host instance. You can then use the AWS CLI to connect to fleets of EC2 instances without exposing another host to the Internet!
I had no idea you could do this. Definitely making this standard part of my arsenal going forward.
Simple bank shutdown goes awry leaving customers without account access
Jacob Kastrenakes at The Verge:
The online bank Simple shut down on Saturday and was supposed to seamlessly transition customers’ accounts over to its parent company, BBVA. But instead, many users found themselves unable to access their bank account at all, as BBVA’s website returned an assortment of error messages, from “system error” to warnings that their account information was mismatched.
I would love to be a fly on the wall for this post-mortem. How do you not transition customers slowly over time?
Developing on Apple M1 Silicon with Virtual Environments
John Rofrano:
This means that Vagrant can control the provisioning of Docker containers just like it controls VirtualBox for provisioning virtual machines. This is somewhat of a unique use case for Docker because the intent of Docker is to provide a consistent, immutable runtime environment; not to be treated like a virtual machine. As with all technology, there are always use cases beyond the original intent and I was about to learn if this use case was viable.
It might surprise some to know that I’m not a fan of developing in containers, but this is a pretty novel setup. I still prefer to dev in my local OS, though. Trying to dev inside containers continues to feel like too much effort for too little benefit.
Grab Bag
Tetris for Game Boy Gets Online Multiplayer
Tom Nardi on Hackaday:
As explained in the video below, the adapter is essentially just a Raspberry Pi Pico paired with some level shifters so that it can talk to the Game Boy’s link port. That said, the custom PCB does implement some very clever edge connectors that let you plug it right into the Link Cable for the original “brick” Game Boy as well as the later Color and Advance variants. This keeps you from having to cut up a Link Cable just to get a male end, which is what stacksmashing had to do during the prototyping phase.
This is straight incredible.