Welcome to All Cloud, No Cattle Weekly #20.
Tech
HAProxy Forwards Over 2 Million HTTP Requests per Second on a Single Arm-based AWS Graviton2 Instance
Willy Tarreau at HAProxy:
Yes, you’ve read the title of this blog post right. HAProxy version 2.3, when tested on Arm-based AWS Graviton2 instances, reaches 2.04-million requests per second!
HAProxy 2.4, which is still under development, surpasses this, reaching between 2.07 and 2.08 million requests per second.
This is just downright insane.
Migrating Millions of Concurrent Websockets to Envoy
The Slack Engineering blog:
While we have been using HAproxy since the beginning of Slack and knew how to operate it at scale, there were some operational challenges that made us consider alternatives, like Envoy Proxy.
The Slack team does a great job outlining the challenges they encountered with their original HAProxy setup, the solutions offered by replacing it with envoy, and the processes needed to make the handover. Great work.
Flipr: Making Changes Quickly and Safely at Scale
Andy Maule at Uber:
Uber’s many software systems require a high volume of changes every day. Because of our systems’ size and complexity, it is a significant challenge to implement these changes without unintended consequences, ultimately slowing down developer productivity. Flipr is a big part of Uber’s solution to solving this problem. Flipr is a tool that we created for dynamic configuration management, such as feature flags, allowlists, incremental rollout, and other advanced use cases.
I first used feature flagging and dynamic config at Bypass, where we had a very similar system that we called Flippy. This is really turning out to be table stakes in any sufficiently advanced system these days.
Disasters I’ve seen in a microservices world
João Alves:
Most engineers forgot, though, that while solving an organizational problem at the software architecture’s level, they also introduced a lot of complexity. The distributed systems fallacies became more and more evident and quickly were a headache for those teams. Even for companies that were already doing client/server architectures where they already existed, this exploded in their faces once they had 10+ moving pieces in their systems.
Distributed systems are hard, whether they’re microservices or just distributed monoliths. João’s list of disasters here is a great discussion about the pitfalls of various approaches.
Management
Executives don’t decide if the company culture is good. Employees do.
Charlie Warzel:
There’s the glossy, official, Comms Department-approved culture — and then there’s the real, lived experience of showing up every day and working at a place. If the difference between those two versions is large enough, the result is generally serious, sustained, employee-management resentment. Let’s call that “culture gap.”
Spot on analysis by Warzel, with a deep dive into the deeper meaning behind blowups like Basecamp and Coinbase.
Grab Bag
Webcurios is back
Webcurios is an on-again-off-again collection of links dating back ages, and yet another inspiration for ACNC.
Why does HTML think “chucknorris” is a color?
…the principle of most astonishment?