John Hyland

“For Graeber, bullshit jobs carry with them a moral imperative: ‘If you’re not busy all the time doing something, anything—doesn’t really matter what it is—you’re a bad person.’ But the flipside of that logic seems to be: if you actually like doing X activity, if it is valuable, meaningful, and carries intrinsic rewards for you, it is wrong for you to expect to be paid (well) for it; you should give it freely, even (especially) if by doing so you are allowing others to profit. In other words, we’ll make a living from you doing what you love (for free), but we’ll keep you in check by making sure you have to make a living doing what you hate.” — Geoff Shullenberger, a writing professor at New York University I couldn't help thinking that here, the professor is describing, how “open source software” thing works. And then, I found: “The free software industry, perversely enough, has become a paradigm in this respect... software engineering work was divided between the interesting and challenging work of developing core technologies, and the tedious labor of ‘applying duct tape’ to allow different core technologies to work together, because the designers had never bothered to think about their compatibility. ... increasingly, open source means that all the really engaging tasks are done for free.” — David Graeber, Bullshit Jobs

1

I've been an engineer on call in both sprawling enterprise IT settings as well as narrower settings such as running Apache Pulsar and BookKeeper at scale. What I'm reading about wide events, canonical log lines and what Charity Majors is calling Observability 2.0 has me saying YES! But... Yes, wide events and canonical log lines are obviously a great user experience for the on-call engineer under pressure. I worked at Splunk for a while, I've seen what a powerful slice and dice machine over structured logs can do. But it is expensive! What I haven't seen yet is a good discussion of infrastructure costs. How this can be done without exploding costs? Have we designed observability data systems capable of storing huge amounts of structured wide events, with low latency queries and reasonable costs? I'd love to see a paper or a blog post or something that explores wide events from an infrastructure cost perspective, discussing the design space, factors such as scale, cardinality, query patterns, and all the associated trade-offs.

49

Excellent article. I agree with the two areas that continue to encounter the most resistance, Continuous Integration (meaning actually continuously integrating via trunk based development) and Whole Team collaboration. Whole Team methods continue to be challenged by the desire for high resource efficacy/utilization.

8

2 Comments

Vital they are to executing ideas, but not to creating them “The most misguided attempt at false collectivization is the current attempt to see the group as a creative vehicle. Can it be? People very rarely think in groups; they talk together, they exchange information, they adjudicate, they make compromises. But they do not think; they do not create. Group advocates would agree that this has been so. But they do not see this as a natural limitation. To them, it is a bug of human relations to be cured, and in the expectation that technique is the key, they are engaged in a wholesale effort to tame the arts of discovery—and those by nature suited for it. In part this effort is propelled by the natural distaste of the noncreative man for the creative, but again, there is the moral impulse. Among many there is a real belief that we can teach the individual to create in concert rather than as an individual and that his acceptance of the organization way will produce a combustion of ideas otherwise impossible. Here would be the ultimate victory of the administrator. The creative individual he does not understand, nor does he understand the conditions of creativity. The messiness of intuition, the aimless thoughts, the unpractical questions—all these things that are so often the companion to discovery are anathema to the world of the administrator. Order, objective goals, agreement—these are his desiderata. Vital they are to executing ideas, but not to creating them.” — William H. Whyte, The Organization Man

1

1 Comment

There’s this persistent concern in the industry I've run across over my career: while it’s fine to automatically deploy to pre-production environments, production deployments should always be manual. I get it—many bright professionals who’ve been burned by bad deployments feel this way. I know how crucial stable infrastructure is—my sleep depends on it too. But here’s why you should deploy with CI/CD. Yes, even to production. 1. Practice Makes Perfect:  Automation codifies your team’s collective knowledge, running consistent processes through all environments. This isn’t just for pre-production—it’s crucial for production too. The errors you come across in lower environments are fixed as you move up through the environments. 2. Reducing Human Errors: When you manually skip the CI/CD trigger, you risk missing critical steps. Have you ever forgotten to generate API documentation or update config files? CI/CD ensures nothing is overlooked. 3. Building Confidence: If you’re still concerned, consider starting with a dry-run job. It shows what changes would be made without actually deploying, giving you a confidence boost. You can also implement a manual approval step for production, where everything is prepared, but the final action requires your green light. The goal? To make full automation so reliable that you won’t need that manual approval. Run it this way until you’re confident your process works—then let CI/CD do what it’s designed to do: make deployments safer, more reliable, and stress-free.

7

1 Comment

Yesterday was National TV Dinner Day, and during a conversation with Christian Hernandez, we stumbled upon a funny (but true) realization: frozen TV dinners are a lot like upstream open-source dependencies. 🍽️💻 ° There’s always a lot of choices—like the countless frozen meals in the freezer aisle, open-source packages are everywhere. But not all are created equal. Some are just... not great—ever picked a dependency that looked good on paper but turned out to be a buggy, poorly maintained mess? Yeah, like that frozen dinner that tastes like cardboard. 🤦‍♂️ ° Within a given box, there’s always something you don’t love—maybe it’s the unnecessary features, the complexity you don’t need, or the way the package interacts with your stack. It’s like when your frozen meal comes with mushy veggies you can’t toss without breaking the meal. ° It’ll keep you relatively satisfied—these packages do their job, and you can get your project up and running. Not fancy, not perfect, but it works... for now. 🍴 ° But it could also make you sick—the real danger is the unknown. A poorly maintained dependency could introduce security vulnerabilities, create performance issues, or break down with an unexpected update. One small glitch, and suddenly your whole system feels like it’s got food poisoning. 😷 And here’s the kicker: ° You seldom find pickles in the box—frozen dinners (and dependencies) rarely come with that perfect something to jazz things up. You almost always need to bring your own “pickle”—that little something extra, like custom code, patches, or special tweaks to make it really sing. In the world of software, just like TV dinners, it's all about choosing carefully, being prepared to discard what you don’t need, and always having your own “pickles” on hand to add a bit of flair. What’s your strategy for navigating the ups and downs of upstream dependencies? #NationalTVDinnerDay #opensource #softwareengineering #devlife #dependencies #softwaredevelopment #pickles #LinkedInLunacy

9

2 Comments

I've appreciated all the folks who have suggested I look at System Initiative this week. The great thing is, I've been tracking them for a bit now. I've even written about them and the other companies who are changing how we work, for the better. All the Next Wave companies are focused on context, helping teams navigate the current level of complexity, and preparing teams for what comes next. Huge believer in this problem and the companies that are solving it. Including my own. We're like dependabot for your entire technical stack. I don't know if you are aware but unplanned work is a big pain point for our Ops friends and it's only getting worse. So why not start with the small stuff no one has time to get to, Automate that away and give teams peace of mind while reducing toil. Don't worry Security, there will be checks and balances including an audit trail. It'll make staying in compliance a snap! All that to be said, SI's GA announcement is cool, and the approach of moving from a factory to a team. A pipeline to something more organic to manage infrastructure is timely. The team has worked hard and it shows in their GA offering! While you are checking out SI, I'd highly recommend checking out some of the other companies in the Next Wave of DevOps, if you haven't already. The number of companies is growing and the founding teams are passionate about this space and the problems. We are better together and all of the companies are amazing! + Anyshift.io + AppMap + CloudTruth + DryRun Security + Overmind + Prequel.dev + RunWhen + Stakpak + Stanza New companies are making the hard parts more approachable with our complex and ever-growing technical stacks. From code to deployment and everything in between #opservations

13

1 Comment

Anthropic released another paper on their ongoing work towards interpretability of large foundational models like their Claude 3 family, built on previous work by many. Their work continues to be enlightening and inspiring. The findings in this work are generally similar to previous findings on smaller models but scaled up to Claude 3 Sonnet: 💡 they find large numbers of "features" corresponding to combinations of neuron activations and these features map to concepts that we recognize/understand: for example, features for transit infrastructure, particular people, concepts (like bias and "objects lacking sentience"). This might sound familiar from classical machine learning, but the interesting thing is that there is no single thing/neuron encoding each of these concepts, it's a particular combination of neuron activations that embeds these concepts in "superposition" in the network. 💡they observe a sort of "distance" of these feature activations that arrange themselves "near" semantically similar features. When they use their sparse autoencoder approach to learn more and more features they observe that the level of abstraction in the detected features becomes more granular, in ways that are interpretable and meaningful. 💡when they artificially fire these activation patterns during inference, it affects the output in ways that relate to the "feature". For example, asking Claude "what's the most interesting science" normally it responds Physics, but if you clamp the feature activations for the feature "Brain Science" to 10x their max values then Claude responds: Neuroscience. Ways to interpret _and_ influence the model. Great readable work [1] along with an interactive explorer of some of the "features" themselves [2] 🎂 There's an easter egg too: if you scroll in the feature explorer tool and try and select one of the "Randomly selected features from 1M" you get a message "if you want to see the rest of the features, we're hiring!" with a link to their recruiting page :) [1] https://lnkd.in/gZbparwp [2] https://lnkd.in/gC9eus53

19

Does anyone else remember sitting in class thinking, "Am I ever going to actually use this?" 🤦‍♂️ I still ask myself this weekly when reading blog posts, listening to podcasts, catching up on massive Slack threads, or diving into documentation and feature listings. I don't always use the info, but it feels great when a memory from six months ago suddenly becomes relevant. I always think of that original Jurassic Park Unix scene. This often happens when discussing DevOps and Serverless, where boundaries blur already. Trying to keep all of the AWS documentation in my head while discussing the design of new features and optimizations feels like a superhuman task. What are some things you wish you had remembered when solving an issue or problem? For me, we designed a Lambda function to fail purposefully instead of attempting to recover for efficiency. Unfortunately, the Lambda control plane self-throttles a function once a failure threshold is reached... slowing our processing to a crawl and forcing some quick code changes. 😅 #AWS #lambda #serverless #learninginpublic #hugops

13

1 Comment

Is Agile dead? TLDR: Almost. It's important to think about the environment when the elite, all white, male group of fancy technologists went on a ski vacation.... Everyone went into the office and most tech groups were colocated. They could swivel chair and chat with each other, and take a flight of stairs to go sit with the business, daily. Now, we are remote and distributed. The "business" is also remote and distributed. Folks are +/- at least 3 hours timezone difference almost in all scenarios. What's more, not only is the physical/geographical landscape different but back then, the product adoption market was much smaller. You could have a dedicated team working on one product, with no outside dependencies. Today's digital footprint is $950B USD market with over 15B connected IoT devices worldwide. So, with remote & distributed teams and with multiple teams working on the same product and sharing a spiderweb of dependencies between both aging systems and almost-aging systems ... TODAYS WORLD IS INDEED DIFFERENT. Fundamentally so. (I've not even scratched the surface of the AI impact) Meaning, these things need to continually be challenged/questioned: - Individuals & Interactions being far more important than processes and tools - Biz and Devs work together DAILY - Face-to-face conversation The Agile framework, as envisioned ... was/is brilliant and beautiful. But so are unicorns and dragons. "Is Agile Dead?" is probably not the best way to ask the question... I think the better question is.... "Does Agile need heavily re-designed?". That answer is resoundingly, YES. Stay tuned for part 2 (discussing agile re-designs and recommendations). #Agile #SoftwareEngineering #Tech

28

3 Comments

Sometimes, you just can’t force it. The more you push, the more you risk things slipping out of control—whether it’s an idea, a project, or just trying to find the right words. That’s where permission to fail becomes powerful. It shifts the mindset from perfection to experimentation, from fear to opportunity. Jerry: "It’s like trying to tighten a bolt that just won’t catch. The harder you crank, the worse it gets. Giving yourself room to stop and reassess isn’t giving up—it’s keeping the thread intact for when the right solution shows up." Todd: "Failure, when managed well, isn’t a setback; it’s data. Every misstep offers insight. In agile terms, it’s iterative learning—each cycle teaches us something new, even when it doesn’t end in success." Jerry: "And here’s the thing—it’s not just about failing. It’s about knowing you can fail and still move forward. That permission shifts everything. It’s like finally exhaling after holding your breath through every decision." Todd: "Exactly. Teams and individuals thrive when they feel safe enough to take risks, knowing they won’t be punished for every stumble. True progress often happens when people experiment without the fear of failure hanging over them." Permission to fail isn’t an excuse; it’s a strategy. It’s not about abandoning the effort—it’s about shifting the focus from the result to the learning along the way. And sometimes, the most productive thing you can do is step back, reflect, and let the process unfold at its own pace.

3

1 Comment

Damn it felt good to see this at BlackHat . Elisity's integration with Claroty is now listed under "Supported Integrations" in their latest UI update. Phase 2 of this integration kicks off this month so stay tuned. 😎 -------------- Elisity seamlessly connects to Claroty xDome to enrich IdentityGraph, Elisity's device identity and attribute database. Detailed device attributes such as device type, manufacturer, model, serial number, operating system, firmware version, Purdue level, custom labels and much more can be leveraged as Elisity classification match criteria to establish either fine grain or coarse grain least privilege access policy for the organization. With this integration, organizations can quickly gain visibility into network assets and traffic flows, assess risks, and begin building least privilege access policies, dynamically enforced at the edge, to protect the most critical enterprise assets from malicious network traffic. Medigate by Claroty Claroty #segmentation

100

7 Comments

The next code smell I want to highlight is one of my personal pet peeves, Primitive Obsession. This means using a primitive or String for a value just because it sort-of fits, rather than using a well-designed object. Yes, there is effort in creating the object, but that effort pays off many times over. What do I mean? How about using a float to represent money just because it has a decimal point. That *always* causes problems. You always have to deal with imprecision and rounding errors. You need a Money class that can store exact values, handle multiple currencies and perform conversions. Another horrible idea is to use a String (or even worse, an int) to hold a phone number. You often need to parse out a country code or an exchange by splitting the String or dividing the int. You can express these concepts directly in a TelephoneNumber class. The same goes for using String to represent an address. Use an Address object to hold a name (which should also be an object), a house number, street name, apartment number, city, province. Postal code and country. The same goes for dates, times, ranges and other types. Of course, Java has already defined classes for those. Any time you have a value in a primitive and keep having to split it apart to use it, you really need to define an object.The appropriate refactoring is “Replace Data Value With Object”. Sometimes, we use a primitive to represent a type code, which can also cause problems. You should use an enum or a class. If you don’t depend on the type code to affect behavior of the system, use “Replace Type Code With Class.” If you do use it for branching, then you can consider “Replace Type Code With Subclasses”, or “Replace Type Code With State/Strategy”. Another smell is storing parts of a value in an Array, then having to pick out pieces by knowing the indices of those pieces. If you find yourself doing this, you can use “Replace Array With Object”. Sometimes, you might have pieces of a value in individual primitive fields and parameters. In this case, you can put them together with “Extract Class” or “Introduce Parameter Object.” Shoe-horning complex values into a primitive is always a bad idea. You may think it is “easier” or “less work”, but it always comes back to bite you. You can find descriptions of the refactorings at https://lnkd.in/gdJjNQcF

19

My #1 management rule that I’ve seen even very senior people get wrong - people’s actions will mostly mirror how you treat them: If you treat devs like children with blinders on they will act like children. If you hide costs from devs they won’t care what things cost. If you don’t have devs on the hook for when things break they won’t care so much about when they break. If you treat office workers as untrustworthy they will act untrustworthy. If you force RTO to keep an eye on people they will waste more time in the office. The tighter you squeeze your fist the more falls through. Thats not to say there aren’t exceptions to this but it’s true in the general case.

57

10 Comments

This circuit board has been my nemesis for months now. From an initial design that was too expensive to be viable, to a full rework in collaboration with some awesome folks from my community, through weeks of battling to create a brand new SDRAM controller in old-school programmable logic and many, many late nights chasing phantom electrical issues, this circuit board has taken me to hell and back. And today, finally, after one destroyed prototype, we got this one working. Perfectly. As an engineer the temptation is to immediately jump in to the next bit, start sorting out the FPGA or extending the software, but that’s the wrong move. It’s important to take a moment to enjoy your successes, and give yourself a moment to recognise your achievements. We forget that too often, so this post is two things: my moment to celebrate my success, and your reminder to make sure you celebrate yours.

29

The most underrated skill in the 2020's: the ability to have honest, open conversations with co-workers. Most of my career was spent in-office, crammed into a basement at a startup or lost inside the large scale cubicle farms of big tech.. no matter which: Employees usually had a spot outside the office, a bar or a cafe, where truth could be told, and where upper management knew not to go. These places don't exist with remote work. There's an artificial barrier in the way of authentic communication. Authentic communication does not happen by accident. That doesn't mean it's not possible, but it has become harder, and has become even more important. In the zoom era, you have to be forward. One of the most powerful moves you can make in your career is to reach out to a co-worker with the following line, and see where it goes: 🤔 Can I be honest about my experience at this company? You have to build trust with colleagues, often slowly, but once you have it, lean in. Shout out to, the Hi Hat, Opal Divines, The Handlebar, and even the Yard House, where the real work of building startup culture took place in Austin for me All of my management and coaching experience, has been remote. As a manager, I know the barriers that must be broken before a culture of candor can be created. As a coach, I know how to help people through the landmines of a toxic culture. The difference between success & failure, comes down to working with people you trust and respect. I have a limited number of free coaching sessions in the upcoming month - anyone who's interested should DM me #careercoaching #coaching

12

I struggle with writing on this platform. I see so many "influencer" types and clickbait headlines that I get annoyed when I scroll through. When I DO write something, I've got a shoulder gremlin that tells me I'm just like those influencers and I'm adding to the noise. I don't have writer's block by any means (I've got 116 drafts on Substack). What I've got is publishing block. It's a weird sort of imposter syndrome where I'm afraid of posting because someone will think it's stupid, annoying, or wrong. Like they'd look at this post and be like "oh my...get over yourself." You might even rolled your eyes when you saw this in your feed! We all deal with some form of imposter syndrome, and mine sometimes gets the best of me. After chatting with a coworker last week with similar feelings we decided to push through the fear and publish more, holding each other accountable to doing so. Having someone to encourage you to "release the damn thing" goes a long way. So, back to a weekly newsletter I go, and I'll do my best to not be too clickbait-y with the content (but no promises...you'll never guess what I've linked to inside 😆). What areas are you struggling with this week? In what endeavors do you need that extra nudge? #imposter #management #engineering

37

2 Comments

••The Sudden, Unbearable Reality of Long Fingernails•• It’s almost comical how fast it happens. One minute, your fingernails are perfectly fine—functional, unremarkable. Then, almost out of nowhere, they’re too long. It’s like your brain flips a switch, and suddenly, you can’t think about anything else. Typing becomes unbearable. That scraping sound when they hit the keyboard? It feels like an assault. The urge to cut them now becomes the only thing that matters. And once you’ve crossed that line, the nail cusp, there’s no going back. It’s truly unbearable. You’re caught in an endless loop of frustration until you can get your hands on a pair of clippers. How much stress and anxiety in the world could be traced back to people simply needing a few quiet minutes with some clippers? Maybe we’re all walking around a little more tightly wound than we need to be, just because we haven’t taken care of those small, nagging discomforts. But a word of advice—never clip your nails in your cubicle. Sure, the urge is strong, but save yourself the embarrassment and your coworkers the discomfort. No one wants to hear that. We all have these fingernail moments—small things that creep up, unnoticed, until they’re suddenly unbearable. So, let’s have an open conversation. What’s your fingernail moment? Here are a few to get us started: • Unread Emails: The inbox that quietly grows, and suddenly the unread count is staring back at you like a monster, demanding attention. • Unused Vacation Days: You tell yourself you’ll take a break someday, but then you realize you haven’t had a real vacation in over a year, and the burnout feels inevitable. • Lingering Feedback: That one piece of constructive feedback you’ve been avoiding dealing with, until it starts nagging at you in the middle of the night. • Neglected Projects: The side project or task that’s been sitting on your plate for weeks (or months), quietly gathering dust, until the deadline sneaks up and panic sets in. What’s yours? Let’s open up the conversation. Maybe we can all start trimming our fingernails a little sooner.

2

1 Comment