Ship It Weekly Host Commentaries

For this Ship It Conversations episode, I wanted to get past the LinkedIn hype cycle around “AI agents for DevOps” and talk to someone who actually wired one into their stack without losing sleep over it.

Gracious has been doing exactly that with TARS, a human-in-the-loop fixer bot that plugs into CI/CD, GitHub, and your containers. What I like about his story is it didn’t start as “let’s build an AIOps platform.” It started with a very boring, very real problem: chasing Docker logs and doing the same incident steps over and over. First he built Friday, a status bot that could safely poke at containers and tell him what died and why. Then he layered TARS on top to correlate with commits, suggest where to look, and eventually help drive rollbacks, all behind hard guardrails.

The guardrail piece is what made this worth recording. He’s aggressively narrow with what each agent can see and do. One workflow can only run safe Docker commands. Another can read GitHub but can’t touch infra. Actions that change the world, like redeploying the last good build, require an explicit phrase from a human, and even then there’s a second layer of validation in the workflow itself. It’s not perfect or “formally verified” or any of that, but it’s a real example of segmenting incident response into sub-workflows and keeping the agent boxed in at each step.

I also appreciated his answer to “where should teams start?” His take: you don’t start with an LLM. You start by writing down your own process, breaking it into steps, and turning those into programs and sub-flows. Only then do you drop an LLM in as glue or a helper. Same thing with skepticism about fully autonomous agents. He’s pro-automation, but he’s still very clear that human judgment, gut checks, and validating every LLM output before acting on it are the non-negotiables.

If you’re the platform / SRE / DevOps person who keeps getting asked about agents, AIOps, or “can we use AI to fix incidents,” this conversation should give you a concrete example and a vocabulary for pushing toward human-in-the-loop systems instead of “give the bot SSH and hope.” Links to TARS, Friday, and Gracious’s posts are down below, along with the other Ship It Conversations episodes.

This episode is my “tooling isn’t neutral” rant, in the nicest way possible.

We keep saying “it’s just automation” or “it’s just the workflow tool” or “it’s just the UI.” But in 2026, that stuff is where secrets live, where approvals happen, where production changes get triggered, and where a lot of hidden risk piles up. When it breaks, it’s not a small break. It’s a pager break.

So the theme of Episode 12 is simple: your automation layer is part of your perimeter, part of your reliability story, and sometimes part of your budget story too.

n8n: your “glue tool” is a control plane now
The n8n vulnerability is the perfect reminder that these workflow platforms are not side projects anymore.

If you’re using n8n as glue between systems, it usually has exactly the kind of access you don’t want compromised. Tokens. Webhooks. Internal endpoints. Automation that can touch real infrastructure. Once it’s doing anything beyond “send a Slack message,” it becomes production-adjacent, whether you meant it to or not.

The point isn’t “wow, a CVE exists.” The point is the operational posture you take around these tools.

If something like n8n is exposed, even accidentally, you want to already have the basics handled: strong auth, minimal exposure, least-privilege credentials, and a patch cadence that doesn’t involve “we’ll get to it.”

Because once a workflow tool becomes “the place where everything connects,” it becomes a high-value target. That’s not paranoia, that’s just how attackers think.

AWS Capacity Blocks: cost is an availability constraint
The AWS price change is a different flavor of the same theme.

Capacity Blocks for ML are basically you paying a premium so you can actually get GPU capacity when you need it. And when AWS bumps pricing there, it’s a reminder that scarce compute behaves like a market. It’s not “cloud is always cheaper over time.” It’s “cloud can change the deal.”

If you’re a GPU-heavy shop, this hits your forecasts and your internal trust immediately. Finance thinks you overran spend. Engineering thinks finance is being weird. Leadership asks why nobody caught it earlier. Nobody is wrong, it’s just a supplier shifting pricing.

The platform takeaway is: you want tripwires. Not just “our EC2 spend is up,” but “the effective rate on the SKUs we depend on changed.” Cost visibility isn’t just a finance thing. It affects how reliable your delivery is, because budgets control what you’re allowed to run.

Netflix + Temporal: stop running critical ops as fragile scripts
This was the most satisfying story of the week.

Netflix basically says: we got sick of transient failures turning into broken deployments and flaky operational workflows, so we built on durable execution patterns using Temporal. And they claim a huge reduction in transient deployment failures.

I don’t even care about the exact number. I care about the pattern.

A lot of ops work is a long-running workflow, but we implement it like it’s a single command. Deploys. Rollouts. Backfills. Migrations. Rotations. Anything that takes multiple steps, calls multiple APIs, and requires verification.

When those workflows live as brittle pipelines or bash scripts, every timeout becomes a human incident. Somebody reruns jobs. Somebody guesses which step ran. Somebody repeats a dangerous step because the logs were unclear. That’s where “toil” actually comes from.

Durable workflow thinking flips it. Make critical ops resumable. Make it safe to retry. Make it obvious what step you’re on. Make it observable enough that the on-call doesn’t have to become Sherlock Holmes while prod burns.

You don’t need Temporal tomorrow to adopt the mindset. You can steal the pattern right now. Pick one workflow that’s always painful, and make it reliable end to end. Resumable, idempotent, and boring.

Lightning round: dependencies die, and supply chain is still the job
Kubernetes Dashboard getting archived is one of those quiet platform realities: stuff gets abandoned. If a tool is unmaintained and it’s in your workflow, you want to move while you’re calm. Not after security flags it.

Docker Hardened Images is a nice signal too. The industry keeps inching toward “secure defaults” for base images because container supply chain risk is not a theoretical problem anymore.

GitHub Spec Kit is interesting in the AI era for a different reason. Specs are turning into guardrails. If code can be produced quickly, the quality bottleneck becomes the clarity of the contract. Spec-first workflows are basically a way to slow down the right part so you can speed up the rest safely.

And then the Cloud Posse links this week were a nice grab bag of practical tooling. Argo diff visibility in PRs is exactly the kind of small thing that reduces surprise later. Pipedash scratches the “why do I have five CI UIs open” pain. Atmos roadmap is just good hygiene for any platform tool you’re betting on. Tonkotsu is another reminder that agent tooling is accelerating fast, but the limiting factor is still controls and review, not raw speed.

Human closer: faster systems compress human reaction time
Here’s the thread that ties the whole episode together.

Speed is cheap now. Automation is cheap now. Agentic workflows are going to get even cheaper.

The hard part is designing systems where speed doesn’t turn into security incidents, budget surprises, and “we don’t know what just happened.”

n8n is the reminder that convenience becomes perimeter.
AWS pricing is the reminder that convenience has a bill.
Netflix is the reminder that convenience still needs durable, boring engineering under it.

So if you take one mindset into the rest of 2026, I’d take this:

Make your automation survivable. Resumable. Observable. Least privilege. Easy to stop. Easy to roll back.

Because the future isn’t less automation. It’s more automation with less time to react.

Links
SRE Weekly #504: https://sreweekly.com/sre-weekly-issue-504/
n8n CVE: https://nvd.nist.gov/vuln/detail/CVE-2026-21858
AWS price increase coverage: https://www.theregister.com/2026/01/05/aws_price_increase/
Netflix: Temporal + reliable cloud operations: https://netflixtechblog.com/how-temporal-powers-reliable-cloud-operations-at-netflix-73c69ccb5953
Kubernetes SIG-UI thread: https://groups.google.com/g/kubernetes-sig-ui/c/vpYIRDMysek/m/wd2iedUKDwAJ
Pipedash: https://github.com/hcavarsan/pipedash
Docker Hardened Images: https://www.docker.com/blog/docker-hardened-images-for-every-developer/
Headlamp for Kubernetes: https://headlamp.dev/
More episodes + contact info: https://shipitweekly.fm

Episode 11 is another Ship It Interviews conversation, and it’s one I wanted to do because internal developer platforms are having a real moment right now, and a lot of teams are getting pulled into them without a clear plan.

I sat down with Danny Teller, a DevOps Architect and Tech Lead Manager at Tipalti, to talk about Backstage, internal IDPs, and the uncomfortable truth behind all of it: the portal is not the platform. We're not related, but he does have an awesome last name :)

Backstage comes up constantly because it’s a legit project and it solves real problems. A catalog, discoverability, templates, an “engineers can find stuff” layer. The issue is what happens next. A lot of orgs treat the portal like a magic reset button. They ship a UI, add a couple plugins, and expect the engineering experience to improve on its own. Then six months later it’s half outdated, nobody trusts it, and it becomes another thing the platform team has to babysit.

Danny’s angle, which I agree with, is that the make-or-break isn’t Backstage versus some other product. It’s whether you have DevEx muscle. Ownership, standards, paved roads, good defaults, support, keeping the catalog data accurate, and doing the boring work that makes self-service actually work. Without that, any portal is basically a fancy bookmark list.

We also talk through a practical build vs buy gut check. When it makes sense to lean on open source, when a managed offering or commercial portal is a better move, and the maintenance trap to watch for: building an internal platform that quietly turns into a second product with an infinite backlog.

If you’re a platform engineer, an EM being asked to “do an IDP,” or the DevOps person who just inherited this space because nobody else wants it, this episode should give you a sane frame for what matters and what doesn’t.

Links and resources are below, and you can always find episodes and extras on shipitweekly.fm.

This episode is basically my “welcome back to reality” message for the start of the year.

A lot of teams are coming into January with momentum: new initiatives, new tooling, new automation, more AI agents in workflows, more platform “self-service.” That’s great. But the trap is thinking speed automatically means progress.

Speed without containment just means you fail faster.

That’s why the theme of Episode 10 is brakes and blast radius.

Cloudflare: “Fail small” isn’t a slogan, it’s a design requirement
Cloudflare’s post hit for me because it’s the type of resilience work that’s invisible when it’s done right and brutally obvious when it’s not.

If you’ve been in ops long enough, you learn that the outages that really hurt are rarely “one machine died.” The painful ones are correlated. The same broken change lands everywhere, the same dependency falls over across regions, the same config causes a cascade across the fleet.

“Fail small” is the mindset that forces you to design systems where problems stay local by default.

You can translate that into normal-company terms pretty easily:

If one bad Terraform module change can break dozens of repos, you don’t have a module problem, you have a blast radius problem.

If one CI permission or runner change can halt every pipeline, you don’t have a GitHub problem, you have a control plane dependency problem.

If one networking change can brick multiple clusters, you don’t have a VPC problem, you have an environment isolation problem.

The answer isn’t “be more careful.” The answer is segmentation and progressive delivery. Make it physically hard for a change to take everything down at once.

If you want a quick “do we fail small?” gut check, it’s this:

Can we roll changes out to a small slice of traffic by default?
Can we stop the rollout quickly?
Can we roll back quickly?
And can we prevent a single change from touching all environments at once?

If any of those are “not really,” you’ve got a reliability project hiding inside your delivery process.

Pulumi’s IaC push: the workflow is the product now
Pulumi’s “all IaC, including Terraform and HCL” post is interesting because it’s not really about Terraform vs Pulumi.

It’s about the control plane around infrastructure changes.

The market has been shifting from “pick an IaC language” to “pick a workflow that your org can live inside.” Approvals. Policy enforcement. Auditing. Drift detection. Visibility. Team-wide patterns. This is where platform work actually lives.

The reason Pulumi’s move matters is that it’s trying to lower the rewrite tax. Most shops don’t want to rebuild everything. They want better guardrails without a multi-year refactor.

So if you’re a Terraform shop, the question isn’t “should we switch?” The question is:

Do we have a real, consistent workflow around infrastructure change?
Or are we still depending on hero knowledge and fragile pipelines?

If you’re already happy with your control plane (TFC/TFE, Spacelift, Atlantis, or your own internal setup), cool. This is still worth watching because it’s a sign of where the “platform baseline” expectations are going in 2026: centralized runs, policy-as-code, least privilege, auditable approvals, and safer defaults.

Meta DrP: turning incident investigation into software
The DrP story is my favorite one because it’s the purest “SRE as engineering” example in the episode.

Humans do the same investigation steps every time an incident happens:
What changed?
What’s the timeline?
What services are correlated?
What’s the dependency health?
What errors spiked?
What deploys landed?

Meta’s angle is: stop doing this manually. Codify the investigation patterns and run them automatically as analyzers.

Even if you never build anything like DrP, the model is worth stealing:

Pick your top recurring incident types.
For each, identify the first three questions you always ask.
Automate those three questions into a consistent “first 10 minutes” incident response.

That can be a Slack bot.
It can be a runbook template with real links.
It can be a script that generates a timeline.
It can be as simple as “incident channel gets auto-populated with deploys, dashboards, and relevant queries.”

The win isn’t AI magic. The win is consistency. It reduces the cognitive load when things are on fire, and it stops your on-call quality from depending on who drew the short straw that week.

Lightning round quick follow-ups
GitHub Actions is still the theme of “control planes matter.” If you caught Episode 6, you already know my take: Actions isn’t just CI anymore, it’s part of the delivery pipeline, the GitOps loop, and sometimes the break-glass path. If that control plane has pricing changes, incidents, or performance problems, it’s not “dev inconvenience,” it’s operational impact.

AWS ECR creating repos on push is one of those features that sounds small until you multiply it across a big org. It’s either a nice automation win or a new sprawl problem, depending on whether you have naming standards and default security controls.

Metrics and MTTR: I love the reminder that averages can lie. MTTR is often dominated by outliers, and that makes “we improved our MTTR by 10%” a pretty weak claim unless you’re looking at distributions and repeatable process improvements.

And drift is still drift. If your IaC story doesn’t include drift detection and a plan to respond, “source of truth” is basically a motivational poster.

Human closer: the ironies of automation
This is the part I want to underline.

Automation doesn’t remove responsibility. It moves responsibility.

Faster automation creates a harder oversight problem. If a system can do more things faster, it can also do the wrong thing faster, and humans have less time to notice and react.

So the real platform job is not just “add automation.” It’s “design the control loop”:

Make actions observable.
Contain blast radius.
Slow down when confidence is low.
Make rollback easy.
Design safety rails that don’t depend on perfect humans.

That’s what “fail small” is really about. And it’s why it’s a perfect theme for the first weekly episode of the year.

Links
SRE Weekly #503: https://sreweekly.com/sre-weekly-issue-503/
Pulumi: all IaC, including Terraform and HCL: https://www.pulumi.com/blog/all-iac-including-terraform-and-hcl/
GitHub Actions direction: https://github.blog/news-insights/product-news/lets-talk-about-github-actions/
AWS ECR create repos on push: https://aws.amazon.com/about-aws/whats-new/2025/12/amazon-ecr-creating-repositories-on-push/
DriftHound: https://drifthound.io/
Superset: https://superset.sh/
Episode 6 (Actions pricing pause backstory): Episode 6Dec 20, 2025⏱️ 12:06GitHub Runner Pricing Pause, Terraform Cloud Limits, and AI in CIEpisode: GitHub Runner Pricing Pause, Terraform Cloud Limits, and AI in CI

Episode 9 is another Ship It Interviews conversation, and it’s one I really wanted to do because it’s the kind of path a lot of people are on right now.

I sat down with Eric Paatey, a Cloud & DevOps Engineer who’s been moving from full-stack web dev into cloud/DevOps. What I liked about Eric’s story is that it’s not “I bought a bunch of tools and rebranded my LinkedIn.” It’s the slower, more real version: build things, break things, fix them, and keep stacking reps until it starts to feel normal.

One of the big themes in this conversation is that DevOps is not a tool list. It’s a set of habits. Systems thinking. Communication. Owning reliability. Being the person who asks, “what happens when this fails,” and then actually doing something about it.

Eric talks about that moment a lot of people hit when they realize code is only part of the job. Shipping is the job. The pipeline is the job. The infrastructure is part of the product. And if you can’t deploy, observe, and recover, it doesn’t matter how clean the code is.

We also spent time on the homelab question, because this comes up constantly. People think they need a rack of servers or a fancy setup to learn DevOps. You don’t. The hardware is not the point. The point is having a safe environment where you can build muscle memory: automate the boring stuff, learn basic networking and IAM, set up monitoring, intentionally break things, and practice recovery.

If you’re trying to break into DevOps, or you’re mentoring someone who is, the most useful part of this episode is probably the “first real project” we talk through. Something you can actually show: take a simple app, Dockerize it, deploy it behind an ALB, wire up a little bit of security and networking, and document what you learned. Not because it’s the most advanced thing in the world, but because it proves you can build and operate something end-to-end.

We also touched agentic AI and MCPs, and the same warning I keep coming back to: don’t give agents full access to anything. AI helpers are going to show up everywhere in ops workflows, but least privilege, policy, and guardrails matter even more when the system is non-deterministic.

If you’re early in your DevOps path, this episode should be encouraging in the right way: you don’t need a perfect setup, you need consistent reps. If you’re already in the job, it’s a good reminder of what “good” looks like when you strip away the buzzwords.

Links and resources are below, and you can always find episodes and extras on shipitweekly.fm.