Ship It Weekly Host Commentaries

This episode is about safety nets.

That sounds comforting, and honestly, it should. Rollback buttons are good. Better alarms are good. Zone-aware routing is good. Secret scanning with more context is good. Internal advisory workflows are good. Better agent guardrails are good. I do not want fewer safety nets in production systems. I want more of them.

But safety nets create a weird kind of risk too. They can make teams feel like the hard part is solved. And usually, it is not.

A rollback feature does not mean the rollback is safe. An alarm does not mean the signal is worth waking someone up. A scanner finding a secret does not mean rotation is easy. An AI coding agent finding a fix does not mean it should run commands. And zone-aware routing does not mean you understand your failure domains.

That is really the thread through this episode. The tools are getting better. The recovery paths are getting better. The visibility is getting better. But the systems underneath are still complicated, still connected, and still full of places where one bad assumption can turn a small problem into a production incident.

Start with EKS version rollbacks.

This is a genuinely useful feature. For a long time, Kubernetes upgrades have felt like a one-way door. You check your cluster version. You check release notes. You check deprecated APIs. You check add-ons. You check admission controllers. You check node versions. You check whatever chart everyone is scared to touch. Then eventually, you upgrade.

And if something breaks, the path back is not always clean.

So EKS adding version rollback is a big deal. It gives platform teams a recovery option when a Kubernetes minor version upgrade exposes some compatibility issue they did not catch ahead of time. That helps, especially if you run a lot of clusters. Especially if you have strict change windows. Especially if every Kubernetes upgrade already involves a calendar invite, a Jira ticket, three approvals, and someone saying “should be fine” in Slack.

But rollback does not erase the work. It changes the risk model. That is the important distinction.

The existence of rollback does not mean workloads are compatible. It does not mean add-ons are safe. It does not mean admission webhooks behave correctly. It does not mean node groups are ready. It does not mean disruption budgets are sane. It does not mean the rollback itself will be instant. And it definitely does not mean the cluster goes back in time like nothing happened.

Rollback is an operation. It has requirements. It has timing. It has dependencies. It can fail in its own special way.

That is why this feature belongs in the upgrade runbook, not in the “we’ll figure it out if it breaks” bucket. The better question is not “can we rollback?” The better question is “have we tested the rollback path?”

What happens to nodes? What happens to add-ons? What happens to workloads? What happens with EKS Auto Mode? What happens if disruption budgets slow the rollback down? What happens if the thing that broke during upgrade also affects the path back?

A rollback button is great. But if nobody owns the upgrade path, the rollback button can become false confidence. That is the thing to watch.

Cordyceps hits the same theme from a different angle.

This one is about GitHub Actions and CI/CD supply chain risk. And the reason it matters is not just “workflows can be vulnerable.” We already know that. The reason it matters is that a lot of teams still treat workflow YAML like it is not real code.

It is in .github/workflows, so it feels like config. It gets copied from a blog post. It gets borrowed from another repo. It gets tweaked until the build is green. And then it sits there for years with production authority.

That is the problem.

GitHub Actions workflows can run commands. They can read secrets. They can publish packages. They can assume cloud roles. They can sign artifacts. They can push containers. They can cut releases.

That is not harmless automation. That is privileged execution.

And the nasty part with CI/CD attacks is that the dangerous path is not always obvious. It might not be one line that screams “this is bad.” It might be a low-trust workflow producing an artifact, then a higher-trust workflow consuming it later. It might be a branch name landing in shell. It might be a pull request title becoming command input. It might be a cache crossing a privilege boundary. It might be a comment, issue body, label, tag, or generated file moving from untrusted context into trusted automation.

Each step looks normal enough on its own. Together, you accidentally built an attack chain.

That is why pipeline security is hard. The boundary is not always one file. It is the relationship between triggers, permissions, artifacts, caches, tokens, branches, environments, and who is allowed to influence each of those pieces.

And now AI makes this more interesting.

Because if AI coding agents are generating CI/CD configuration, they can reproduce bad patterns quickly. Not because the agent is evil. Because insecure workflow patterns are everywhere. If the training data, examples, old repos, or copied snippets include bad assumptions, the agent may hand you a pipeline that works beautifully and trusts the wrong thing.

That is a rough combination.

Fast generation. High privilege. Weak review. Production authority.

So the fix is not “stop using GitHub Actions.” That is not realistic, and it is not the point. The fix is to treat workflow logic like production code.

Review it. Threat model it. Limit permissions. Separate trusted and untrusted workflows. Be careful with pull_request_target. Watch shell interpolation. Be suspicious of artifacts and caches crossing privilege boundaries. Assume your CI/CD system is part of your supply chain, because it is.

And attackers already treat it that way.

Then agentjacking takes the trust boundary even closer to the day-to-day work.

This story is about fake telemetry influencing AI coding agents. And I think this one matters because it changes how we think about logs, errors, and incident context.

For years, telemetry has mostly been evidence. An error happens. A log line appears. A stack trace gets captured. A Sentry issue gets opened. A human reads it and decides what to do.

The text might be noisy. It might be wrong. It might include user input. But a human is still sitting between the evidence and the action.

AI agents change that.

Now the same text may be read by something that can act. It might inspect a repo. It might suggest a patch. It might run tests. It might install packages. It might execute commands. It might open a pull request. It might touch credentials sitting in the local environment.

That changes the boundary.

The question is no longer just “can an attacker compromise the server?” The question becomes “can an attacker put text somewhere the agent will trust?”

That is a much lower bar.

And it is uncomfortable because our systems were not always designed with that in mind. We sanitize logs for display. We redact secrets. We limit access to observability tools. But we have not always treated error text as something that could steer an automated actor.

Now we have to.

If an agent can read hostile context and then take action, that context is part of the security model. Logs are not just logs. Issue text is not just issue text. Stack traces are not just stack traces. Runbooks are not just documentation. They can become instruction material.

So the control cannot just be “the agent is smart.” That is not a control.

The control is authority.

What can the agent read? What can it run? What can it write? Can it inspect without executing? Can it propose without applying? Can it separate trusted remediation guidance from user-controlled error content? Can you audit what it saw? Can you audit what it did? Are there live credentials nearby when it makes a bad decision?

That is the platform question.

Not “does this make developers faster?” That is a product question.

The platform question is “what happens when it is wrong, and what can it touch while being wrong?”

That is where agentjacking lands for me. It is not just an AI story. It is a trust boundary story. The text around your systems can now cause action. So the trust model has to include the text.

CloudWatch log query alarms are a much more normal operations story, but they fit the theme too.

This is a useful feature. A lot of real incidents show up in logs first. A specific exception. A repeated auth failure. A dependency timeout. A database connection error. A queue consumer screaming before the metric catches up.

For years, turning that into an alarm meant extra glue. Metric filters. Custom metrics. Another vendor. A Lambda path. A weird pipeline where a log pattern becomes a metric and the metric becomes an alarm.

Now CloudWatch can create alarms from log queries directly. That is good. It shortens the path from signal to action.

But it also shortens the path from random scary log line to pager noise.

And that is where teams can hurt themselves. Because the hard part of alerting is not creating the alert. The hard part is deciding whether the alert should exist.

Who owns it? What does it mean? How often does it happen? Is it user-impacting? What threshold makes sense? What does the on-call do when it fires? Does it flap? Does the log format change every deploy? Is it a symptom, or just a noisy internal detail?

If nobody can answer those questions, you did not create an alert. You created a search shortcut with a pager attached.

And every on-call team has lived through some version of that.

Someone sees an error. Someone says “we should alert on this.” Nobody wants to be the person arguing against “better monitoring.” So the alert ships. Then three weeks later, it is waking people up for a condition nobody understands and nobody owns.

That is how observability becomes punishment.

So yes, use log query alarms. They are useful. But use them for known failure modes. Clear owner. Clear threshold. Clear action. Clear reason to interrupt a human.

That is the difference between faster signal and faster noise.

The lightning round keeps adding to the same shape.

ECS Service Connect zone-aware routing is a good example. On the surface, it sounds like a networking improvement. Prefer same-AZ traffic when possible. Reduce latency. Reduce cross-AZ cost. Improve behavior during partial failures.

That all sounds great.

But it also reminds us that service-to-service routing is never just “can service A reach service B?” It is where traffic goes. How much it costs. What happens when an Availability Zone gets weird. What fails open. What fails closed. What happens when an old service needs a redeploy to pick up the new behavior.

Networking defaults matter because defaults become architecture.

etcd 3.7 is another quiet but important one.

Most people do not wake up excited to talk about etcd. Fair. But Kubernetes reliability still depends on the boring distributed database underneath it.

RangeStream, performance improvements, cleanup of older v2 store remnants, protobuf updates, dependency updates. None of that is flashy. But control plane reliability is built out of boring things working correctly under pressure.

The API server, scheduler, controller manager, kubelet, CNI, CSI, admission webhooks, and etcd all have to hold together. So when etcd gets better, Kubernetes gets a little more room to breathe.

GitHub innersource advisories are also easy to underplay.

Public CVEs get attention. Internal dependencies often do not. But a lot of companies run on private packages, internal platform libraries, shared GitHub Actions, private modules, and internal service templates.

Those things need security workflows too. Disclosure. Affected versions. Dependabot notifications. Remediation paths.

A vulnerable internal package can still be a real supply chain problem, even if it never touches the public internet.

Secret scanning metadata is similar.

Finding the secret is not the whole incident. It is just the beginning.

Is it real? Is it live? Who owns it? What system uses it? Is it expired? Can it be revoked safely? Will rotating it break production? Does it need a workspace URL, endpoint, project ID, or other context to validate?

Better metadata matters because incident response is full of these small, painful questions.

The scanner saying “I found something” is useful. The scanner helping you figure out what it is, where it belongs, and whether it is still active is much more useful.

And CloudWatch Application Signals service events hit the same operational theme.

When something is slow, you do not only need to know that it is slow. You need to know what changed.

Was there a deploy? Was there an exception spike? Was there a latency event? Did the problem start after a rollout? Did the shape of traffic change?

Observability gets better when the signal and the context live closer together. But again, only if someone owns the events and knows how to use them.

Otherwise, it is just another wall of things that happened.

That is why the closer lands on ownership.

The episode is not anti-tool. It is the opposite. Use the tools. Use the rollback button. Use log query alarms. Use zone-aware routing. Use better scanner metadata. Use internal advisories. Use AI agents where they help.

But do not confuse better tooling with transferred responsibility.

The system is still yours. The pager is still yours. The blast radius is still yours. The customer impact is still yours. And the judgment is still yours.

So the practical questions this week are pretty simple.

Where are the safety nets? Who owns them? Have they been tested? What do they not cover? What do they make easier? What do they make riskier? What authority do they inherit? What happens when they are wrong? And who is responsible when everyone assumed the tool meant the thinking was done?

That is the work.

Safety nets are good.

But they do not replace ownership.

---

Extra links worth including:

Amazon EKS Kubernetes version rollbacks
https://aws.amazon.com/blogs/aws/upgrade-amazon-eks-clusters-with-confidence-using-kubernetes-version-rollbacks/

Novee Security: Cordyceps supply chain research
https://novee.security/blog/cordyceps/

Tenet Security: Agentjacking through fake Sentry errors
https://tenetsecurity.ai/blog/agentjacking-coding-agents-with-fake-sentry-errors/

Amazon CloudWatch log query alarms
https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-cloudwatch-log-alarms/

ECS Service Connect zone-aware routing
https://aws.amazon.com/about-aws/whats-new/2026/07/ecs-service-connect-zone-aware/

etcd 3.7 announcement
https://etcd.io/blog/2026/announcing-etcd-3.7/

GitHub innersource security advisories
https://github.blog/changelog/2026-07-08-innersource-security-advisories-are-generally-available/

GitHub secret scanning extended metadata and multipart validation
https://github.blog/changelog/2026-07-07-secret-scanning-extended-metadata-and-multipart-validation/

CloudWatch Application Signals service events
https://aws.amazon.com/about-aws/whats-new/2026/06/cloudwatch-service-events/

This week’s On Call Brief
https://learn.tellerstech.com/on-call-brief-news/2026-W28/

More Ship It Weekly episodes
https://shipitweekly.fm/

This conversation with Evan hit on something I think a lot of platform and DevOps teams feel, even if they do not always say it directly: deployment is still weirdly painful.

That sounds almost ridiculous at this point. We have had so many waves of tools that were supposed to make this easier. Cloud platforms. Containers. Kubernetes. Terraform. GitOps. Internal developer platforms. PaaS products. CI/CD systems. Now AI agents that can generate a bunch of code and infrastructure glue before you have even had enough coffee to regret the prompt.

And yet, for a lot of teams, the actual act of taking an application and safely getting it into production still feels harder than it should.

I liked Evan’s framing of deployment as the “final boss” of software delivery, because it really does sit at the intersection of everything else. Deployment is not just “run the app.” It is the app, the runtime, the container image, the registry, the network path, the secrets, the config, the database, the rollback, the logs, the thing that wakes somebody up, and the thing the next person has to understand when something breaks.

That is why deployment tools tend to get complicated so quickly.

Every app is shaped a little differently. One HTTP app is not the same as another HTTP app. A Ruby app is not the same as a Go service. A JavaScript app is not the same as a Java service. A background worker is not the same as a public API. Then you multiply that by every cloud provider, every orchestration model, every security requirement, every team structure, every compliance need, and suddenly the “simple” deployment platform has to make a lot of choices.

And that is really where the conversation got interesting to me.

A lot of engineers say they want flexibility. And honestly, we do. We like having knobs. We like knowing that we can tune something if we need to. We like not being boxed in.

But there is a cost to that.

Every knob is a decision. Every decision is something somebody has to understand. Every exception is something the team has to support. And eventually the platform that was supposed to make delivery easier becomes another product the team has to maintain.

That is not always bad. Sometimes you really do need the big flexible platform. Sometimes Kubernetes is exactly the right answer. Sometimes Terraform gives you the control and consistency you need. Sometimes the team has enough scale, enough complexity, or enough operational maturity that the extra surface area is worth it.

But that is not every team.

Evan kept coming back to small teams. Teams that are application-focused. Teams that just want to ship. Teams that do not have a full platform engineering group sitting around waiting to build and maintain abstractions on top of abstractions. Teams that maybe need deploys, logs, rollbacks, config, HTTP apps, and a path to production that does not require them to become experts in every layer underneath it.

That is the part I think a lot of platform conversations miss.

The goal is not always to build the most powerful platform. Sometimes the better question is: how little platform can we get away with while still shipping safely?

I also liked the Terraform Enterprise and Terragrunt thread because it gets at the same idea from a different angle.

The hard part of Terraform was never only the language. It was the shape around it. How do you organize repos? How do you manage state? How do you split modules? How do you reduce blast radius? How do you avoid one giant infra repo where every change feels like it is poking a thousand resources?

That is why tools like Terragrunt became useful for a lot of teams. Not because everyone loves another layer, but because people wanted an opinion. They wanted guardrails. They wanted someone to say, “put this here, structure it like this, and stop inventing a new pattern for every environment.”

That desire for opinionated tooling is not a weakness. It is usually a sign that the team has better things to do than debate folder structure forever.

The OpenTofu discussion was interesting too. The license change obviously mattered, but Evan’s point was that OpenTofu also had a backlog of features people had wanted for a while. Some of those features were things Terraform had intentionally avoided, and in some cases for valid operational reasons. Encrypted state was the example that stood out. It sounds good, and in many ways it is good, but it also creates another availability and recovery problem if the keys become the thing blocking your infrastructure operations.

That is a very real infrastructure tradeoff. Security, recoverability, simplicity, and operability are always pulling on each other.

The Waypoint section may have been my favorite part of the conversation.

I remember Waypoint being one of those tools that seemed like it should make a ton of sense. A deployment workflow engine from HashiCorp? That sounds like something a lot of teams would want.

But Evan’s takeaway was pretty blunt: a pure workflow engine was not enough product.

That stuck with me.

Because if a tool makes you do a ton of homework before it gives you value, you are going to lose people. If the user has to set up the registry, wire the backend, understand where images go, configure ECS or Kubernetes or whatever else, and then finally maybe get to deploy something, that is a lot to ask.

Especially if the person’s actual job is not “become an expert in deployment plumbing.” Their job is to ship the application.

That also maps directly to the internal platform problem. How many companies have a deployment system someone started building because the team needed something better, then the boss told them to stop working on it because, technically, the company is not in the deployment platform business?

So now everyone is stuck with a half-finished platform that mostly works, but only if you know the weird ritual.

That Jenkins comment in the episode was painfully real. I think a lot of us have had some version of that. The job works, nobody knows why, nobody wants to touch it, and the entire deployment process is held together by fear and one person’s ancient Groovy choices.

The AI angle makes all of this even more important.

AI can absolutely help with infrastructure and deployment. It can generate Terraform. It can wire together services. It can produce config. It can read documentation and maybe map an app to a platform faster than a person could.

But Evan’s question was the right one: is the AI on call?

When the generated infrastructure breaks, who owns it? Who understands why those services were wired together that way? Who knows which parts are safe to change? Who knows whether the AI made a reasonable long-term operational choice or just produced something that worked once?

That is the risk I keep seeing with AI-generated infrastructure. It can make the first draft easier, but the first draft is not the same thing as operational ownership.

And maybe that means good abstractions matter more, not less.

If the platform surface area is small and obvious, AI has fewer ways to get it wrong. Deploy. Roll back. Get logs. Set config. Check status. That is very different from telling an agent to wire together a dozen AWS services and hoping it invents a sane deployment platform your team can operate for the next three years.

So the real future might not be AI replacing deployment platforms.

It might be AI making the difference between good and bad platform abstractions even more obvious.

The practical takeaway for me is this: be careful which opinions you adopt, and be even more careful which opinions you accidentally create.

If you choose Kubernetes, you are adopting opinions. If you choose Terraform, you are adopting opinions. If you choose Terragrunt, OpenTofu, ECS, Heroku, Waypoint, Miren, or an internal deployment system, you are adopting opinions there too.

The question is whether those opinions match your team.

Do they match your application? Do they match your operational maturity? Do they match the amount of infrastructure ownership you actually want? Do they help your team ship safely, or do they just move the pain somewhere else?

That is why I liked this conversation. It was not really a “this tool versus that tool” conversation. It was more about the shape of deployment work, and why smaller, clearer, more opinionated systems may be the right answer for more teams than we usually admit.

Sometimes the best platform is not the most powerful one.

Sometimes it is the one that lets the team ship, understand what happened, roll back when needed, and then get back to building the thing customers actually care about.

---

Additional Links Mentioned

Miren:
https://miren.dev

Miren Discord:
https://miren.dev/discord

Miren on GitHub:
https://github.com/mirendev

Evan Phoenix:
https://evanphx.dev

Evan on Bluesky:
https://bsky.app/profile/evanphx.dev

Puma:
https://puma.io

Rubinius:
https://github.com/rubinius/rubinius

Terraform:
https://www.terraform.io

Terraform Enterprise:
https://developer.hashicorp.com/terraform/enterprise

Terraform Cloud:
https://developer.hashicorp.com/terraform/cloud-docs

Terragrunt:
https://terragrunt.gruntwork.io

OpenTofu:
https://opentofu.org

HashiCorp Waypoint:
https://github.com/hashicorp/waypoint

Kubernetes:
https://kubernetes.io

Docker:
https://www.docker.com

AWS ECS:
https://aws.amazon.com/ecs/

Heroku:
https://www.heroku.com

Knative:
https://knative.dev

HashiCorp Vault:
https://developer.hashicorp.com/vault

This episode is about execution boundaries.

That sounds like a security phrase, but I think it is really a platform phrase now. Modern systems are full of places where something gets permission to run, inspect, transform, block, or decide.

That something might be an IDE extension. It might be a language server. It might be an AI coding assistant. It might be a VS Code task. It might be a WAF parser. It might be a sandbox running user-generated code. It might be an advisory database feeding alerts into a vulnerability management process.

The common thread is execution.

Before something runs, trust is still mostly theoretical. After something runs, trust becomes a blast radius.

That is why the Amazon Q Developer and AWS Language Server CVEs matter. The scary part is not just that there were bugs in an IDE plugin. The scary part is where those tools sit.

They sit next to source code. They sit next to local credentials. They sit next to AWS profiles. They sit next to project configuration. They sit next to terminals, repo context, MCP configuration, and sometimes production access.

So when a workspace can cross into command execution, that is not just a weird local developer bug. It can become a cloud credential problem, a source control problem, or a CI/CD problem.

And yes, the user still had to trust the workspace. But that is exactly the uncomfortable part.

Developers trust workspaces constantly. You clone a repo, open it, click through the prompt, and get back to work. That is normal behavior. But normal behavior becomes risky when the repo is no longer just source code. The repo is config. The repo is tasks. The repo is language server behavior. The repo is MCP setup. The repo is local execution context.

That is a much bigger boundary than most teams treat it as.

The JFrog story pushes the same idea even harder.

The hijacked npm and Go packages were not relying on the usual npm lifecycle-script path. They used hidden VS Code tasks configured to run when a folder opens.

That is the part that should make people pause.

The ecosystem is getting better at package-manager security. npm is changing install-script defaults. CI pipelines are paying more attention to dependency execution. Teams are getting more suspicious of install-time code.

That is good. But attackers adapt.

If the package manager gets harder, they move into the IDE. And the IDE is a great execution surface because developers open folders all day.

Example repos. Proofs of concept. Vendor samples. Issue reproductions. AI-generated apps. Random repos from Slack, GitHub issues, Discord, or wherever else work happens.

The editor is not passive anymore. It runs extensions. It loads config. It starts language servers. It runs tasks. It reads environment variables. It can access terminals and local secrets.

So “don’t run random code” is not enough advice anymore.

Sometimes opening the folder is the thing that runs code.

The AWS WAF HTTP/2 story is different, but it still fits.

A WAF is supposed to inspect requests and block bad ones. But that only works if the WAF sees the request the same way the rest of the stack sees it.

If HTTP/2 request bodies are split across frames, and the inspection layer only evaluates part of the body before making a decision, then the security control is operating on an incomplete view.

That is the whole problem.

Security controls are not magic shields. They are parsers, policies, and defaults. And if the parser has a blind spot, the policy may be correct and still not protect you.

That is why “do we have WAF?” is not the right question.

The better question is: what does the WAF actually see?

Where does TLS terminate? Is HTTP/2 involved? Are request bodies accumulated before inspection? Does ALB behave the way the protection model assumes? Do the proxy, load balancer, WAF, app server, and backend framework all agree on what the request means?

Because when those layers disagree, attackers get room to move.

Then AWS Lambda MicroVMs are the other side of the same episode.

The first few stories are about unexpected execution paths. Lambda MicroVMs are about building a better place for execution to happen.

That matters because running untrusted code is becoming normal.

AI coding assistants need sandboxes. Users upload code. Agents generate scripts. Security tools analyze unknown payloads. Data platforms run user-defined logic. Developer platforms execute test cases. Plugin systems need isolation.

The question is no longer “will we run untrusted code?”

For a lot of teams, the better question is: where are we already running it, what can it reach, and how badly can it hurt us?

Containers are useful, but shared-kernel isolation is not always the right boundary for untrusted code. Full VMs are stronger, but heavier. Serverless is convenient, but not every sandbox use case fits the traditional request-response Lambda model.

So the product direction here is interesting even if you do not adopt this specific AWS feature. Sandboxing is becoming a first-class platform primitive.

That is the larger trend.

Execution is becoming something platform teams need to design, not something they inherit accidentally.

The lightning round keeps reinforcing the same point.

GitHub’s advisory database hitting record volume is a reminder that vulnerability management is not just about getting more feeds. It is about deciding what matters before the queue buries the team.

Git 2.55 is a reminder that developer platforms are infrastructure too. At enough scale, Git performance becomes engineering productivity.

Valkey 9.1 on ElastiCache is a reminder that cache engines are production systems. Memory efficiency, throughput, command behavior, and isolation matter when the cache sits directly in the request path.

And the Fable 5 callback is a reminder that AI dependency risk does not disappear just because the model came back online. If a model can go away for policy reasons and then return with new access rules or new behavior, that is still something reliability teams need to track.

That is why the closer lands where it does.

Execution is where trust becomes real.

A repo is not just source code if opening it can trigger an assistant, language server, or task runner. An IDE is not just an editor if it can run commands, load tools, and inherit credentials. A WAF is not just a checkbox if it only inspects part of the request. A sandbox is not optional if your product runs code from users, agents, or plugins. An advisory feed is not useful just because it is complete. Someone still has to decide what matters.

So the practical questions this week are pretty simple.

Where does code run? What can it reach? What can it read? What can it write? What authority does it inherit? What logs prove what happened? Who owns that boundary?

And if the answer is “we’re not sure,” that is the work.

Because before execution, trust is a policy.

After execution, trust is a blast radius.

---

Extra links worth including:

AWS security bulletin: Amazon Q / AWS Language Server CVEs
https://aws.amazon.com/security/security-bulletins/2026-047-aws/

JFrog: Hijacked npm packages using VS Code tasks
https://research.jfrog.com/post/hijacked-npm-vscode-tasks-blockchain/

AWS security bulletin: AWS WAF HTTP/2 inspection issues
https://aws.amazon.com/security/security-bulletins/2026-048-aws/

AWS Lambda MicroVMs
https://aws.amazon.com/blogs/aws/run-isolated-sandboxes-with-full-lifecycle-control-aws-lambda-introduces-microvms/

GitHub Advisory Database record volume
https://github.blog/security/supply-chain-security/inside-the-advisory-database-and-what-happens-when-vulnerability-volume-breaks-records/

Git 2.55 highlights
https://github.blog/open-source/git/highlights-from-git-2-55/

Amazon ElastiCache Valkey 9.1
https://aws.amazon.com/blogs/database/announcing-valkey-9-1-for-amazon-elasticache/

Claude Fable 5 and Mythos 5 model docs
https://platform.claude.com/docs/en/about-claude/models/introducing-claude-fable-5-and-claude-mythos-5

This week’s On Call Brief
https://learn.tellerstech.com/on-call-brief-news/2026-W27/

More Ship It Weekly episodes
https://shipitweekly.fm/

For this Conversations episode, the part that stuck with me is that AI security is not really one conversation.

It is a speed conversation.

It is a context conversation.

It is a fundamentals conversation.

And honestly, it is still very much a people conversation.

That is what I liked about talking with Kat Traxler from Vectra AI. She is not dismissing the AI security concern. There are real changes happening. Models are getting better at finding certain classes of bugs. Researchers can move faster. Bug hunters can narrow huge codebases down more quickly. Attackers can also use the same kind of tooling to move faster.

That part matters.

But the part that gets lost in a lot of the panic is that speed is not the same thing as understanding.

AI can help find suspicious code paths. It can help with injection bugs. It can help with RCE patterns. It can help generate queries, first drafts, summaries, and research direction. It can make the first pass through a messy problem less painful.

But it does not automatically know the threat model.

It does not always understand privilege.

It does not know what your users expect.

It does not know what your business logic means.

It does not know whether something is actually exploitable in your environment, with your data, your permissions, your deployment shape, and your weird internal assumptions.

That is where human judgment still matters.

And that is probably the most important theme of this episode.

The model can help find needles, but the human still has to know which needles matter.

That came up a few different ways. Bug hunting, writing, cloud analysis, IAM, prompt injection, insecure-by-design flaws. The pattern is pretty consistent. AI can accelerate parts of the work, but if you do not have the expertise to challenge it, constrain it, and validate it, it can make you faster in the wrong direction.

Kat’s example with VPC flow logs is a good one. If a model tells you that you can infer packet-level details from data that does not actually contain those fields, that answer might sound useful if you do not know better. But if you do know the system, you can push back and say, no, that is not possible. Here are the fields we actually have. Here is what can and cannot be inferred.

That is the difference between AI as leverage and AI as a confidence machine.

And that matters a lot in security.

Because security does not reward confident guesses.

The other thing that stood out to me is Kat’s pushback on the full zero-day apocalypse narrative.

There is a real issue underneath it. The time between vulnerability disclosure and exploitation has been shrinking. AI may compress that even further for certain classes of vulnerabilities. That is not fake. Security teams should pay attention to that.

But the jump from “attackers may weaponize faster” to “global technology collapses in a few months” is a much bigger leap.

And Kat’s point was basically, let’s spend more time on the “and then what?”

If the zero-day clock keeps shrinking, what actually happens?

Which teams are most exposed?

Which systems are most at risk?

Which attackers change behavior?

Which organizations need to respond differently?

And which problems are still the same boring problems they were yesterday?

Because that is the part that I think a lot of DevOps, SRE, platform, and security teams need to hear.

The scary future does not erase the current fundamentals.

Credentials still matter.

IAM still matters.

Misconfigurations still matter.

Known vulnerabilities still matter.

Patch management still matters.

Source code secrets still matter.

Human fatigue still matters.

Attackers are practical. They are going to take the lowest-friction path that works. A lot of the time, that path is not a cinematic AI-generated zero-day chain. It is a leaked key. A reused credential. An over-permissive role. A service account nobody remembered. A server that should have been patched six months ago.

That does not mean AI risk is overblown.

It means we need to hold both ideas at the same time.

AI may make certain security problems move faster.

But if your IAM is already a mess, your secrets are everywhere, your patching is inconsistent, and nobody owns the exposed attack surface, the future AI threat is not an excuse to ignore the thing already sitting on fire.

The IAM part of the conversation felt especially true.

IAM is hard because it looks like a technical service, but it is really where people, process, and technology collide.

It is credentials. It is authentication. It is access. It is human behavior. It is shortcuts. It is tired people. It is teams under pressure. It is old roles. It is permissions nobody wants to remove because something might break. It is attackers knowing that credential abuse is often the easiest way in.

That is why IAM stays painful.

Not because nobody understands least privilege as a concept.

Everyone understands the concept.

The hard part is making it work in a real organization where people are trying to ship software, troubleshoot production, rotate keys, grant access, clean up old permissions, and not break the thing that pays the bills.

So when Kat says IAM is really about people, I think that is right.

The insecure-by-design discussion was another useful thread.

A lot of AI security focus is on code-level vulnerabilities. Can the model find injection bugs? Can it find RCE? Can it find XSS? Can it narrow the search space?

That work matters.

But insecure-by-design flaws are different.

Those are not always bad lines of code. They are bad assumptions.

The AirTag example is a good way to think about it. The device may function as designed, but the design missed a real-world abuse case. The issue is not just technical correctness. It is whether the product accounts for how people can misuse it, who can be harmed, what customers expect, and what happens when the system is used in a way the builders did not intend.

That is a different class of security work.

And I suspect it is going to stay valuable, even as AI gets better at finding some code-level bugs.

Maybe especially then.

If AI makes some of the easier code-level searching faster, more of the important work may shift toward context, abuse paths, design flaws, threat modeling, and the uncomfortable question of what the system allows someone to do that we did not intend.

That is harder.

It is also probably more important.

So my takeaway from this episode is not “panic about AI security.”

It is also not “everything is fine.”

It is more like, keep your head.

Use AI where it helps. Let it speed up the annoying parts. Let it help with first passes, rough drafts, code search, query generation, pattern matching, and research workflows.

But do not outsource judgment.

Do not let the AI output become the security conclusion.

Do not confuse a confident answer with a correct one.

And do not let the excitement around future AI-driven attacks distract from the fundamentals that are still hurting teams right now.

Threat model the new stuff.

Patch the known stuff.

Clean up IAM.

Protect credentials.

Own your exposed attack surface.

Write down what your system actually promises.

And when AI gives you an answer, treat it like a starting point, not the finish line.

Because the future may move faster.

But the work still has to be done.

This episode is really about the control plane getting wider.

That sounds like a platform-engineering phrase, but it is becoming one of the more important ways to think about modern production systems.

A few years ago, when people said “control plane,” they usually meant something fairly specific. Kubernetes API server. Cloud API. CI/CD system. Maybe an internal deployment platform.

Now it is messier than that.

Your container runtime is part of the control plane because it decides how workloads actually start on the node.

Your database failover automation is part of the control plane because it decides whether recovery is safe or reckless.

Your AI incident-response agent is part of the control plane because it can inspect telemetry, summarize what changed, recommend action, and maybe someday trigger work directly.

Your Kubernetes API server egress path is part of the control plane because a stale route table or broken firewall path can stop admission webhooks, OIDC, and aggregated API calls from working.

Your credential revocation tooling is part of the control plane because compromised access has to be cut off fast.

Your cloud console is part of the control plane because operators still need a way to reach the environment during an incident.

Even object metadata starts to matter when data, AI, search, and agent workflows depend on understanding what an object is, not just where it lives.

That is the through-line in this episode.

containerd disclosed a batch of CRI plugin vulnerabilities, and the lesson is that Kubernetes security does not stop at pod specs, RBAC, admission control, or image scanning. Eventually the node runtime has to pull the image, unpack it, restore it, wire up devices, handle logs, and start the container. That runtime layer is not invisible plumbing. It is a trust boundary.

The Datadog PostgreSQL HA story is a different kind of control-plane lesson. Their gameday did not just ask whether PostgreSQL could fail over on Kubernetes. It exposed the harder question: can it fail over safely? If every standby is behind, promotion may be possible, but it may not be correct. And in databases, correct usually matters more than fast.

That is the part I love about the Datadog writeup. It is not the fantasy version of HA where automation magically fixes everything. It is the real version where replication lag, synchronous writes, RPO, RTO, and promotion safety all collide. Failover is only useful if the recovery path does not create a bigger problem.

The AWS DevOps Agent and Datadog MCP Server story pushes this same theme into AI operations. AI incident response is moving from demo to production workflow. That is exciting, but the question cannot just be “is the agent smart?” The better question is “what authority does it have?”

Can it only read?

Can it write?

Can it open tickets?

Can it trigger automation?

Can it roll back?

Can it page someone?

Can it make things worse quickly and very confidently?

That is the uncomfortable part. AI incident tooling can be genuinely useful, especially during the early chaos of an incident when everyone is jumping between dashboards, traces, logs, deploy history, and Slack threads. But once an agent sits near the operational control plane, it needs the same boring guardrails as any other production automation: least privilege, audit logs, approval boundaries, rollback rules, and a clear line between recommendation and execution.

Then there is EKS customer-routed control-plane egress, which is one of those features that sounds boring until you think through the failure modes. Routing Kubernetes API server outbound traffic through your own VPC is a real win for private and regulated environments. But it also means your route tables, security groups, NACLs, firewalls, and private connectivity can now become control-plane dependencies.

That is powerful.

It is also something you bring to a design review.

The lightning round kept hitting the same idea from different angles. GitHub credential revocation is incident-response infrastructure. AWS Console Private Access pulls more operator workflow behind private network boundaries. Vercel Connect points toward short-lived, task-scoped credentials for agents instead of long-lived secrets sitting around forever. S3 annotations make object metadata more directly attached, mutable, and queryable instead of living in another side table that drifts from reality.

Different stories, same shape.

Authority keeps moving.

Trust keeps spreading.

The blast radius keeps expanding.

And that is where Marc Brooker’s post on waiting fits so well as the closer.

Your dashboards may measure averages, but your users do not experience averages. They experience the time they spend waiting. A ten-minute outage and a ten-hour outage might both count as one incident in a tracker, but they do not feel the same to the people stuck inside them. A service with a decent average can still feel terrible if users keep landing in the tail.

That is the reliability lesson underneath the whole episode.

When the system breaks, users do not experience your architecture diagram. They do not care whether it was the runtime, the database, the AI agent, the credential system, the route table, the cloud console, or the metadata layer.

They experience waiting.

Waiting for a request.

Waiting for recovery.

Waiting for a deploy to stop failing.

Waiting for a credential to get revoked.

Waiting for the control plane to come back.

Waiting for someone to find the right context.

So the practical question is not just “is this system up?”

It is also:

Where is authority hiding?

What has to work before recovery can happen?

Which defaults are trusted?

Which control-plane paths are invisible?

Which tools can make changes?

Which systems can block deploys?

Which dependencies only show up when something breaks?

And most importantly, what does this feel like from the waiting side?

Because your dashboards measure the average.

Your users feel the wait.

---

Extra links worth including:

containerd CRI plugin vulnerabilities / AWS security bulletin
https://aws.amazon.com/security/security-bulletins/2026-046-aws/

Datadog: PostgreSQL high availability on Kubernetes
https://www.datadoghq.com/blog/engineering/postgresql-ha-kubernetes/

AWS DevOps Agent and Datadog MCP Server
https://aws.amazon.com/blogs/devops/production-ready-autonomous-incident-resolution-with-aws-devops-agent-now-ga-and-datadog-mcp-server/

Amazon EKS customer-routed control-plane egress
https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-control-plane-egress-through-your-vpc/

GitHub self-service credential revocation for incident response
https://github.blog/changelog/2026-06-24-self-service-credential-revocation-for-incident-response/

AWS Management Console Private Access
https://aws.amazon.com/about-aws/whats-new/2026/06/aws-management-console-private/

Vercel Connect
https://vercel.com/blog/introducing-vercel-connect

Amazon S3 annotations
https://aws.amazon.com/blogs/aws/amazon-s3-annotations-attach-rich-queryable-context-directly-to-your-objects/

Marc Brooker: Waiting, latency, MTTR, and the inspection paradox
https://brooker.co.za/blog/2026/06/19/waiting.html

This week’s On Call Brief
https://learn.tellerstech.com/on-call-brief-news/2026-W26/

More Ship It Weekly episodes
https://shipitweekly.fm/