Ship It Conversations: Human-in-the-Loop Fixer Bots and AI Guardrails in CI/CD (with Gracious James)

Transcript

Hey, I'm Brian Teller. I work in DevOps and SRE,

and I run Teller's Tech. Ship It Weekly is where

I filter the noise and focus on what actually

matters when you're the one running infrastructure

and owning reliability. Most weeks, it's a quick

news recap. In between those, I drop interview

episodes with folks who are actually building

in the space. Today is one of those interviews.

We're talking with Gracious James about TARS,

his human -in -the -loop fixer bot wired into

CICD. We get into how he segments incident response

into sub workflows, how he keeps agents from

turning into SSH with a chat bot, and the guardrails

he thinks are non -negotiable if you're going

to let AI anywhere near production. Today, I'm

joined by Gracious James. He's been building

TARS, a human in the loop fixer bot wired into

CICD. And we're going to talk about what it takes

to make AI automation safe enough for real teams.

Gracious, thank you for joining me. Thanks, Brian.

Great to be here. Like you do it. I'm doing really

well. Thanks for asking. So I'm really interested

in this TARS bot that you built. Can you give

me a thesis of what you were actually building

with TARS? And why is human in the loop the core

idea? All right. So I am a software engineer.

And when I coded and pushed my code into, you

know, CI CD pipelines, I saw that it Took a lot

of time to get through a lot of teams, especially

the DevOps ones. So I started looking at the

processes of how DevOps go through, the incident

response and everything like that. There's a

lot of steps, manual steps that each and every

person has to cover through. Going through it,

I realized that all of these things are very

manual, very repetitive. And don't get me wrong,

everything is important and can break the system

at any point. So having a human in the loop is

very important. But looking at the processes,

I also see that all of these things can be automated.

So having an AI agent where human in the loop

approach increasingly helps us to fasten the

processes is what got me going into all of this.

Interesting. So what problem were you trying

to solve that like existing tooling didn't handle?

Right. So all of this, all of the tools that

I used into this workflow. are existing tools,

only that these are not custom made for, you

know, my requirement, my company or whatever

it is. So I had to custom make the workflow using

multiple tools from different, different domains.

So I custom made, you know, the SS access in

which there are, you know, validated SSH commands

that my AI agent can utilize to find out what

is going on into my Docker containers. to get

a proper system status report. And in order to

do a comprehensive check, I compare those logs

or the errors or whatever it is that the Docker

is pointing out to my previous GitHub comments,

be it the previous ones, the latest ones, or

whatever it is, I compare them and get a comprehensive

check. And based on those checks, it helps me

to figure out where I should narrow my focus

to. And once I do that, I kind of understand

what are the fixes that I should be figuring

out in a very short incident response time that

I get. That makes sense. Okay, so this was originally

Friday, and then you morphed into TARS, right?

Yes. Can you walk me through this evolution from

Friday? Like, what was Friday and what is TARS?

Well, Friday in itself is an Aniton workflow.

Now, Aniton is a... high -level abstract API,

sorry, multiple API application system where

we can connect different programs, subsections,

and sub -workflows in which we take the data

from one workflow, one action, and use it for

the next process and step -by -step so that based

on our custom input, we get a custom output which

we utilize for something. But Friday in itself

used a lot of tools, like some of the tools I

myself created and named, like visit development

server, visit production server, visit this API

container, that container. And based on all of

this, my LLM creates a system status report in

which I get to know whether some container is

down or up or what is going on, if everything

is working fine. And if anything is down. Then

it sends me a report via Telegram. And based

on that report, it's kind of an alarm going off

like in a plane's cockpit. And I am the pilot.

All I see that, okay, Friday is telling me something

is wrong over there. So it's my duty to go and

evaluate, check what's going on over there. So

yeah, it's kind of an alarm alert system. Interesting.

How do you keep this from turning into like a

chatbot just with like shell access? Okay, so

yeah. Everything is revolving around guardrails.

So when it comes to prompt, the AI agent for

doing system checks has its own prompt, and the

prompt involves not just doing any actions. So

it's very limited to commands like Docker PS,

Docker logs, the errors, and all of this information

that it can gather to make a comprehensive update

as to what the system is going through. It actually

cannot do commands that are harmful for the system.

And that's given in the prompt. Now I get your

question as to like, the LLM can go ahead and

do anything if it has the SSH access. For that,

what I have included is that in the tools like

visit development server or production server,

those are sub workflows. And those sub workflows.

don't just contain the SSH access. It has some

validation parameters. It won't just let the

LLM run any command it wants. It has validation

checks based on what I have programmed it for.

For logs, it has only Docker PS, Docker logs,

and it can never actually go and maybe Docker

stop or Docker restart because all of those commands

I have deliberately blocked in that sub workflow.

So it is kind of a manual override over there.

But yeah, that is a limitation that I have translated

as a guardrail. That makes sense. So, OK, speaking

on guardrails a little bit more online, you had

mentioned explicit rollback phases. What do those

look like? OK, rollback phases. So, well, as

a software engineer, when I do push some version

into my development branch or from development

to production, there's a lot of chances like.

after the CI, CD testing and integration, something

does break when it actually goes out into the

real world. So what we do know always as a software

engineer is that the previous version, well,

right, the previous version was pretty stable.

So our go -to response is, you know, of course,

fixing everything and putting it into the right

place. But if it's going to take time, if we

know based on the system status check. that,

okay, this error is a big one and it's going

to take time rather than, you know, just sitting

on it for a couple of days and, you know, just

letting the clients know that, okay, I'm going

to take my own time and do this. I'm just going

to roll back to the previous stable version so

that their work don't get, you know, hindered

or whatever. You get it, right? Oh, that makes

sense. Okay, so given the guardrails that you

have in place, where do you require... human

approval or is there human approval like in the

process yeah there is there is the human approval

uh does not come during the system status check

so i don't really tell the ai agent you know

where to go to read the data where to go to debug

the code or whatever it is i have given it a

set of instructions like you can check the docker

logs you can check the terminal logs you can

check the logging system that i have implemented

per code base you can check the previous github

commits to correlate as to what's going wrong

but i don't really say it to you know go and

check over here i think it might be wrong over

there it doesn't do that what the human in the

loop approach where it comes in is to do some

kind of action now the action involves you know

pressing some kind of button or running a pipeline

or a workflow that actually changes some things

in the system. All of these things, the system

checks and everything is just reading the data

and creating a status report. But where it actually

comes to change something, change in the system,

that's where the human in the loop approach comes

in. And without my, you know, verbal API key,

me actually telling the agent to run this pipeline,

run this workflow. It won't really do that. Okay,

so you had mentioned too that you had prompts

with like guardrails. What are the hard never

run this rules that you have set up? So it is

kind of a corollary. What I'm trying to say is

without a specific phrase, like run the CI CD

pipeline to deploy this, this version. This is

the exact phrase it requires to run the CI CD

pipeline. So in my prompt, I have written it

so that it. looks for the specific phrase to

actually activate that function. So without it,

without an actual phrase, without those actual

wordings, it can't run it. And that's in the

prompt, I get that. But as I told you before,

as a guardrail, I have put it also into the sub

workflow so that it doesn't just run anything.

Once Friday gives me the comprehensive system

check, I have two options usually. One is to

ask it again for a comprehensive system check

where it has access to GitHub repositories and

the GitHub actions. And that's when TARS activates

because Friday doesn't have access to the GitHub,

TARS does. So TARS goes through the normal Docker

logs and compares it with the GitHub commits

and what might have gone wrong. And it gives

me a proper report as to this is the area that

I need to focus in. Now, it does ask me for a

response as to what to do next, because after

all, something is wrong in the system. So it's

waiting for my approval or some kind of message

that I need it to get. So I can again ask it

to do some kind of comprehensive check. But if

I'm good with whatever information it has given

me, what I'll do is I can tell it to, you know,

run the CI CD pipeline to redeploy the previous

stable version, the previous GitHub commit. So

what it does is that based on this specific phrase,

it activates the CI CD pipeline because in the

sub workflow of the CI CD. pipeline, I have actually

written that in order to run this, you need the

specific phrase, run the CSED pipeline to redeploy

the previous stable version. So without this

phrase, so if this phrase involves like, okay,

run the pipeline without the CSED or run the

pipeline that I last deployed in the GitHub actions,

it won't ever run because it does not match that

specific phrase. And that's kind of rudimentary,

but it gives me more, you know. believable focus

as to, you know, it wouldn't just run anything

just because I wrote it. Like, it is not up to

its comprehension. That makes sense. Okay, so

let's say that a team wants to adopt human -in

-the -loop AI SRE automation. Where do you think

they should start? Okay, they should start, honestly,

where they feel that a lot of steps that they

do are not just rudimentary, but repetitive in

tasks. Because that's where I started. I saw

that the incident response procedure in itself

has a lot of manual and repetitive steps. And

not that I wanted to automate all of it, because

after all, it requires access to SSH and a lot

of system. And I can't just let an LLM run wild

into it. So kind of segmenting. Yeah, that's

the right word. Segmenting the procedure into

different steps. converting those steps into

sub workflows and integrating all of those sub

workflows into a pipeline one by one where i

have the control between the input of one sub

workflow to the output of other one so that i

get in i keep in control as to what the whole

picture is without actually letting one llm know

what the entire big picture is so that it can

manipulate is the go -to for integrating these

type of tools. How would you pitch human -in

-the -loop AI SRE automation to someone that's

maybe more skeptical? All right. So if a person

has their own skepticism about AI actions, LLMs,

thinking on their own, having access to all of

these tools, I get that. I completely get that

because that's where I started as well. But when

you delve deep into how an LLM works, it works

on an input and an output. So as long as you

can use that particular stage of giving an input

to an LLM and phrasing that output in a way that

I want to see and using that output into another

LLM so that that LLM only gets the input that

I am giving it, keeps the big picture to myself.

and never the AI letting it know what the big

picture is. So you obviously introduced this

because you're trying to, you know, reduce toil.

You're trying to iterate quicker. What was the

first thing that you used this AI tool to help

solve? Okay, so... Where'd you start? Yeah, it

never started as a fixer bot. It actually started

just on the basis of, you know, go into the system

and tell me what went wrong and why did the Docker

container stop? That's all. What I used to do

is creating a simple LLM workflow in which I

copied the output of the Docker logs, pasted

it into the LLM, letting it know whatever the

data I have, I'm giving it to you, tell me what

went wrong. So then it explains me what went

wrong one by one. And that's when I thought that,

okay, this is one portion of it. Why not I use

the same portion for every single container that

I have, but I want to automate it so that all

I tell it is that, okay, go to this container

and check it out rather than me copy pasting

the logs and putting it and whatever it is. So

for that, I needed to give it access to the terminal,

right? So that's where the SSH access came in.

And that became a sub workflow because I still

needed to keep it safe so that the LLM just can't,

you know, write anything. I put the guardrail

over there so that it can only write stuff like

Docker PS or Docker logs. And it did the same

thing for me, told me what is going wrong or

how the system is working. And that's where it

started. And that's where it saved tons of time,

me just copy pasting stuff, me just looking in

the wrong places, because I have to look to all

of those containers to see what is going wrong.

So instead of that, I know exactly where I need

to narrow my focus. That's where it all started.

And then you just iterated on that and started

adding more features. Yeah, yeah, absolutely.

What does future development look like for TARS?

Like, let's say the next three, six months, assuming

you have the runway and you have the time, what

features would you like to add? That is a really

good question. Well, TARS in itself is, I don't

want to say limited. Limited is not the right

word because two, three months ago, I was pretty

astonished as to what TARS could do with the

tools that it had. And that itself was a huge

accomplishment over there. But now after a couple

of months when I use it regularly, I feel that

I could go more. So I could give it access to

more tools. I can give it access to the infrastructure,

maybe Terraform and go from understanding what

resources my system needs. And if it's lagging

somewhere, what is the focus that I should maybe

increase some resources somewhere or maybe give

it access to. change stuff in Terraform or something

like that. That's Terraform side of things, infrastructure.

And that's one more tool to think about other

than just, you know, GitHub actions and SSH access.

Let's say you have teams that are interested

in experimenting with agents in CICD, but they're

apprehensive. Do you have any advice for how

to get started or where to start? Yep, yep. So

it never starts with an LLM, I'll tell you that.

Yes, we are trying to embed AI into it, but it

never starts with an LLM. It always starts with...

actually getting your thoughts and putting it

into a paper, putting it into steps, logical

steps that I can program first and then take

that program and understand how an LLM will be

effective in what positions. Because if you don't

really have a work plan or a game plan to begin

with, then the AI will also be as confused as

you were. Yeah. That's where I'll start. So what's

your take on fully autonomous AI ops agents?

It's scary. Because I myself am not a very big

fan of, you know, I don't want to say it out

loud, but I don't really believe a lot in experience

counts. Because I believe if you're smart enough

to do the smart thing, then sometimes even you

can, you know, outgrow experienced people. If

you know where to look and how things work very

logically instead of, you know, just gut feeling.

Because experienced people tend to do that more.

And that's where the AI and LLM lacks. It's kind

of a hypocritical thing to say that thinking

about the AI and LLM, always taking the smart

choice instead of, you know, just having lots

of experience, just a human like a human has.

But what I'm trying to say is that I still believe

in the gut feeling that I truly hate in a lot

of people. And that gut feeling is never going

to be at least. With the information that I know

about AI and LLMs, it has never proved to have

a conscience or a gut feeling that even if it

sees all the data pointing to one thing, maybe

a human knows that where exactly the error is,

but the LLM will always go towards the probable

cause instead of the gut feeling. So that's what

the... That's where LLMs lack. And that's why

they can never replace humans because we have

that creative aspect to us. And it's just a machine

at the end of the day. Yeah, that's fair. Okay,

so going back to guardrails, what do you think?

We've talked about guardrails a lot. We've talked

about AI and... autonomy a lot. What would you

say is the single most important guardrail for

an AI fixer bot? In terms of maybe single most,

I would say validating every output that an LLM

gives before you use that output for anything

else. Yeah, that's fair. Okay, so closing thoughts,

just curious, what would be like one of the main

reasons to start using AI and CICD? What can

that buy you? Well, AI is like an assistant.

And it works to automate a lot of repetitive

tasks that humans don't want to do. And that

primarily saves time. And time saved is money

saved. But whenever we do tend to automate these

things, we tend to get carried away in a way

that, okay, I can automate this. I can automate

that. I don't need you. I don't need that. I

can put all of this into code. Well and good,

but... Anything can go wrong when an AI hallucinates.

Yeah, for sure. All right, so closing out, what

advice do you have for people using AI in their

pipelines? Well, I understand there are a lot

of disadvantages for a full -scale AI taking

over a traditional system, but don't let that

hamper you. Don't let that stop you from understanding

your own process, understanding your own thoughts,

writing it down. writing a program and, you know,

running along with it because everyone gets their

own time and you know how to utilize that time.

So you can always stop at a point, introspect

as to what you have done till now, how it's going

to affect the future, how it has changed your

past, how it's going to help other people, how

it's going to hamper other people and take all

of these things into confidence and building

the next step. Makes sense. Appreciate it. Thank

you, Gracious, for coming on. Thanks a lot, Brian.

It was great being on this podcast with you.

All right. That's the conversation with Gracious.

I really liked his angle of designing the process

first and only then dropping LLMs in as helpers,

not magic. If this episode was useful, share

it with a platform, SRE, or DevOps friend who's

been playing with AI agents and trying not to

blow up prod. Hit follow wherever you listen

so you don't miss the weekly news recaps plus

these guest interviews. We'll be back with a

regular Ship It Weekly news episode later this

week. See you then. Thank you.

Ship It Conversations: Human-in-the-Loop Fixer Bots and AI Guardrails in CI/CD (with Gracious James)

Chapters

Transcript

Catch This Episode

Host Commentary

Show Notes

More from Ship It Weekly

CISA’s GitHub Leak, AI Root Cause Analysis, Copilot Agents, Claude Code in CI/CD, and Kubernetes Seccomp Risk

Ship It Conversations: Gareth Kersey on IaCConf 2026, AI, and Corey Quinn’s Terraform Keynote

EKS Rollbacks, GitHub Actions Supply Chain Attacks, AI Agentjacking, CloudWatch Log Alarms, and Why Safety Nets Don’t Replace Ownership

containerd CRI Vulnerabilities, Datadog PostgreSQL HA on Kubernetes, AWS DevOps Agent with Datadog MCP Server, EKS Control Plane Egress, and Why Users Feel the Wait

Get the next episode in your inbox