Ship It Conversations: Ang Chen on Project Vera, AI Cloud Emulation, and Safer Infrastructure Testing

Transcript

Everybody wants AI to help run infrastructure.

A lot fewer people are asking where that AI is

allowed to fail. Because the hard part is not

getting an agent to suggest a change. The hard

part is making sure that change can be tested,

challenged, and debugged before anything touches

real cloud infrastructure. And that is what made

this conversation interesting to me. Not just

the AI angle. The idea that before we trust agents

with real systems, they may need a real training

ground first. Thank you. Hey, I'm Brian Teller.

I work in DevOps and SRE, and I run Teller's

Tech. Ship It Weekly is where I filter the noise

and focus on what actually matters when you are

the one running infrastructure and owning reliability.

Most weeks, it's a quick news recap. In between

those, I do interview episodes with people building

tools, systems, and ideas that could actually

change how this work gets done. Today is one

of those conversations. I'm joined by Ang Chen.

associate professor at the University of Michigan.

He's working on Project Vera, which is now being

positioned as a high -fidelity multi -cloud emulator

you can run locally on your laptop with support

for AWS EC2 and GCP compute. At a practical level,

the pitch is pretty straightforward. Test cloud

infrastructure locally, use standard tooling,

avoid real accounts, real spend, and real blast

radius while you are iterating. But the bigger

idea behind Vera is what really got my attention.

Ang frames this as part of a longer -term vision

for giving AI agents a safe learning environment.

or what he calls a kind of world model for digital

systems, where they can build operational intelligence

before ever touching real infrastructure. So

in this conversation, we get into what high fidelity

really means, how Vera works at the API layer,

how it can sit under workflows that already use

CLI tools, SDKs, or Terraform, and why that matters

if you want faster feedback without pointing

tests at the real cloud. We also get into the

skeptical operator questions. How close does

something like this actually need to be before

you trust it? Where is it strong today and where

is it still early? And if AI is going to play

a bigger role in infrastructure, what kind of

safety layers should exist first? That's the

real conversation here. Not whether AI can generate

infra work. Whether it can be forced to prove

itself somewhere safe before it earns access

to the real thing. If you like these kinds of

conversations, follow the show wherever you listen.

Subscribe on YouTube and check out ShipItWeekly

.fm or TellersTech .com for more episodes, show

notes, and everything else that I'm building.

All right, let's jump in. Today, I'm joined by

Ang Chen, an associate professor at the University

of Michigan. He's working on Project Vera, which

is basically trying to build a high -fidelity

cloud emulator using AI agents, starting with

EC2. Ang, thank you for joining me. Thank you,

Brian. I'm excited to be here. So tell me about

Project Vera. What is it? It's an effort that

automatically generates a digital twin of your

cloud deployment. DevOps can be very tricky to

get right. And we don't want any downtime or

security issues when we actually push the program

to the cloud. So you can think of it as a sandbox.

That's a digital copy of your actual infrastructure.

Within this sandbox, the DevOps programs can

be tested. They can be debugged. You can even

deploy an AI agent to play with the sandbox and

get to know more about your deployment without

actually reaching into the actual deployment

itself. And what's interesting about the sandbox

is that it's actually generated by AI agent itself.

Have an AI agent that reads. the cloud documentation,

and it could also observe traces and logs about

the deployment. And it uses a very specialized

program synthesis pipeline to generate an emulator

framework. The emulator framework will mimic

the behavior of EC2, for instance, in terms of

how to respond to a certain call, what should

be the responses and formats in a very high fidelity

manner. And the same idea generalizes to other

services in AWS and it generalizes to other clouds

as well. Actually, we're building it for Azure

and GCP and other clouds as well. So it's an

agent building a simulator of the cloud. And

on that cloud, on that simulator, the DevOps

engineers can do a lot of their works much easier.

What's the target audience for a tool like this?

Right. It would be primarily for DevOps engineers

who... want to test their programs in the sandbox.

The DevOps engineer can deploy their programs

in the sandbox and observe the behavior and debug

their programs before they push it to the actual

cloud. And that sandbox can also be used to support

DevOps like AI assistance. AI is getting very

powerful every day, but we often don't want the

AI to directly work on the infrastructure. because

it could hallucinate. So having an AI agent testing

its proposed actions in the sandbox before putting

it to the cloud would be another use case of

the sandbox. Is it interfacing with like IAC,

like a Terraform or CloudFormation? Or how does

that integrate? Right. The simulator emulates

the cloud at the API level. Basically, every

API that creates virtual machines and subnets

is captured here. So basically, it can support

SDK scripts, but it can also support CloudFormation

and Terraform because eventually they all call

into the APIs. And in the release that we have,

we have like CRI test cases that mimic Amazon,

but also Terraform programs that can be booted

on this emulator. When you say high fidelity,

what does that mean in practice? Right. It means

that there are two key properties of this emulator

because this is generated by an AI co -developer.

so to speak that reads the cloud documentation

and test against the cloud we want to make sure

that this is not vulnerable to hallucination

ais are getting very good but they still have

hallucination and we have two ways to prevent

this from happening and the first is the ai agent

that we have built that's behind vera is using

formal abstractions is using formal methods and

verification to make sure that the code eliminates

classes of hallucination problems. So it's built

to be correct by construction without suffering

from arbitrary errors that an AI model would

otherwise introduce. And the second is that the

AI agent also takes this simulator and strategically

tests this against Amazon. Because this emulator

is generated by the agent, the agent understands

the inner workings of the emulator and it can

understand what might be some edge cases and

what might be some places where strategic testing

would be helpful. So the agent also takes this

emulator and produce traces and send it to the

cloud, observe whether the behaviors are the

same or not. And if they're same, that's what

we mean by high fidelity. And if there are discrepancies,

the AI agent will then consume these two traces

and automatically patch the emulator so that

in the next test case, they will be aligned with

each other. Interesting. I guess I'm curious,

how does it get around the non -deterministic

behavior of an AI or an LLM specifically? Right.

And that's a very good question. That's where

the formal abstractions come in. Instead of having

the AI write code in a freeform style, We actually

have a lot of scaffolding. That's the key part.

The structure of the emulator is a deterministic

framework. And what we ask the AI to do is essentially

fill in the blanks that we have left out instead

of being creative about writing everything about

the emulator. So it's a combination of neural

and symbolic methods where the neural framework

constrains the behavior. and it's fully deterministic.

And there are strategic parts where the AI needs

to read the documentation and understand what

it's supposed to do. And it's only filling in

these blanks in a way that's constrained by the

scaffolding. So it's like a spec then that you're

built around, okay. Exactly. Or in Cursor or

Kiro, it's like a plan file that it's reading.

Is it like a pre -prompt or is it more specific

than that? It's more specific than that. So we

use a special kind of phone methods that builds

classes and abstractions, almost like a template.

And the template has a very well -defined structure.

And we know that the structure cannot go wrong

because it's deterministic. But the structure

also has certain stops. And the stops are where

the AI agents will generate code and insert them

into. So it's more specific than a pre -prompt,

almost like... a class that can be inherited

and can be turned into a compute instance, can

be turned into a subnet, a firewall, and so forth.

So can you walk me through the shape of the system?

Like if I'm calling an EC2 API, what's happening

behind the scenes? Right. So if it's calling

into the EC2 API, so that API will be captured

by the emulator framework and it will create

a class, so to speak. that captures the behaviors

as specified in the EC2 virtual machine documentation.

For instance, there is a run instance which creates

the virtual machine. And you could destroy it,

you could attach disks to it, and so forth. And

that will trigger some state modification, almost

like IAC, where Terraform contains the state.

So here, the emulator mimics that workflow, and

it also contains state. But now we have a virtual

machine. And the virtual machine could have a

specific name. And if there's another API that

attaches a disk to the virtual machine, the emulator

will also capture that by modifying the internal

state. So it is a hierarchy of these services

where you could instantiate a virtual machine

and the virtual machine could be contained in

a VPC. So when you're updating the virtual machine,

for instance, the emulator knows that it also

must. updates the VPC. So talking about state,

how do you deal with weird edge cases like eventual

consistency, retries, throttling, quota errors,

that sort of thing? Right. So the emulator framework

itself is generated by an AI agent that reads

the cloud documentation. So the cloud documentation

describes the key behaviors of the cloud, but

it doesn't describe everything. So the question

that you ask is a very important class of problems.

which are not fully documented in the documentation.

As an example, eventual consistency and consistent

guarantees are often not described in detail.

But the API behaviors, how it should perform,

is documented very extensively. So what we are

doing here is that we are taking the emulator

and bootstrap it to a fully functional emulator,

but doesn't capture some of the nuances regarding

throttling, rate limiting, consistency. But we

have another simulator. in the backend that can

supply some of these semantics. So this functional

emulation can be, if there's a call into an API,

that API, we can also emulate latency for that

API throughout behaviors and quotas. So there

are an orthogonal subsystem that supplies that

kind of intricate detail to the emulator. That's

a great question. What's the success bar? Is

it like same response, same timing, same failure

mode? Yeah. So there are two milestones. The

first milestone is that it should enable the

same inputs and outputs for the APIs so that

DevOps engineers doesn't have to actually go

to the cloud to understand whether their program

is working. So then we just tested it and...

observe the actions in this emulator. And the

second milestone is that actually this emulator

can help with DevOps perform better debugging

than the cloud can. And the reason is that when

the cloud has an error, it gives you some trace,

but that trace is often verbose. It doesn't really

help with pinpointing which line of code is problematic

in your Terraform file or in your SDK file. Because

there's an AI agent living in the sandbox. The

agent can analyze the traces and produce better

debugging information and even pinpoint the problems

in Terraform. So the second milestone is actually

to do better debugging than what the cloud can

do. Interesting. So how do you prove that it's

not lying to me? That is a heart of the question.

How do we make sure that this emulator is actually

producing the same responses in the first release?

in the github we have more than 200 test cases

and these are cri command lines that you would

type into aws and we run a test between vera

and an existing emulator so what we've shown

is that vera is already doing much better than

existing emulators but the same set of test cases

I've also shown that Vera sometimes fails to

produce the same behavior because this is a agent

that continuously improves itself. And the first

release gets a 70 % based on our measurement.

And by this agent, we have another version that's

continuous running and improving itself until

it hits all the test cases. So one way that people

test it is to use a leading emulator called local

stack. Local stack is this really nice tool.

that emulates AWS APIs. It's not one -to -one.

I've found it's good in some ways, but yeah,

it's... It's not one -to -one. It's close enough

to enable classes of DevOps testing. In the open

source release, actually, we did a comparison

between Vera and local stack. So what we found

is that Vera covers more than 70 % of the cases,

whereas local stack covers 40%. So the first

version of Vera is already performing. quite

well in that regard. And we also have another

version that's continuously improving itself.

And the goal is to simulate the behavior of the

cloud to Terraform and DevOps programs, almost

like the Turing test. The ultimate goal is when

we run a DevOps program against the simulator

versus against the cloud, the DevOps program

doesn't feel any difference. It doesn't necessarily

mean that has to be line by line. character by

character the same regarding the logs and the

outputs, but we want it to be high fidelity enough

that DevOps engineers can test it thoroughly

in this simulator. So if I'm a platform team,

where would I actually plug this in? Local dev

versus CI versus prepod, validation, like what

would be a good first step? Right. One way of

using this is to integrate it to the CI -CD pipeline.

When there are code changes, there's a new Terraform

file. The agent can take the changes and validate

it in the sandbox first and suggest changes to

the program if there are errors and fix these

errors and generate corrections for the DevOps

engineers so that this would be integrated to

the CICD before it's actually pushed to the cloud.

Is there any use cases where maybe it's not well

suited for yet? Maybe it doesn't have enough

testing around it or validation. So there are

two things that we know about the limitations

of Vera. The first limitation is that it doesn't

yet cover all resources in AWS. It does cover

EC2, which is a key service. There's also a lot

more beyond EC2. That's the first limitation

that we know. And the second limitation is that

the current version, the current version doesn't

do some of the things that we... are thinking

about. For instance, I've talked about AI -based

debugging suggestions to DevOps engineers. So

that tooling is not fully ready yet. So currently,

if there's a bug, error doesn't automatically

diagnose the bug for you, which is part of our

ongoing plan. For the first limitation, which

is that it doesn't support all APIs and it doesn't

support customization. For instance, it doesn't

automatically understand how a specific deployment

is like. That deployment for an enterprise may

not all use all the APIs. They may use the APIs

in a very specialized way. So these kind of customizations

are also not there yet, but they are on our agenda.

So it sounds like EC2 is its area where you've

had a lot of focus and it seems like you have

trusted output there. What's the nastiest EC2

edge case that you've had to emulate? There's

a very interesting edge case that we have found

in this exercise, which is that sometimes our

AI co -developer that writes the emulator uses

one types of string formats, like a camel cases,

where it's easy to make format the same string

differently. And that's very interesting because

a Terraform program expects a certain type of

format. And if it's formatted slightly differently,

then Terraform program won't run. So there we

had to create specialized directions for the

agent so that you would only produce camel cases

when it's supposed to be camel cases. In other

cases, produce snake cases and so forth. I thought

that was a very interesting and unexpected edge

case where the initial version of Vera didn't

produce the exact response and we had to do extra

engineering to make that align. Interesting.

Do you think clouds will ever make official emulators

good enough or is learned emulation the path?

Right. One thing that's very special about cloud

emulation compared to other types of emulators

is that the cloud is a moving target. There are

new services every week and there are API changes.

Many of these changes will introduce different

behaviors. So it's building an emulator for the

cloud. Our experience is that there are two key

challenges. One is that the size of the cloud

is so big and there are so many different clouds

with different behaviors. Beyond AWS, we've also

investigated Azure and GCP, which is on our agenda

as well. They all have different APIs. They have

different behaviors. The emulation for one cloud

doesn't really generalize to the other. The second

is that the APIs go through constant evolution.

Because the clouds need, they want to stay competitive.

They're introducing new services, new ways of

using these services. We really believe that

learned and AI agent built emulator is the path.

Because the AI agent doesn't have to spend a

lot more extra effort once the emulator framework

is there. It still has to align the emulator

periodically. Whenever there's an API change,

the agent has to understand what has changed.

It has to generate strategic test cases. to realign

that API, but it doesn't have to do everything

from scratch. So this agent can keep up with

the changes that happen in the cloud and it can

gradually expand to different clouds. So this

is an ever -expanding emulator that can catch

up with the speed of the cloud. And that's something

that we are very excited about regarding this

learned emulation. Have you done much as far

as GCP training yet? I'm just curious because...

The IAM approach in GCP is completely different

than the IAM approach in AWS. Right. Or even

like Cloud Run versus Lambda is also completely

different. Fundamentally different services,

although the same general idea or same general

focus. Yeah. And that is a very good question.

The clouds call the same service differently.

So they're almost aliases. So here is where AI

will shine. Because as long as we can... make

the AI understand. There are certain concepts

across clouds that are similar. Instances are

called virtual machines in a different cloud.

Then there are certain knowledge base in the

AI that can transfer from one cloud to another.

So there's a core of the learned knowledge that

can transfer, but not everything. The APIs are

still different and the services do not always

have a counterpart across clouds. What I'm excited

about this approach is that some core knowledge

of the cloud can be transferred so that when

we're building the second emulator for GCP, it

will be much faster than the first one for AWS,

where it has to learn everything, all the concepts

from scratch. Where can people find more information

about Vera? We have an open source release called

Project Vera. And it's project -vera .github

.io, where we have an open source release, as

well as the publication that we had over the

years that eventually led to this paper, to this

simulator. And that could be a good source of

information that not only is about the release.

but also the rationale behind the release and

the specific approach that we take in designing

Vera and other tools that we have built in the

past couple of years surrounding AIOps and DevOps.

And we are looking for contributors to help us

improve Vera. And if you're interested in contributing

to the open source release or contributing new

ideas, or if you have a service that you would

like to see emulated, this is something that

we are very excited to help you with. What's

the license model for Vera? currently? It's under

MIT license in open source. That's good to hear.

Too many new open source projects like to limit

their open source initiatives, which is a little

frustrating. Right. It's fully open source under

MIT license. Thanks. Well, thank you, Ang, for

coming on. Really appreciate it. Thank you very

much, Brian. It's great to be here. All right.

That's my conversation with Ang Chen. My biggest

takeaway for this one is that AI for infrastructure

gets a lot more believable when it has to survive

a sandbox first. That's what makes Vera interesting

to me. Not just that it emulates cloud behavior,

but that it is trying to create a local, high

-fidelity environment where cloud workflows can

be tested without real credentials, real billing,

or real production risk. And since we recorded

this, the project has clearly kept moving. It's

now being presented publicly as a multi -cloud

emulator, not just an EC2 -focused idea, with

AWS and GCP support and a stronger public story

around local testing and safer iteration. I also

liked that the bigger vision was not just AI

does ops. It was more grounded than that. Give

agents a training ground. Let them learn inside

something rule -based. See how they behave, see

where they fail, then decide what, if anything,

Thanks for listening, and I'll see you later

this week.

Ship It Conversations: Ang Chen on Project Vera, AI Cloud Emulation, and Safer Infrastructure Testing

Watch this episode here

Chapters

Transcript

Catch This Episode

Host Commentary

Show Notes

👤 Guest

More from Ship It Weekly

CISA’s GitHub Leak, AI Root Cause Analysis, Copilot Agents, Claude Code in CI/CD, and Kubernetes Seccomp Risk

Ship It Conversations: Gareth Kersey on IaCConf 2026, AI, and Corey Quinn’s Terraform Keynote

EKS Rollbacks, GitHub Actions Supply Chain Attacks, AI Agentjacking, CloudWatch Log Alarms, and Why Safety Nets Don’t Replace Ownership

containerd CRI Vulnerabilities, Datadog PostgreSQL HA on Kubernetes, AWS DevOps Agent with Datadog MCP Server, EKS Control Plane Egress, and Why Users Feel the Wait

Get the next episode in your inbox