How to Govern AI Agents Effectively

Intro 0:01

Welcome to the She Said Privacy/He Said Security podcast. Like any good marriage, we will debate, evaluate, and sometimes quarrel about how privacy and security impact business in the 21st century.

Jodi Daniels 0:21

Hi, Jodi Daniels here. I’m the founder and CEO of Red Clover Advisors, a certified women’s privacy consultancy. I’m a privacy consultant and certified informational privacy professional, providing practical privacy advice to overwhelmed companies.

Justin Daniels 0:36

Hi, I am Justin Daniels. I am a shareholder and corporate M&A and tech transaction lawyer at the law firm Baker Donelson, advising companies in the deployment and scaling of technology. Since data is critical to every transaction, I help clients make informed business decisions while managing data privacy and cybersecurity risk. And when needed, I lead the legal cyber data breach response brigade.

Jodi Daniels 0:58

And this episode is brought to you by mine. Can boop. Okay, Red Clover Advisors. We help companies to comply with data privacy laws and establish customer trust, so that they can grow and nurture integrity. We work with companies in a variety of fields, including technology, e-commerce, professional services, and digital media. In short, we use data privacy to transform the way companies do business. Together, we’re creating a future where there’s greater trust between companies and consumers. To learn more and to check out our best-selling book, Data Reimagined: Building Trust One Byte at a Time, visit RedCloverAdvisors.com. Now, normally we talk about something different, but I know you’re so excited about this topic that I think we should just dive right on it. Let’s

Justin Daniels 1:42

go.

Jodi Daniels 1:43

That’s your turn.

Justin Daniels 1:44

Oh, all right.

Jodi Daniels 1:45

That was your hinting.

Justin Daniels 1:46

Oh, I understand. So today we have Camilo Artiga-Purcell, who is the general counsel at Kiteworks, where he leads the company’s global legal operations across commercial contracts, data privacy, and cybersecurity compliance, M and A litigation and regulatory strategy. Welcome.

Camilo Artiga-Purcell 2:05

It’s nice to be here. Nice to meet you both.

Jodi Daniels 2:07

So you have a very wide purview of of different responsibilities across the company. But tell us a little bit about how you got there in your career journey.

Camilo Artiga-Purcell 2:20

Happy to. So I took a more circuitous path than most to the the general counsel seat. I actually began as a litigator and a trial attorney specifically for an excess of a decade. I started in the traditional law firm setting, and then went off and and built my own law office. And at some point, probably at the eight-year mark, had a lot of repeat commercial clients that that kept getting into problems, and so I pivoted from just doing the pure play trial work to consulting to try to avert more litigation, and then at some point in that journey, I I started working on retainer with Kiteworks. The stars aligned, and I I joined as the general counsel. Now I, in in my world as a trial attorney, going in house meant sort of a lifestyle change, and that is not something I ever wanted. I sort of like the the stress and and the hit in the grindstone. It works because it’s a very active practice, and we don’t farm out all the interesting work. So selfishly, I’m intellectually stimulated. It’s it’s been a great journey thus far.

Jodi Daniels 3:38

We’ve actually seen a couple of those different types of journeys along the way, and I do think though you might be one of the few who enjoys the high stress.

Camilo Artiga-Purcell 3:48

It keeps you on your toes. keeps keeps life interesting.

Jodi Daniels 3:52

Indeed, it does.

Justin Daniels 3:53

There’s a certain belief in yourself that comes when you have to perform under a high stress situation. You can do it again and again, repeatedly, you just get certain confidence. I can’t quite explain. The

Jodi Daniels 4:04

world needs all different types of people, so this this works. Okay, this is important

Justin Daniels 4:10

Indeed,

Jodi Daniels 4:12

your turn.

Justin Daniels 4:12

Oh, my turn. So,

Jodi Daniels 4:14

someone needs more lunch. Not the one you didn’t eat yet.

Justin Daniels 4:18

1. It’s fine. So, as companies deploy more AI agents, what new legal risks should privacy, security, and legal teams be thinking about?

Camilo Artiga-Purcell 4:29

I think it’s the the existing risks and the the existing legal frameworks, but on steroids because instead of managing 100 employees, 500 employees. You’re now managing 5000, 10,000 plus agents who are more difficult to govern unless you’re extremely intentional about putting in guardrails. But the reaction. Is we’ve got a growing patchwork of state regulations governing AI. We have some, but unequivocally unclear direction from the federal government, and so my view is the the current regulatory framework, statutory and common law, nationally in the U.S. and also internationally, particularly in Europe, with with GDPR and the various privacy laws there, it all you should assume it all applies with full force, not just to the workforce, but also to AI agents, which should be viewed as employees of the company.

Jodi Daniels 5:40

As you’re thinking about all of these employees of the company trying to audit them, make sure that we understand what it is that they are doing, apply to those different regulations. In your mind, what does an evidence quality audit trail look like for an AI agent?

Camilo Artiga-Purcell 6:02

Again, I think it’s an extension of best practices for for human employees. So the before I even get to the AI agent, the the first question becomes at the data layer: How are you making sure that when you funnel sensitive data into an LLM, you do so safely to avoid leakage. Implementing air gap systems would be important, and then once you unleash the AI agents against that data, just as you would with employees, you need to build out policies, and so you don’t leave it up to chance. So, for example, if you’re in a regulated institution and you’re barred from sending certain types of data to Iran, to China, to North Korea, you would implement geo fencing, apply it to all the agents, so that they’re physically blocked from going to those jurisdictions, and so you build out similar policies depending on the type of data you’re you’re dealing with to make sure that you can control what the agents do. The other part of it is how do you prove that you’ve done this? And so again, just like you would with an employee’s email address, you would put a wrapper around the agent so that you actually bake out an audit trail, so you can have a spreadsheet on the back end that identifies the agent, what was the prompt, and then where did it go, what what did it do, so you can prove out that it was in compliance. the The other side of that is, of course, if it was not compliant, if it goes rogue, you’d have a system to to show the audit trail of exactly what it did, when it did, where it went.

Jodi Daniels 7:54

I can imagine some people listening are newer to this process, and they they would like to maybe start asking the questions of how they would do it in their company. Are there any tools or starting point that you might suggest for them?

Camilo Artiga-Purcell 8:11

I don’t want this to become a sales pitch. Kiteworks, of course, does have the the infrastructure to handle secure AI gateway as well as tools that that are the wrapper that you put around agents to govern on the front end implement policies and on the back end generate an audit trail, but again, I don’t want to turn this into a sales pitch. We have that infrastructure built, and it’s it’s on sale now. But there are other companies that are they’re building out the same thing. You mentioned that folks might be new to it and and sort of dipping their toe in the water. The beauty about this is it’s moving quickly, but it’s still in its infancy. So I don’t think anybody should feel like they’re you shouldn’t feel inertia because you’re behind. You just got to jump in the deep end, sort out what tools are out there, test drive them, and move quickly because otherwise you’re going to be left behind very quickly. But but I think I wouldn’t worry about being behind. I was behind six months ago, and now we’re moving at warp speed.

Justin Daniels 9:22

So I wanted to ask you a follow-up question about that, and it’s something we talked about in the pre-show, which is from my standpoint, one of the things I’m seeing with the pressure that’s about to be put to bear, where prompts aren’t so free anymore because you know companies like Anthropic are going to go public and need to recoup their investment. There’s been a lot of talk about, well, we’re going to use open source models. We’re going to build our own proprietary AI. But now, as you alluded to, well, if we use the Chinese models, which are less, now you get into a real issue over. Well, I have to prove where that data goes, or I need to make sure it’s not. Going to China, or you know, I have to prove that I’m not really using that. So, do you think maybe with this inflection point that the kinds of things that you’re talking about with geo fencing, as well as really having a wrapper around all these LLMs, is just really going to accelerate?

Camilo Artiga-Purcell 10:20

It will accelerate. You you put your your your finger on a very sensitive issue, especially from our standpoint dealing with customers that are in regulated industries and customer bases that are serving regulated industry. I’ll I’ll be blunt: we would not be comfortable under any circumstances using a model that bakes in infrastructure built in China. There’s a history of espionage. There’s a history of IP theft. I mean, there’s there’s no way to sugarcoat it. It’s not a risk we think is palatable for any regulated or regulated tangent company, and we we just won’t do it. I I totally understand that things are moving quickly, and there’s this balancing act between driving revenue and and teeing Up a humongous liquidity event, but I think it’s wrong-headed to balance that against security. I think it can only end poorly. And again, each institution will make its own decisions, but we will not play in those waters.

Jodi Daniels 11:40

Justin, thoughts? You have lots of strong views.

Justin Daniels 11:44

I couldn’t agree with him more. I just think it’s really interesting that when companies are out there faced with the choice to use some of these models versus what we talked about, you’re seeing another pattern, which is the revenue, the market share is winning out over what I think is common sense. That what you articulated with the security risk-I mean, that’s just plain to see-and yet decisions are being made, which I think must be motivated by market share, fear of missing out, and trying to control your costs because token use has gone through the roof because everyone is using

Camilo Artiga-Purcell 12:20

1. Yeah, I’m sure you will hear arguments around the the ability to bake in security infrastructure, basically poo-pooing the risk. But but the reality is, you don’t know what you don’t know, and with the the speed at which these models are are developing, there is no way, in our view, to to safely implement these models. I get that they’re powerful. I get that at scale they can be a whole lot cheaper if you’re a consumer and you’re and you’re just using it to to create something for fun with your friends in connection with some NFL game or or calendar in the next picnic, fine. But for anybody that’s serious about security and anybody that’s dealing with sensitive data, it’s beyond reckless.

Justin Daniels 13:14

It’s funny you say that because the way I’ve come around to look at it is, you can vibe code, and if you’re using an underlying model, say from China, and you build some kind of software, I’ve come around to look at it, and this is Justin’s pet theory. I call that the fruit of the poisonous LLM. I borrowed it from from the Supreme Court, and I put it into this context. But that’s what I don’t think people are thinking. Oh well, I’m not using the Chinese the Chinese LLM. I can put it onto another LLM, but no, you built it off that. And what I’m concerned about is, was it coded in a way that if you put in a certain prompt, malware gets injected into your code that you’re trying to build, and it’s persistent. It’s not easy to find.

Camilo Artiga-Purcell 13:59

It’s as powerful as AI is, it’s it’s the new shiny toy, and we all need to be using it. But this is not a new pattern. There there’s a almost a FOMO quality. You want to use the latest, most powerful model. You obviously don’t want to spend too much money, and so the whole security discussion that should always be happening at the beginning, unfortunately, as we as we see with with all these new developments, is going to happen at the back end. Unfortunately, after we have a series of calamities, a series of hugging face type incidents, and this is-I wish it wasn’t that way, but that’s that’s sort of the way we see this playing out. Companies will get there, and security infrastructure will be put in place, but I fear there will be quite a bit of collateral damage because. Was the the impulse to consume a lot cheaply today, at least, is is more compelling for a lot of institutions than safety first, which invariably slows the train down a bit. I

Jodi Daniels 15:15

think one of the pieces I’ll add is a reminder because one company might be evaluating a particular AI vendor, and we’re having this conversation of of trying to understand where is that going, but remembering that that vendor is also potentially using different tools, and to really understand the the deep processor and sub processor chain, and figuring out how to stay on top of that, which brings me to controls and AI governance, and so to kind of wrap it all around, some of the other areas are things like access logs and purpose limitations, which is you know a prop a popular privacy term. I have a lot of peas going on today. So, what would be some of the other AI governance controls that you might suggest, and how that might connect to e-discovery and any kind of existing data governance practices companies might have?

Camilo Artiga-Purcell 16:14

For for e-discovery, I think it’s simple. So long as you, so long as you’re intentional on the front end about putting in policies and tracking what your agents are doing, on the back end you will have the audit trail. You literally will have a spreadsheet that you can produce that that shows the entire every touch point of the agent. It it no different than current e-discovery. You get sued, you put a litigation hold in, and then in discovery, the exercise is to go pull all the relevant or arguably relevant data. Unfortunately, where discovery became a nightmare with the introduction of instant messenger and just the the massive quantity of emails. It’s going to get even worse now because we’re going to have a new data set, which will be these AI logs, and because they’re moving at the speed of light, the quantity of data is going to be immense, but the exercise will be to plug it into your e-discovery system, vet it, produce it. That there will be new experts, there will be new jobs, experts that deal with explaining the audit trail. But I think it’ll be plug and play into the existing e-discovery and discovery infrastructure, but the the variable sort of dovetailing into the prior discussion is you have to be intentional about putting in that infrastructure at the beginning because if you wait until the agent goes rogue and and completes the task, but in doing so trespasses into some competitor system. If you don’t have the audit trail, you’re going to have no clue on on what it actually did. To actually recreate the path will will be immensely difficult, especially if this is something that happened six months ago. I mean, it will be a mess. It will be a mess, and I think that’s where we’re headed. And then companies will put in systems to to create an audit trail, and it won’t be easy. You’ll have a huge data set, but it’ll live with the existing e-discovery system. I think the in terms of governance and controls. I think I would start with the most elemental, and that is having early and frequent discussions across the employee base to educate them on what they can do, educating them on how to use these tools responsibly, and making sure there’s clarity on the do’s and don’ts. Because again, there’s an impulse to use whatever new tool is out there because it can make work easier without thinking about the consequences. I think the the surest way to get into problems is to have an uneducated workforce around AI systems and responsible use of AI. I think the first and easiest thing to do is trainings, but then have frequent conversations because this is developing in real time.

Justin Daniels 19:36

I guess another thing I wanted to ask you about AI governance, and we touched on this in our pre-show is organizations have a real need for all of their employees to really become conversant with AI, but at the same time as we talked about, we’re coming to this inflection point about how much the models will cost for prompts and outputs and whatnot. And I’d love to have you share the questions that your organization and your team is asking about. Hey, we want people to use AI, but maybe not everyone needs to use the Ferrari model for an email. But yet, we want them to use it, and maybe how that, what those questions are, and then how does that kind of bleed through to what AI governance looks like?

Camilo Artiga-Purcell 20:23

Yeah, it’s a fantastic question, and I full transparency. We’re in the midst of answering it, so that a year ago the AI spend would have been sub 50,000 a month today, without getting too specific, it’s in excess of $200,000 a month just to Anthropic alone. And initially, the exercise was to get folks excited about how you could use AI for each of us to make ourselves more efficient, to make ourselves smarter and faster. And in that context, there really were no guardrails. It was you figure out how to use it, identify for each discipline, map all the inefficiencies across your work streams, and start to build two agents to to make yourself smarter and faster. Within the first, I’d call it four to six months, there was different adoption across the the institution, but it was hard for us to tell. Okay, we’ve got a super user, but is it actually an efficient use of the tool? In our case, Claude. In other words, do you need to use Fable for the workstream, or could Opus Five suffice? Could Sonnet suffice, and so that that’s where we are now. It’s if you’re in the engineering department, it’s palatable to use deeper inference models to to do the highly sophisticated work. If you’re in a discipline where you’re drafting emails or creating high level pitch decks, it’s just not worth the spend. So we’re in the process of mapping, trying to understand exactly what the the more active users are actually using it for, and then we will create rules around which which models are accessible depending on your discipline. It’s not a one size fits all, but but we’re still fine tuning, and I think what we’re seeing because of the competition you alluded to earlier among the various LLMs outside of the turbocharged models like Fable and whatever will come in the next days, weeks, months for the the sort of mid tier models, I think there’s going to be quite a bit of downward pressure in pricing, which will help further adoption. But but the sweet spot, I don’t have the answer yet. I don’t think the LLMs have the answer quite yet.

Jodi Daniels 23:16

I think that’s an interesting observation about the downward pressure. So good old market economics at work because right we we’re seeing costs going up in some some way for them to be able to capture, but then you’re going to have so much pressure from all these companies who have built, and they’re not going to be able to absorb, so they’re going to switch. It just I think it’s going to be very interesting time to figure out how that’s going to play out.

Justin Daniels 23:36

I think the really interesting point both of you are bringing up is the debate you’re going to have between the closed sourced LLMs like Anthropic and OpenAI versus the people who want open source because I think the closed source people want it their way so they can have a moat and charge more whereas the open source people want it their way to where they think pricing will be different and I think it’s really important to understand that because once you understand the incentives behind the argument people are making, you understand why it’s going there, and maybe it’s not so altruistic.

Camilo Artiga-Purcell 24:07

Yeah, no, that that’s a fair point. I I think the we’re very pro open, so I think both ecosystems will coexist, and how they coexist is is yet to be seen. But for both the open source backers and the the closed source LLMs that the LLMs that are ahead of the curve already and have a vested interest in staying ahead of everybody else, in both camps, invariably self interest is involved. Neither is altruistic.

Jodi Daniels 24:46

With everything that you know about data and privacy, security risk, what is the best personal privacy or security tip you would offer the audience? As it

Camilo Artiga-Purcell 25:04

relates to to the use of AI and AI agents in particular, I would say that, at least for me, the best advice, and it’s it’s sort of a creating the habit of always checking the outputs, always making sure that you’re using primary sources, and and for any work that matters, going through the exercise of building a checklist on the back end where you can pressure test the results. So if you’re you’re dealing with a large data set, you don’t need to read every single word, but you need to at random create a checklist and then do a deep dive so you’re you’re you’re certain that to a 95% likelihood the output is is correct. But I think the other piece is is educating the team. We we spend a lot of time and money in creating what, in our view, is the most secure infrastructure available. But ultimately, the the soft spot is going to be an employee who doesn’t use the tool responsibly, and by that I don’t just mean people. I also mean an employee that is your AI agent that doesn’t behave responsibly in trying to satisfy the prompt. So for human beings, It’s education for the AI agents. It’s education, but really putting in guardrails because if you don’t do that, because it wants to please you and provide the the answer you asked it, unless it has the guardrails, it’s going to do anything, including violating applicable law, to get to the answer, so at a high level, that those are my views on on on two security nuggets that I think can make everybody better.

Jodi Daniels 27:10

I appreciate you offering those. Thank you.

Justin Daniels 27:13

So, when you’re not thinking about AI agents, security, and privacy, what do you like to do for fun?

Camilo Artiga-Purcell 27:20

Oh gosh, for fun. Well, with the with the NFL season upon us, I am a lifelong 49 er fan, and and my church is Levi’s Stadium. So I there’s nothing there’s nothing more fun than meeting up with with old friends, having a beer, shooting the breeze, and then watching watching the 40 Niners hopefully crush the the rest of the NFC West. Other than that, I’m I’m pretty simple. I when I’m not working, I spend a bunch of time with with my two and a half year old, and it’s while it has its challenges, it’s it’s a very special time in life, and it’s not lost on me. So I’m trying to soak that up and spend as much time as I can with her on on the weekends and in the evenings. Go going to the park, going to the creek, going to the ice cream shop probably more often than we should. These are the the simple pleasures of life, and I I’m very blessed to be able

Jodi Daniels 28:21

to do it. Too much ice cream. I I have a daughter who would say there’s no such thing as too much ice cream.

Camilo Artiga-Purcell 28:27

I mean, the the dentist told me ice cream’s okay. It’s it’s the it’s the gummies and the chocolate that’s the problem. So I’m sticking by that no matter what the pediatrician. There you

Jodi Daniels 2

8:37Have two scoops, no toppings. That’s the way this works.

Camilo Artiga-Purcell 28:40

Exactly.

Jodi Daniels 28:42

Oh, we’re so glad that you joined us. If people would like to connect with you or learn more about Kiteworks, where should they go?

Camilo Artiga-Purcell 28:49

They they can email me directly at Camilo.apursell@kiteworks.com. I’d be happy to to chat with anyone.

Jodi Daniels 29:01

Wonderful. Well, thank you again. We really appreciate it.

Camilo Artiga-Purcell 29:04

Thank you both. Take care of a wonderful day.

Outro 29:10

Thanks for listening to the She Said Privacy/He Said Security podcast. If you haven’t already, be sure to click subscribe to get future episodes and check us out on LinkedIn. See you next time.