I had the honor of keynoting Information Security Summit (ISS) Cleveland 2026 this year. The topic was When Context Becomes Control, and it focused on how to build security into trust boundaries around the LLM for automated workflows. After the talk, an attendee said my talk made sense but they were still struggling to understand where and how to begin this kind of work, especially with executives pushing for massive AI adoption immediately.
That’s a great question. Defenders have a long history of threat modeling systems and testing misuse cases involving attackers. In an AI workflow, we also need to account for model output becoming part of the decision path: the surrounding software may use it to choose information, save guidance, or take action. An ordinary mistake can cause harm without an attacker being involved.
By converting or creating AI-driven workflows, we can materially change the risk model of an existing workflow without realizing it, because a check or control that worked when a person read a document or completed a form may not work the same way when AI becomes part of that process.
Every new or revised workflow should be threat modeled. Once the “what are we building” and “what can go wrong” steps are complete, the concept of trust boundaries, the points where information can gain more influence or authority, can be a powerful foundation for addressing the question of “what can we do about it?” If you’re new to this, my earlier explanation of agent harnesses and trust boundaries is a useful starting point. Each transition gives us a place to ask what should be trusted and what must be checked.
For the workflows below, I’ll focus on Runtime and Actions, Knowledge Integrity and Retrieval, and Memory and Persistence: where information becomes action, authoritative context, or reusable guidance.
To put that into practice, ask:
- What should the agent be allowed to access and do?
- Which information should the AI be allowed to rely on?
- What should the system be allowed to remember and reuse?
In addition to asking how someone could attack the system through intentional context manipulation, also ask what could go wrong through an ordinary mistake, missing context, or an outdated source.
What should the agent be allowed to access and do?
Start with the job you want done, then work out which systems the agent needs to reach and what it needs permission to do. Shane Mac, CEO of XMTP Labs, wanted a useful check on his personal finances. He set up a Grok Bot agent, “Personal CFO,” with read-only access to his bank accounts and instructions to send its reports privately. In an October 9, 2026, account, Mac said the October 1 report instead went to his company’s Slack, exposing personal financial information to colleagues.
The intended destination was his Grok Bot group, “My Personal Exec Team.” The report went to the company’s “Exec-team” Slack chat. Mac said the Grok team’s investigation found that his agents shared connections: the Slack account he’d connected for another agent was also available to his CFO agent. Mac also reported a permission fix; the incident and explanation here are based on his account, not an independently verified technical postmortem.
Now let’s threat model that workflow. What are we building? A private monthly financial report. What can go wrong? The system can read the right data, produce a useful report, and still send it to the wrong people. The impact is a confidentiality breach even if no money moves and every number in the report is accurate.
Read-only banking access is a sensible control. It limits what the agent can change at the bank. It doesn’t determine where the agent can send information it has read. The identity it runs under, credentials it can reach, and tool permissions it has determine its access; naming it “Personal CFO” doesn’t create that separation.
Access control underpins all of these trust boundaries. A private finance agent shouldn’t have authority to post to a work-environment Slack at all. So the first protection is to isolate its identity, connections, and credentials from the work environment, including any work credentials reachable through shared storage.
Now follow the information: bank data becomes a generated report, then a request to send it through a connected tool. That handoff is the Runtime and Actions boundary, where the disclosure occurred. For messaging services the agent is authorized to use, a second layer can bind the reporting task to an approved destination ID and have the software controlling the send independently check that destination before acting. That check reinforces access isolation; it doesn’t replace it. A similar-looking channel name mustn’t be enough to authorize disclosure.
Here’s how we could test the controls we’ve designed. First, verify that the finance agent can’t use the work Slack connection or reach its credentials. Then, within the messaging services it is allowed to use, test two similarly named destinations and check that a send to the wrong one is blocked. If approval is needed, show the person the actual service and destination, not just a friendly name. We can keep the useful monthly report without making its privacy depend on the model choosing the right chat every time.
Which information should the AI be allowed to rely on?
Next, decide which sources are allowed to influence an answer. Here’s a real example from a company that will remain anonymous. The company had created a policy agent for compliance that was used by internal employees in addition to being part of a larger workflow automation rollout that depended on it for downstream policy assessment and action decisions.
Traditionally, the company had stored draft policies and live policies in the same directory without any incident. People knew to reference the version marked <current> and to ignore the <draft> and <expired> versions. When a developer called a policy in an automated workflow, they pointed it to the <current> version.
But moving those workflows into ones driven by AI introduced a problem in a trust boundary that the implementers hadn’t considered.
Retrieval-augmented generation, or RAG, supplies retrieved material to a model so it can use that information when producing an answer. But a passage can be relevant to the question and still come from the wrong version of a policy.
You can see how quickly this could spiral in an example like this. A user (human or automated) wants to know if a $750 expense needs manager approval. The current policy requires approval above $500, but, with inflation, a draft proposes raising that threshold to $1,000. An AI retrieving from the shared repository might answer no based on the draft, even though the active policy says yes.
Keeping files in one folder doesn’t automatically cause a failure. The problem is assuming that the way people recognize an authoritative document will carry over without being made part of the retrieval process. To the model, a draft label is just more text. It might weigh it, or it might not. We shouldn’t make its interpretation of that label our security control.
This is the Knowledge Integrity and Retrieval boundary: deciding which source information can be trusted and supplied to the model. The organization needs a defined way to provide only the current policy to the AI agent. Then the software handling retrieval needs to enforce selection of the approved, currently effective version that applies to the question, before sending passages to the model.
Once the trust boundary fix has been defined, there are multiple ways to implement it. The company could have a current policy repository and point the agent only at that. Or the software handling retrieval could use metadata filtering on approval status and effective dates maintained by the policy owner, allowing only the applicable approved version into the context window. A recent date alone doesn’t make a policy authoritative. When a policy changes or is retired, that change also needs to reach the search index and its stored passages; changing the original file doesn’t necessarily remove stale copies from retrieval.
What should the system be allowed to remember and reuse?
Also decide whether an answer should become reusable guidance, and what must be checked before it’s used again. Suppose the answer that a $750 expense needs no approval is stored in the agent’s memory and is used as reference for future questions. A later response may reuse it without consulting the policy at all. Now we’ve crossed the Memory and Persistence boundary: a wrong answer has become information that influences future work and actions.
In this instance, fixing the original source may not fix the information already copied from it and entered into the agent’s memory.
For a workflow that saves and reuses guidance, the controls need to extend to those saved outputs. One option is to store the source policy’s ID, version, and effective date alongside the answer. Then, when that policy changes, the workflow must mark dependent answers as stale and block their reuse until they’ve been checked against the approved policy. Those tags make the answers traceable; they don’t invalidate anything by themselves. Another option is to require a current-policy lookup each time the answer depends on what’s in force now. If no valid source is available, or the sources conflict, withhold a definitive answer and refer the question to the policy owner.
Applying this to your own workflows
These three questions provide an approach that can be used with any workflow you’re creating for or moving to AI.
Start with what you’re doing and why. Then ask what can go wrong and how much it would cost. Follow the information through the new process. Look for the point where it becomes trusted guidance, a saved record, or permission to do something. Finally, decide what you can do about it. Decide what must be checked, where, and how you’ll test it.
We can secure agentic workflows, but we can’t expect them to secure themselves.
This week, pick one workflow that’s already moving to AI and work with the business owners to trace its information through these three questions, and agree who’s accountable if something goes wrong. When the pressure is to adopt AI immediately, that gives you a concrete place to start the business conversation for building security in.
