One of an organization’s most valuable assets is its data. Unfortunately, many of the companies that are embracing artificial intelligence (AI) solutions within their organizations – approximately 88 percent of all organizations, if a recent AI study by global consulting firm McKinsey & Company is to be believed – are operating a bit faster and looser with this valuable asset than they probably understand or appreciate.
There are two likely reasons for this. First, AI tools are relatively new, and organizations are somewhat inexperienced working with them. This is a challenge, because AI is unlike traditional software applications and solutions that these organizations have adopted in the past. Second is the somewhat unstructured and organic adoption of AI that has happened across most organizations – being embraced by employees looking to be more efficient in their jobs before eventually being adopted and embraced by the organization, as a whole.
In fact, recent surveys and reports, including this report from Deloitte into AI trends in 2025, show that concerns about data security and AI governance are one of the single largest barriers to enterprise AI adoption. And for good reason.
A Whole New, Nondeterministic World
In the past, when an organization sent data to another solution or system, there was a well-established methodology for transmitting, securing, and using that data. That data would most likely be encrypted. It would be validated for integrity. And the application or system to which it was being sent would be tasked with completing a highly standardized, deterministic task with that data, which would not change it or affect its integrity in any way.
Also, the organization often owned these solutions or had a good understanding of data ownership from the SaaS platform providers via stringent terms that segmented and protected their data.
AI agents and models don’t work that way.
AI solutions are, by their nature, nondeterministic. It’s almost impossible to know 100 percent of the time what the system will do with an organization’s data. Worse, as today’s AI solution providers enhance their Large Language Models (LLMs) with additional capabilities – abilities like running code, giving access to other parts of the organization via MCP servers, or executing tool calls – the question of what an LLM does with an organization’s data only becomes larger and more difficult to predict.
And, unlike the SaaS and on-premises software solutions that organizations owned in the past, AI models use and learn from the data users shared with them, by design.
Ultimately, AI creates a whole new paradigm in which organizations no longer know with 100 percent certainty what will happen to their data once it is sent out. Yes, many providers have configuration settings that suggest they will not use user data to train their models, but how trustworthy are those statements?
Worse, some employees may not even be aware that they’re sharing this data outside of the walls of their organization. Given the natural language interface of chat solutions and the immaturity of most early users, there is a good chance that employees within an organization can be unwittingly and inadvertently sharing data with outside applications.
Traditional software applications and IT systems were adopted and secured by the IT team, which structured frameworks for how, when, and where data flowed. But AI growth is not just focused on engineers. Everyone within the organization can access and utilize AI tools, many of whom may not understand the underlying tech and how or where the data is used. When you consider that AI is effectively a “black box” that can collect and use data, call other tools, and reference other data sources, this becomes a real problem!
The uncertainty in how and where AI tools leverage an organization’s data is a real problem, especially when you consider that data is one of their most important and valuable resources. It’s an even bigger challenge in highly regulated industries, where data privacy and data use regulations and restrictions – regulations like Sarbanes-Oxley, HIPAA, and GDPR – could result in stiff penalties for organizations that leak personally identifiable information (PII) or other sensitive data.
Organizations in these industries – including financial services, healthcare, insurance, and even government agencies – might think they have their data locked down by simply denying access to AI tools or by implementing acceptable use policies for AI solutions. But history shows that restricting the use of AI solutions or simply putting policies on the books for how and when AI tools can and should be used isn’t enough to safeguard an organization’s data.
Restrict At Your Own Risk
In my previous experience with other disruptive technologies – particularly the cloud – I saw just how futile it is for organizations to try to restrict employees’ access to advanced tools and solutions. Technologies that make jobs easier will be used, regardless of whether the organization is ready for its employees to use them or not.
When I was working with Red Hat, I would interact with organizations all the time that would swear their employees were not using cloud solutions from Amazon Web Services (AWS). But I could see all of the Red Hat licenses that they were using through AWS. The CIO or CTO at these organizations didn’t know those instances existed.
This shows that if organizations don’t give their employees regulated access to AI, they’ll use it anyway. It’s too much of an “easy button” to ignore it.
You don’t have to take my word for it. A recent study by IBM showed that “shadow” or “rogue” use of AI tools is rampant across organizations.
“While 79 percent of full-time office workers said they use AI at work, only 25 percent rely on enterprise-grade AI tools, signaling a widening disconnect between employee expectations and enterprise readiness,” the study claims. “The rest rely on a mix of personal and employer tools (33 percent) or entirely on personal apps (21 percent).”
It’s clear that just saying “no” to AI usage isn’t going to cut it for most organizations. And, honestly, very few of them really want to.
Every organization wants its employees to work better, faster, and more efficiently. In fact, most IT departments are under extreme pressure from their senior leaders to make AI tools available to employees in an attempt to increase their productivity. Others look at AI usage as an essential skill for employees to learn for the future, and as a recruitment tool that will help entice a new AI-savvy future workforce to the organization.
This is why many organizations have already tailored their existing governance and acceptable use policies to include AI. In fact, if these policies had been written well in the first place, AI solution providers would simply be considered another vendor, with restrictions on which kinds of data should and should not be shared.
Unfortunately, guidance and policies without enforcement are effectively useless. This is particularly true for AI. Practically every application, operating system, and other software solution used across the enterprise has some AI functionality baked in. Also, modern AI Chat tools and solutions simply require that data be “copied and pasted” into a prompt, which every employee can do.
Today, Microsoft’s Copilot solution is being integrated into email applications and even employee desktops. This solution will read and summarize emails and typed notes on the screen – sometimes without even being asked to. But what happens when an employee is emailed sensitive data that AI should not be allowed to access? What happens when that information is discussed on a call, which is recorded, transcribed, and summarized by a 3rd party AI model like ChatGPT?
Simply putting policy and governance guardrails in place for appropriate AI usage isn’t enough to ensure that sensitive data stays within the enterprise. So, what is an organization supposed to do?
Evaluate and Block
To help ensure that sensitive data isn’t used inappropriately or leaked, some organizations have turned to niche AI solutions and vendors that promise to scrub PII. Or they’re leveraging AI solutions built specifically with regulations like HIPAA in mind. In other cases, they’re adopting point solutions to address AI vulnerabilities such as prompt injection.
In many instances, these solutions don’t guard against the inappropriate use of AI. This is a significant shortcoming. While it’s important that organizations stop the leakage of sensitive data, it’s also important to keep employees from using AI in inappropriate or unacceptable ways.
Also, many of these individual point solutions fail to pull back the covers and examine how AI solutions are being used. They don’t show their work or guarantee that things like PII redaction are being done.
Worse, leveraging individual point solutions for each and every potential vulnerability or regulation could ultimately result in a tangled, confusing Web of tools and solutions that need to be implemented and managed by already overtaxed AI professionals and IT departments.
What organizations really need is a single pane of glass or a single management dashboard where they can see all of the ways AI is being used across the enterprise and aggregate all of their disparate governance tools and solutions. Thankfully, a new generation of AI management tools, including the solution that we’ve built at Holokai, is working to give organizations the transparency and simplicity they need to oversee, manage, and regulate the use of AI across the enterprise.
These solutions detect prompt injection risks from employees or from 3rd parties, such as customers, partners, or malicious actors that are leveraging an organization’s AI tools. They also provide a single platform that aggregates much of the functionality and capabilities of these disparate point solutions, with the flexibility and scalability to expand as new solutions are introduced. For example, if the organization identifies an effective solution for detecting and blocking prompt injection, that solution can be implemented and managed within the organization’s AI management solution.
AI tools don’t operate like traditional software solutions. They’re nondeterministic and can expose an organization’s data to the outside world in ways that they couldn’t anticipate or plan for. Blocking the use of AI across the organization is a fool’s errand that will only result in rogue use of AI agents and models. And simply altering acceptable use policies will not be enough to protect sensitive data.
Modern AI management solutions can make it easier for organizations to monitor AI use across their enterprise and implement the security and data privacy tools they need to ensure sensitive data, PII, and other important information stay where they belong, even as AI adoption accelerates.
The author, David Kang, is the Chief Technology Officer at Holokai, which helps organizations navigate their AI Journey at a time of unprecedented technical innovation, sky-high expectations, and uncertain outcomes.