AI, SaaS, and the real meaning of sensitivity labels
Sensitivity labels were designed to quietly shape how data is handled: who can see it, where it can go, and what controls apply. As organizations plug AI assistants and SaaS tools into their everyday workflows, those labels move from simple metadata to governance signals. These two stories explore what happens when the signals are missing, ignored, or not designed for the way people really work.
Story 1: The AI assistant that saw too much
A large enterprise rolled out a generative AI assistant to help employees search, summarize, and draft content from internal documents. The assistant was wired into a broad document index: incident reports, playbooks, postmortems, and customer communications.
On paper, the organization had a mature classification program. Incident reports involving regulated data were labeled “Restricted Customer Data”. Playbooks and internal guidance were marked “Internal Security Operations”. Access to the source systems themselves was reasonably controlled.
The AI assistant, however, treated the index as a single undifferentiated pool. It did not consume sensitivity labels as inputs to its access rules. To the model, a “Restricted Customer Data” incident report and an “Internal Security Operations” runbook were just tokens in the same search space.
A junior marketing analyst, curious about how often customer data incidents really happen, asked the assistant for a summary. The model generated a neat overview: incident counts, typical root causes, common weak points in the environment, and remediation themes, drawn from documents the analyst was never supposed to see in full.
Nothing was downloaded in the traditional sense. No file was emailed, no screenshot was taken. But a synthetic summary now existed in the analyst’s workspace, exposing patterns that had previously been confined to a small circle of responders and risk owners.
What actually failed?
The labeling program was technically correct; the incident reports carried the right sensitivity tags. The failure was architectural: the AI assistant operated with broader visibility than any individual user, and it was never taught to respect the organization’s labels as hard constraints.
In effect, the organization introduced a new super‑user that could traverse boundaries the classification program assumed would hold. Labels existed, but nothing downstream used them to decide what the model could see or answer for a given persona.
Story 2: The SaaS ecosystem that forgot about labels
In a fast‑moving SaaS company, core systems such as CRM, document storage, and email implemented sensitivity labeling. Contracts and pricing were “Confidential Commercial”, customer PII was “Restricted Customer Data”, internal roadmaps were “Internal Product”.
Over time, the go‑to tools for work shifted. Sales notes lived in a third‑party note‑taking app. Product teams discussed roadmaps in a collaboration platform. Support teams used browser extensions to capture screenshots and snippets into ticketing systems.
None of these tools understood the organization’s sensitivity labels. Data left the core systems and entered an ecosystem of SaaS tools as plain text and files without any classification or handling instructions attached.
A salesperson, preparing for an important renewal, exported a customer health summary from the CRM and pasted it into an unapproved dashboard tool. The summary included references to prior incidents, pricing concessions, and roadmap commitments, information all labeled “Confidential Commercial” in the original systems.
Months later, a partner reviewing that dashboard forwarded a screenshot to a third party as an example of how we track customer health, inadvertently sharing details the company considered highly sensitive. Every upstream system had done the right thing. The downstream SaaS tool simply did not know that sensitivity labels existed, and no controls were designed around its role in the data flow.
What actually failed?
The classification scheme assumed that core systems were the whole universe of work. It treated integrations and shadow SaaS as exceptions rather than as primary places where people think, write, and decide.
Labels were local, not portable. Once data crossed an API boundary or a copy‑and‑paste event, it became unclassified content in another tool with different defaults and different risk assumptions.
Lessons learned: designing labels for the way people really work
These stories are not about bad users or broken tools. They are about classification and sensitivity labeling programs that do not fully account for how AI and SaaS reshape access patterns, context, and boundaries.
- Labels must be governance signals, not decoration. If downstream systems such as search, AI, and SaaS do not consume sensitivity labels as inputs to authorization and policy, then labels are metadata without control power.
- Architect for the omniscient services. AI assistants and global search tools see more than any single user. They need explicit constraints: which label tiers they can index, which personas can query which labels, and how answers should be filtered or declined.
- Follow the data past the core systems. Mapping real data flows such as copy‑and‑paste, exports, browser extensions, and integrations is as important as designing the label taxonomy itself. Classify and control where people actually work, not just where policies were first written.
- Design for portability. Sensitivity labels should travel with content via APIs, file formats, and enforcement points, so that downstream tools can inherit and honor the original intent.
- Align labels with business consequences. A label should mean something specific in terms of harm, regulation, and decision‑making. That clarity is what allows engineers and operators to wire labels into controls without guessing.