Why AI Is Forcing Privacy And Security Teams Back Together — And What That Means For Your Organization

One of the most consequential structural decisions facing organizations today is whether to continue operating privacy and security as separate functions or to begin integrating them in anticipation of the AI-driven regulatory and operational landscape that is rapidly taking shape. Having spent years advising clients on both HIPAA-era compliance frameworks and the new generation of AI governance requirements, I want to share some thoughts on why this question has become urgent and what the answer likely means for your org chart, your budgets, and your risk exposure.

The HIPAA Precedent: We Used to Get This Right

It is worth remembering that when HIPAA was enacted in the late 1990s and its implementing regulations rolled out in the early 2000s, the framework treated privacy and security as two sides of the same coin. The Privacy Rule and the Security Rule were distinct but deeply intertwined, reflecting an understanding that you cannot have one without the other. For a time, that integrated thinking influenced how organizations built their compliance programs.

But somewhere along the way, privacy and security drifted apart in many organizations, becoming separate teams, separate budgets, separate reporting lines, and separate regulatory conversations. Artificial intelligence is now forcing a reckoning with that drift, and the organizations that get ahead of the reintegration curve will be better positioned — legally, operationally, and reputationally — than those that do not.

AI Systems Collapse the Privacy-Security Boundary

Traditional data governance drew a relatively clean line: security protected the perimeter and the infrastructure, while privacy governed what you did with the data once it was inside. AI systems obliterate that distinction. A large language model trained on sensitive data does not merely store that data; it internalizes it in ways that make extraction, inference, and reconstruction possible even without a traditional "breach." The security of the model is the privacy of the individuals whose data it consumed. These are no longer separable concerns.
For clients evaluating their internal structures, this means that any team responsible for AI model governance needs fluency in both domains. A privacy professional who does not understand model architecture, and a security engineer who does not understand data subject rights, will each see only half the picture.

The Threat Model Has Changed

When we wrote security policies a decade ago, we were primarily worried about unauthorized external access and insider threats involving discrete data sets. In an AI world, the threat model expands dramatically. Prompt injection, model inversion attacks, membership inference, and data poisoning all represent novel vectors where a security vulnerability directly produces a privacy harm. Privacy teams need to understand these threats, and security teams need to understand why they matter from a rights and regulatory perspective. 

If your current organizational structure routes these issues through separate channels with separate escalation paths, you are creating latency in your response capabilities at the exact moment when speed matters most.

Regulatory Convergence Is Coming Whether You Are Ready or Not

Look at the EU AI Act, the draft updates to the NIST AI Risk Management Framework, and the state-level AI legislation proliferating across the United States. These regimes do not respect the organizational silos we have built. They impose requirements that sound like security obligations — robustness, integrity, monitoring — but are motivated by privacy and civil rights concerns. If your privacy team and your security team are learning about these rules separately and building separate compliance responses, you are doubling your work and creating gaps.

From a practical standpoint, this means that regulatory mapping exercises, gap analyses, and compliance roadmaps for AI governance should be joint undertakings from the outset. Clients who stand up separate AI compliance workstreams within their privacy and security functions are almost certainly going to discover redundancies and inconsistencies down the road — and those inconsistencies are precisely the kind of thing regulators and plaintiffs' counsel look for.

Data Minimization Now Requires Security Architecture Decisions

Privacy professionals have long championed data minimization, but in an AI context, minimization is not merely a policy choice — it is an engineering and security architecture decision. Techniques like federated learning, differential privacy, synthetic data generation, and on-device processing are simultaneously privacy-enhancing technologies and security design patterns. Neither team can implement these alone. The CISO and the CPO need a shared vocabulary and a shared roadmap.

For organizations evaluating how to structure their teams, this is perhaps the strongest argument for integrated reporting lines or, at minimum, formalized joint governance mechanisms. The technology decisions that will define your privacy posture over the next five years are being made by engineers, and those engineers need coordinated guidance from both functions.

Incident Response No Longer Fits Neatly into One Lane

When an AI system hallucinates personal information, or when a fine-tuned model is found to be memorizing and regurgitating training data, is that a security incident or a privacy incident? The answer is both, and organizations that run separate playbooks for breach response and privacy complaints will find themselves flat-footed. You need unified incident taxonomies, unified escalation paths, and unified communication strategies.

This is an area where we are actively advising clients to revisit their incident response plans. The 72-hour notification windows under GDPR, the evolving state breach notification requirements, and the emerging AI-specific incident reporting obligations under frameworks like the EU AI Act all demand a coordinated response that draws on both privacy expertise and security forensics simultaneously.

What Should You Do Now?

The organizations that will navigate the next wave of AI regulation most effectively are those that begin integrating their privacy and security functions now — not necessarily by merging them into a single team, but by creating shared governance structures, joint reporting to senior leadership, unified risk registers, and cross-trained personnel. The era of clean separation between these disciplines is ending, and the sooner your organizational structure reflects that reality, the better positioned you will be.

To view Foley Hoag's Security, Privacy and The Law Blog please click here

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.