Fighting Agents with Agents

Why the AI Era Demands a Unified Security Data Platform

For two decades, the security operations center has been fighting a losing battle with its own data. Every new cloud platform, identity system, SaaS application, and endpoint agent produces more telemetry, and every byte of that telemetry carries an ingestion price tag in a traditional SIEM. The result is a quiet, uncomfortable compromise happening inside most Security Operations Centers (SOCs): teams filter, sample, truncate, and delete the very data they would need to detect an attack. Industry estimates suggest security teams discard up to 75% of their data simply because retaining it is unaffordable.¹

That compromise was survivable when attackers were human. It is not survivable anymore. AI has industrialized the offensive side of security: autonomous agents now scan environments continuously, discover vulnerabilities, and chain exploits at machine speed. The window between vulnerability disclosure and active exploitation has collapsed from weeks to days, in some analyses, to hours. Meanwhile, the attacks that matter most in 2026 (social engineering, insider threats, prompt injection against enterprise AI systems) hide in data that legacy SIEMs were never designed to process: chat logs, collaboration platforms, business applications, even audio and video.

This is the asymmetry that defines security in the AI era: attackers analyze everything and strike anywhere, while defenders see only a fraction of their own environment and respond at human speed. No amount of analyst heroics closes that gap. The gap is architectural, and closing it requires treating security as what it has quietly become: a data and AI problem at enterprise scale.

Security Has Become a Data Problem. So Solve It Like One.

The data community has seen this before. Data warehouses once forced the same trade-offs of expensive ingestion, proprietary formats, siloed data locked to narrow use cases, until the lakehouse architecture disrupted the model with open formats, low-cost cloud storage, decoupled compute, and support for any data type. That architectural shift is now arriving in the SOC.

In March 2026, Databricks entered the security market with Lakewatch, an open, agentic SIEM built directly on the Databricks Data Intelligence Platform.² Rather than bolting AI features onto a legacy detection engine, Lakewatch starts from a different premise: unify 100% of security, IT, and business telemetry in one governed environment, the security lakehouse, and let embedded AI agents defend it at the same machine speed attackers now operate at.

Three architectural decisions make this more than a rebranded log platform:

Open formats end the ingestion tax. Telemetry lands in your own cloud storage in Delta Lake and Apache Iceberg, normalized to the Open Cybersecurity Schema Framework (OCSF). You pay commodity storage rates, compute is decoupled and scales independently, and there is no vendor lock-in on your own security data. Databricks cites total cost reductions of up to 80% versus legacy SIEMs, which converts the oldest trade-off in security operations, visibility versus budget, into a false choice. Teams can finally retain years of full-fidelity telemetry instead of months of filtered fragments.

One governed platform replaces the silo sprawl. Because Lakewatch runs on the same platform as enterprise data, security telemetry sits next to the HR, identity, application, and business context needed to actually investigate an alert, governed end-to-end by Unity Catalog. Access auditing, lineage, and fine-grained permissions are native, not integrated after the fact. For regulated industries facing frameworks like DORA and NIS2, multi-year retention with full auditability stops being aspirational and becomes the default posture.
Agents defend against agents. Lakewatch embeds AI where the data lives. Genie lets analysts of any skill level hunt threats across petabytes in natural language instead of proprietary query languages, and automates parsing new log sources, authoring detections from current threat intelligence, and tuning rules to cut false positives. Security teams can build custom defensive agents with Agent Bricks to triage, investigate, and summarize incidents end-to-end, reducing mean time to detect and respond while every action stays inside the governed environment. Detections themselves become detection-as-code: versioned in Git, reviewed like software, deployed through CI/CD

It is telling that Databricks paired the launch with a deepened Anthropic partnership (Claude models power Lakewatch’s agentic reasoning, and Anthropic runs its own security lakehouse on Databricks) and with acquisitions in agent authentication (Antimatter) and large-scale detection engineering (SiftD.ai).³ This is not a feature release; it is a statement that the next generation of security operations will be built on data platforms, not on appliances.

Why This Is a Platform Decision, Not a Tool Decision

The strategic value of a unified security data platform extends beyond the SOC’s budget line. When security telemetry lives on the Databricks Data Intelligence Platform, the same skills, governance model, and infrastructure that run your analytics and AI workloads now run your defense. Data engineers who build Lakeflow pipelines can onboard log sources. The Unity Catalog permissions model your data teams already trust now audits who touched what security data, when, and why. Fraud models, insider-risk analytics, and compliance reporting draw from one governed copy of the truth instead of three disconnected ones.

For organizations already invested in Databricks, this means adding security is an extension, not a migration. For organizations still paying the fragmentation tax across a legacy SIEM, a separate data lake, and a growing stack of point tools, it reframes the modernization conversation entirely: consolidating security onto the platform where your data intelligence already lives is one of the clearest TCO and risk-reduction opportunities available in 2026.

Where Infinitive Fits: Cyber Expertise Meets Databricks Delivery

Infinitive has been building lakehouse-centric cyber data solutions since well before agentic SIEM had a name. Our Cyber Data Solutions practice helps clients use the Databricks Lakehouse for SIEM augmentation, incident response, and AI-enhanced threat detection, as well as ingesting and retaining massive telemetry volumes at lower cost, surfacing the incidents that matter, and reducing the false-positive noise that burns out SOC teams. As a Databricks consulting partner with more than 20 years of business transformation experience, we bring deep vertical expertise in financial services and other regulated industries, exactly the environments where unified governance, access auditing, and compliance reporting carry the highest stakes.

We have watched security and data platforms converge from both sides of that divide. Our perspective is simple: the organizations that win the next five years of cyber defense will be the ones that treat security data as a first-class citizen of their data strategy. That means governed in the same catalog, engineered with the same rigor, and activated by the same AI capabilities as the rest of the enterprise.

Ready to Explore the Security Lakehouse?

For teams that want a structured on-ramp, the Security module of Infinitive’s Guided Activation offering provides a fast-track path: a security posture assessment built on the Databricks Security Analysis Tool, Lakewatch deployment with Unity Catalog integration, a detection-as-code rule catalog, integration with your existing alerting platforms, and compliance reporting configuration, all delivered in weeks, with fixed scope and measurable outcomes. It is a pragmatic first step toward unified, AI-powered security operations, mapped to your data strategy rather than bolted alongside it. If the asymmetry described in this post feels familiar, we would welcome the conversation.

Learn more on the Infinitive Guided Activation page or explore our work as a Databricks Consulting Partner.

Why the AI Era Demands a Unified Security Data Platform

Security Has Become a Data Problem. So Solve It Like One.

Three architectural decisions make this more than a rebranded log platform:

Open formats end the ingestion tax. Telemetry lands in your own cloud storage in Delta Lake and Apache Iceberg, normalized to the Open Cybersecurity Schema Framework (OCSF). You pay commodity storage rates, compute is decoupled and scales independently, and there is no vendor lock-in on your own security data. Databricks cites total cost reductions of up to 80% versus legacy SIEMs, which converts the oldest trade-off in security operations, visibility versus budget, into a false choice. Teams can finally retain years of full-fidelity telemetry instead of months of filtered fragments.

One governed platform replaces the silo sprawl. Because Lakewatch runs on the same platform as enterprise data, security telemetry sits next to the HR, identity, application, and business context needed to actually investigate an alert, governed end-to-end by Unity Catalog. Access auditing, lineage, and fine-grained permissions are native, not integrated after the fact. For regulated industries facing frameworks like DORA and NIS2, multi-year retention with full auditability stops being aspirational and becomes the default posture.

Agents defend against agents. Lakewatch embeds AI where the data lives. Genie lets analysts of any skill level hunt threats across petabytes in natural language instead of proprietary query languages, and automates parsing new log sources, authoring detections from current threat intelligence, and tuning rules to cut false positives. Security teams can build custom defensive agents with Agent Bricks to triage, investigate, and summarize incidents end-to-end, reducing mean time to detect and respond while every action stays inside the governed environment. Detections themselves become detection-as-code: versioned in Git, reviewed like software, deployed through CI/CD.