The data layer, made clear.
A short, guided read on the data world of 2026: what a warehouse, a lakehouse, and a catalog actually are, who the major players are, and why the whole industry re-organized itself around one job, feeding AI systems that can be trusted. No technical background needed.
New to data? The four words to know.
Every product in this reader, every vendor, every trend, is doing a job for one of these four words.
The raw record of what happened: a payment, an alarm, a signal reading, a support ticket. Cheap to create, and created everywhere, all the time.
Data about the data: what tables exist, where they came from, who owns them, when they were updated. It is how anything gets found and trusted.
The agreed definition. Does "availability" include planned maintenance? Which of five "revenue" numbers is the real one? Meaning is where companies quietly disagree with themselves.
The point of the whole exercise: a call someone makes and owns. Renew the contract, reroute the traffic, credit the customer. AI raises the stakes because software starts making calls too.
Getting oriented: why 2026 turned out to be about data
Two years of enterprise AI projects produced a surprisingly consistent lesson, and it was a data lesson.
In 2024 and 2025, companies pointed powerful AI models at their businesses and mostly got demos, pilots that stalled before production. MIT's State of AI in Business research found the failures were rarely about model quality. The models were fine. What they lacked was context: which table is the right one, what "churn" officially means here, which exceptions apply, who is allowed to act.
Then OpenAI published a look inside its own internal data agent and admitted the same thing: to make an AI agent useful against its own company data, it had to hand-build six layers of context around the model, schema notes and lineage, query history, expert descriptions, definitions mined from code, institutional knowledge, and a memory of corrections. The venture firm a16z summarized the industry consensus in one sentence: your data agents need context, and context has to become durable infrastructure rather than something rebuilt per project.
What this means for the rest of the reader
The chapters ahead walk the same route the industry walked: first the containers data lives in (databases, warehouses, lakes, the lakehouse), then the plumbing that moves it, then the parts that made 2026 interesting, the catalogs, meaning layers, and governance that turn raw records into something an AI can be trusted with.
The family tree: where your data actually lives
Four containers, invented in order, each fixing the previous one's weakness.
The next two pages take these four in pairs, because they were invented in pairs: the systems that run the business, and the systems that study it.
Databases and warehouses: run the business, then study it
The oldest split in data. One system answers "what is this customer's balance right now?", the other answers "how did balances trend over eight quarters?"
Runs the business minute to minute: current records, many small fast reads and writes. When a NOC ticket updates or a payment clears, a database did it. PostgreSQL ("Postgres"), Oracle, MySQL.
A separate, reorganized copy built for big questions across time. Slow to update, fast to scan billions of rows. Snowflake, Google BigQuery, and Amazon Redshift made this a cloud service you rent.
Why you keep hearing about Postgres in 2026
Postgres is the open-source database developers reach for by default. It became the strategic prize of the AI era for one reason: AI agents that act need somewhere fast and familiar to keep their working state. Databricks paid about $1B for Neon and Snowflake about $250M for Crunchy Data, both Postgres companies, so agents built on their platforms would have an operational home. When two rivals buy the same technology in the same year, it tells you where they both think the puck is going.
Lakes, the lakehouse, and the quiet standards war that ended
The lake solved cost. The lakehouse solved trust. Open table formats solved lock-in, and that one matters more than it sounds.
A giant, cheap storage pool (usually cloud object storage like Amazon S3) that accepts anything raw: logs, sensor telemetry, documents, images. Without discipline it turns into a "data swamp," which is exactly what happened at many companies.
Warehouse-grade reliability, governance, and SQL speed layered directly on lake storage, so you keep one copy of data instead of two systems. Databricks coined the term; by 2026 Snowflake, Microsoft, Google, and AWS all sell the same idea.
Open table formats: the plumbing standard worth knowing by name
Apache Iceberg and Delta Lake make a pile of files in a lake behave like a reliable database table. Because they are open standards, any engine can read the same table: Snowflake, Databricks, and a dozen others, one copy of data, no vendor toll. The two formats have largely converged technically, and the industry phrase is "write once, read anywhere."
Moving the data: pipelines, ETL, and streaming
Data is only useful where the questions are, so an entire industry exists to move and reshape it.
Extract, Transform, Load: pull data out of source systems, clean and reshape it, land it in the warehouse. Modern practice flips the last two steps (ELT): land it raw, transform it inside the warehouse. Fivetran built the extract-and-load business; dbt built the transform business.
Moving each event the moment it happens, a call drop, a payment, a telemetry reading, as a continuous feed instead of a nightly batch. Apache Kafka is the dominant technology; Confluent is the commercial leader.
Why the plumbing made headlines in 2026
Two deals said the quiet part out loud. Fivetran and dbt merged (closed June 2026), pitching themselves as "the data infrastructure for trusted AI agents." IBM paid $11B for Confluent (closed March 2026) on the thesis that real-time data is the engine of AI agents. Nobody buys plumbing companies at those prices for dashboards; they buy them because agents that act in real time need event-time data, not yesterday's batch.
Why AI needs a data layer, in 90 seconds
A language model is a brilliant new employee with no badge, no institutional memory, and no idea which of your five revenue numbers is real.
Large language models arrive knowing the public internet and nothing about your company. Everything they do for a business runs through the data they are handed at the moment of the question. Hand them clean, well-described, well-governed data and they are remarkable. Hand them a swamp and they answer fluently from the swamp.
Then agents raised the stakes. A chatbot that answers a question badly wastes a minute. An agent, AI that plans, uses tools, and takes multi-step actions, that reads a silently broken table is wrong at machine speed, in production, with your customer on the other end. The moment AI started acting rather than just answering, data quality, governance, and shared meaning stopped being hygiene and became safety equipment.
- Find: of your ten thousand tables, which one is the right one? (The catalog's job.)
- Trust: is it fresh, complete, and accurate? (Quality and observability's job.)
- Understand: what does "availability" or "churn" officially mean here? (The meaning layer's job.)
- Act safely: what is this AI allowed to see and do, and can we audit what it did? (Governance's job.)
The meaning problem
This is the single most useful idea in the reader, and it is not a technology idea.
Every enterprise system was built by a different team, in a different decade, for a different job. The billing system, the network management system, the CRM, and the finance ledger all contain "the customer," and all four describe it differently. None of them is wrong. They just never agreed.
A concrete version from the satellite world: a NOC's "availability" number excludes planned maintenance windows. The contract's SLA definition doesn't. Finance measures outage cost by month; the network measures it by event. When an AI is asked "did we meet the SLA, and what did it cost us?", it needs one agreed answer to what those words mean, and in most companies today that agreement exists only in a few experts' heads.
Why dashboards didn't fix this
Companies spent a decade connecting systems and building dashboards. Connected is real progress, but connected data is still described in each source's own language. Linking moves the data; it does not reconcile what the data means. That reconciliation, done once, governed, and reusable, is what the industry now calls the context or meaning layer, and it is where the 2026 action is.
The three worlds of data: analytics, prediction, operations
The run-versus-study split is only the start. Modern Data 101 calls the full picture the three-body problem of data, and it explains more stalled AI than any technology gap.
Dashboards and BI over the warehouse. Historical, visual, often accurate, and not wired to intervene: no chart ever raised a purchase order or rerouted a shipment. In most companies analytics is still an export function: platform to slide deck to inbox.
ML models and forecasting engines. They look ahead, and they stop at the signal: the model knows the SKU will run out, or the device will fail, and relies on someone else to hear it and act. It knows the train is off the track. It doesn't hit the brake.
ERPs, order management, ticket queues, the NOC. Where the business breathes: reacting, resolving, routing. Usually with no idea an insight was generated or a prediction was made upstream. Running on process standards, not on intelligence.
Where the three worlds actually get reconciled today: a person, a workbook, and a deadline. People cling to it because it is the one place that matches their mental model and gives them control. Million-dollar operations genuinely run on it.
The 2:00 PM test
A truck is late, an inventory scan mismatches, and twenty orders are at risk. The delivery manager checks the dashboard: it reports yesterday's SLA misses. She checks the prediction tool: it says this vendor tends to miss pickups on Tuesdays. What she needs is a call to action, now, and no system in the building is built to produce one. So she does what every operator does: goes with her gut and hopes. Each system knew something. None of them acted together.
What fixes it: the action layer
The resolution is a shared surface where the three worlds converge and something happens: signal in, meaning applied, decision made, action executed, loop closed. The article's analogy is a nervous system, and it is worth keeping.
The 2026 stack, on one page
Left to right: where data comes from, how it moves, where it lives, and who consumes it. The shaded column is the one that grew the most.
What changed since 2020, in one sentence
In 2020 this stack existed to get clean data to human analysts; in 2026 it exists to get trusted context to AI systems that act, and every difference between the two pictures follows from that one change of customer.
The life of one data point
The stack diagram shows the architecture. This page shows the movie: one record's trip through it. Say an antenna at a teleport logs a link degradation at 2:14 PM.
- Born. The monitoring system writes the record. At this moment only that system knows what the fields mean.
- Moved. A streaming feed carries it in seconds (if someone must act now) or a batch pipeline carries it tonight (if the analysis can wait).
- Landed. A raw copy settles in the lakehouse, in an open format any engine can read.
- Shaped. Transformation joins it to the site, the service, and the customer it touches.
- Checked. Observability (monitoring for data; defined two pages ahead) confirms the feed is fresh and complete, so nothing downstream consumes a silent failure.
- Registered. The catalog records that it exists, where it came from, and who owns it.
- Given meaning. The semantic layer (the shared dictionary of definitions, coming up shortly) ties it to the agreed definition of "availability" and the SLA terms it counts against.
- Consumed and decided. A dashboard shows it, a model scores it, an agent reaches it through MCP (the standard AI-to-data socket, defined soon), and a person commits the call: proactive credit, maintenance window, or watch and wait. The decision itself becomes a record.
From a DIY stack to converged systems
The biggest strategic shift in this market isn't any single product. It's who does the assembly.
The unbundled era (roughly 2018 to 2023)
The "modern data stack" was a philosophy: pick the best tool for each job, one for ingestion, one for storage, one for transformation, one for BI, one for cataloging, one for quality, and wire them together yourself. Venture funding produced a specialist for every seam, and a mid-size data team could easily run fifteen to thirty vendors. The buyer, in effect, became their own systems integrator.
What the seams actually cost
Every integration point is a place where meaning gets lost, quality breaks silently, and lineage stops. Human analysts papered over those gaps with tribal knowledge and Slack threads. AI agents can't. The moment companies tried to put agents on top of a DIY stack, the seams went from an annoyance to the failure mode.
Why it reversed
- The integration tax came due. Post-2023 budget rationalization made thirty vendor contracts and the glue-work between them hard to defend.
- Agents raised the reliability bar. Acting AI needs one governed, semantically consistent surface, which is exactly what a hand-assembled stack doesn't produce.
- Open formats commoditized the bottom. Once Iceberg made storage interchangeable, vendors had to bundle upward, into catalogs, semantics, and agents, to keep growing.
The result is the consolidation you'll see throughout this reader: Fivetran and dbt merging into one data-movement company, IBM buying Confluent, Salesforce buying Informatica, and the platforms bundling catalogs, semantic layers, Postgres, and agent tooling into one bill. Point tools are becoming features; the industry phrase for what buyers now want is closer to "a system that works" than "a stack I assemble."
Finding and trusting data: catalog, governance, lineage, quality
Four unglamorous words that decide whether anything downstream, human or AI, can be trusted.
The searchable inventory of everything the estate contains: what tables exist, what they mean, who owns them, how fresh they are. A library card catalog for data. Collibra, Alation, Atlan, Microsoft Purview, Databricks Unity Catalog. In 2026 catalogs rebranded as "context platforms," because an agent facing ten thousand tables needs the catalog to find the right one.
The rules: who can see what, what "customer" officially means, which quality standards and privacy laws apply. Governance is what makes an AI answer defensible to a regulator, an auditor, or a customer.
The recorded path a number took from source system through every transformation to the dashboard, or the AI answer. When an agent asserts a number, lineage is how you audit where it came from.
Monitoring for data, the way IT monitors applications: watch the pipelines and tables, alert when something silently breaks. Monte Carlo leads the category and extended it in 2026 to watching AI agents themselves. Surveys say 73% of enterprises won't ship an agent without this.
The meaning makers: semantic layers, knowledge graphs, data products
Three different tools for the same underlying job: getting the company to agree with itself, in machine-readable form.
Records, once, that "net revenue" equals this formula over these tables, excluding refunds. Humans and machines then ask for the concept, not the raw columns. dbt, Cube, AtScale, and now Microsoft's Fabric IQ. Without it, an AI asked about "revenue" picks one of five plausible definitions and answers confidently wrong.
Stores facts as a network: entities (customer, cell site, invoice) connected by typed relationships (owns, feeds, depends-on). It answers the questions tables can't, like "which enterprise customers are affected by this fiber cut?" Neo4j leads the database category.
The agreed vocabulary and rulebook behind a graph: what entity types exist and how they may relate. Think of it as the contract that creates shared meaning between the people and systems that produce data and the ones that consume it. Palantir made "ontology" a household word in enterprise software, and its growth validated the idea.
A unit of data managed like a product rather than a byproduct: it has an owner, a contract of what it guarantees, named consumers, versioning, and quality checks. The practitioner phrase is "context packaged as a first-class asset": everything an AI would otherwise have to reconstruct comes built in.
The AI side of the seam: RAG, vectors, MCP, agents
Four terms that describe how AI actually touches all that data.
An embedding turns text or images into a long list of numbers that captures meaning, so similar meanings land near each other mathematically. Vector search finds "nearest neighbors" fast, which is how AI searches by meaning instead of keywords. Once a standalone product category (Pinecone), by 2026 it is a built-in feature of nearly every database.
Retrieval-Augmented Generation, the pattern behind most enterprise AI assistants: before the model answers, the system retrieves the relevant company documents or data and hands them over as context. The model answers from your data, with citations, without retraining.
Model Context Protocol, the open standard Anthropic released in late 2024 for how AI applications connect to tools and data, often described as USB-C for AI. From a side project to roughly ten thousand public servers and adoption across more than a quarter of the Fortune 500 by mid-2026. One caution travels with it: MCP is only the wire. Plugging an agent into ungoverned data just delivers the confusion faster.
AI that plans, uses tools, and takes multi-step action toward a goal: detect the anomaly, query three systems, open the ticket, draft the customer notice. Both Snowflake and Databricks led their 2026 conferences with agents. Agents are why every other term in this reader suddenly has budget attached.
Pick your focus
You now have the vocabulary. What changes is the question you're trying to answer. Jump to yours, or read all three.
You don't need to be technical. You need the story.
Here is the entire 2026 data story in five sentences you can say in a meeting.
- Everyone bought AI, and most of it stalled, and the post-mortems agree the models were fine; the companies' data couldn't support them.
- The blocker is context: data that is findable, trustworthy, and means one agreed thing, and most enterprises have none of the three across systems.
- So the entire data industry repositioned around AI: storage standardized and got cheap, and the value moved up into catalogs, meaning layers, and governance.
- The giants consolidated to chase it, a dozen major acquisitions in barely more than a year, every one justified by the same phrase: data infrastructure for AI agents.
- The winners of the next phase own the trust seam, the layer where data becomes something a business, and its AI, can safely act on.
Here's the path we'll walk
- The market, on a napkin. A map you can draw for a customer, and three questions that place any vendor on it.
- The consolidation wave. Who bought whom and what it signals.
- Trends you can say out loud. Conversation-ready one-liners, each backed by a fact.
- Where CodeMettle and DataOS win. The seam between running networks and deciding with their data.
The market, on a napkin
Four neighborhoods. Every vendor you will ever meet lives in one, or is trying to move to a better one.
Three questions that place any vendor
- Where does the data live? If in their system, they are a platform and their agenda is gravity: pull everything in.
- Who defines what it means? If they hold the definitions, they are in the meaning seat, and that seat controls what AI can be trusted to do.
- Who acts on it, and does the customer keep the result? The sharpest question in 2026. Consultants act and leave nothing behind; platforms make the customer do the work. What a buyer keeps, capability, definitions, decisions, is the differentiator.
The consolidation wave: who bought whom, and what it signals
Roughly fourteen months, a dozen deals, one sentence of justification repeated by every acquirer: data infrastructure for AI agents.
| Deal | Size / status | What it signals |
|---|---|---|
| IBM buys Confluent | $11B, closed Mar 2026 | Real-time data is the feed for agents that act in the moment. |
| Salesforce buys Informatica | ~$8B, closed Nov 2025 | Even the CRM giant needed a governed data foundation for its agents. |
| Fivetran merges with dbt | Closed Jun 2026 | The plumbing consolidated into one neutral "data layer for AI" company. |
| Databricks buys Neon, Tecton, Panther | ~$1B + undisclosed, 2025-26 | Buying every part an agent needs: state, features, security. |
| Snowflake buys Crunchy Data | ~$250M, 2025 | Same move as Databricks, same reason, same year. |
| Neo4j buys GraphAware | announced Jun 2026 | Graph vendors moving up from storage toward intelligence analysis, aiming at Palantir. |
| Alation buys Numbers Station | 2025 | Catalogs bolting on AI so they can be the agent's front door. |
Eight trends you can say out loud
Each one is a sentence you can use in conversation, with the fact that backs it up.
Open table formats (Iceberg, Delta) largely converged; every platform reads the same tables. Lock-in now happens at the catalog and meaning layer.
Google renamed its catalog "Knowledge Catalog"; Atlan sells "the context layer"; Microsoft shipped Fabric IQ. Same shelf, new label: feeding AI.
MCP connects agents to anything in minutes. Gartner expects ~60% of agent-analytics projects built on MCP alone, with no agreed meaning layer underneath, to fail by 2028.
Q1 2026 revenue up 85% year over year selling ontology-driven decisioning. The debate about whether meaning layers monetize is over.
An agent acting on bad data is wrong at machine speed. That is why governance, lineage, and observability budgets grew even where AI budgets flattened.
Fivetran+dbt, IBM+Confluent, Salesforce+Informatica, all inside a year, all citing agents. Point tools are becoming features of bigger platforms.
80%+ of enterprise data is documents, tickets, transcripts, images. Pipelines that make it AI-ready are a fresh category with real budgets.
Buyers are choosing between consultants (deliver, keep nothing) and platforms (enable, you do the work). What a customer keeps and compounds is the deciding question.
When the customer says…
Six sentences you will hear in real rooms, and a calm answer to each. The pattern never changes: agree with the strength, then name the seam it doesn't cover.
Good, keep it: that is where analytical data should live, and nothing here moves it. Two things the platform still can't see: the multi-vendor network estate, whose telemetry never lands there cleanly, and the decisions your business owns across every system including the ones outside that platform. That seam is what we add, on top of what you already pay for.
Bundles are real, and for generic analytics they are fine. Your estate is not generic: M&C systems, element managers, two SLAs per site. Fabric IQ can hold business concepts once someone reconciles what the network data means, and that reconciliation is the actual work. We do it with your team, on your systems; Fabric stays a happy consumer of it.
Respect, it proved this seat pays for itself. Two honest differences: Palantir doesn't operate the NetOps edge, and its model lives inside its platform. Buyers in this market increasingly want the model open and owned by them; ours is, and the network layer comes with it.
No, and we won't rebuild it. A catalog describes what data means so people can find and govern it. We ground in your catalog and write governed definitions back to it, and we add the layer where people decide with the data and the business keeps the outcome. Describe versus decide.
Eventually, maybe, for workflows inside their stack. Every one of those agents assumes clean, semantically consistent network data underneath, and in most estates that layer doesn't exist yet. It is what we build, and the agents you buy later inherit it. Waiting doesn't build it.
Then step one is done: something sees the estate. The question no NetOps surface answers is what an event means for the contract, the customer, and the money; that is the layer above, and it works with whatever surface you run. Where the estate isn't covered, CodeMettle's operating surface completes the picture.
Where CodeMettle and DataOS win together
Everything in this reader lands on one seam, and it happens to be ours.
What each side brings
CodeMettle makes a heterogeneous, multi-vendor network operable by ordinary operators on one surface, with automation that is deterministic and auditable, the kind network teams actually let touch the network. DataOS sits above that surface and delivers named intelligence products on its operational truth: Multi-Platform Service Assurance (one service picture tied to SLAs), Revenue Assurance (which network events are costing money), Ground Station 360 and Fleet Health (ops state reconciled to contracts and assets), Spectrum & Capacity Intelligence (sell, groom, or reprice capacity). Live in weeks on the systems the customer already runs, and theirs to keep, with each product landing faster than the one before.
Why now: the 2026 buying triggers
- Post-merger integration pain: SES + Intelsat spends 2026 reconciling two NOC stacks, two contract systems, two definitions of everything; Zayo is absorbing 90,000 route miles of incompatible inventory and monitoring from Crown Castle.
- Multi-orbit and hybrid networks: one customer service now stitched from two or three networks that describe capacity and outages differently.
- Autonomy ambitions outrunning data: operators chasing TM Forum Level 4 autonomy discover the blocker is shared meaning across domains, and only about 4% have reached Level 4 overall.
- Agentic tooling that will starve: the agent platforms operators are buying assume clean, semantically consistent network data that mostly does not exist yet.
The platform giants: Snowflake, Databricks, and the hyperscalers
Five companies where most enterprise data physically sits, all converging on the same pitch: bring your AI to us.
| Player | Started as | 2026 posture |
|---|---|---|
| Snowflake | Cloud warehouse | "AI Data Cloud." Agents for business users (CoWork) and coders (CoCo); Cortex Sense assembles context automatically; bought Postgres (Crunchy Data); embraced Iceberg. |
| Databricks | Lakehouse / ML | Most aggressive acquirer (Neon, Tecton, Tabular, Panther). Agent Bricks claims 100K+ agents built on the platform. Wants to be where AI is manufactured. |
| Microsoft Fabric | Bundled analytics SaaS | Fabric IQ: a shared meaning layer for agents, defined once, queried by business concept. Wins on bundling with Office and Azure. |
| Google BigQuery | Serverless warehouse | Gemini woven through; catalog relaunched as Knowledge Catalog, a "context graph that grounds AI agents." |
| AWS | Everything store | Standardized its lakehouse on Iceberg (S3 Tables); SageMaker Unified Studio stitches its dozen tools into one workbench. |
The rivalry that shapes everything
Snowflake versus Databricks is the defining contest. They started at opposite ends (warehouse vs. lake), and by 2026 sell nearly identical portfolios: lakehouse storage, Postgres, governance, and agents. Their shared strategic bet: if agents run inside the platform's governance perimeter, the platform captures the AI budget. Their shared weakness: neither can be neutral about the other, and real enterprises run both, plus SAP, Salesforce, and forty years of everything else.
How the giants actually make money
You don't need their income statements. You need three mechanisms.
1. The meter (Snowflake, Databricks)
Storage is priced near cost. The money is in metered compute: every query, pipeline, model run, and agent action burns credits (Snowflake credits, Databricks DBUs). This is why both companies now lead with agents: an agent is a customer that runs queries all day, every day. The strategy in four words: land the data, meter the work.
2. The commit (the hyperscalers)
Big enterprises sign multi-year committed-spend agreements with AWS, Azure, or Google (pledge tens or hundreds of millions, get a discount). Almost anything bought through the cloud's marketplace draws down that commitment, which is why Snowflake and Databricks sell through those marketplaces: the customer pays with dollars already promised. The hyperscaler collects either way, from its own services or a percentage of everyone else's. And a bundled tool like Fabric can feel free because it rides an agreement the company already signed.
3. The gravity loop
Cheap storage pulls data in; the friction of moving it out keeps it there; workloads follow the data; the meter spins faster. Read the acquisitions in this light: Postgres companies, feature stores, agent tooling are all meter extensions, new kinds of work that can burn the same credits.
What this means in your deals
Ask early: whose cloud commitment would this draw from? A deal that can route through a marketplace can spend money the customer already committed, which is a very different conversation from asking for new budget.
Nearly every platform customer has a cost-overrun story. A meaning layer that kills duplicate pipelines and wrong queries is also a cost story, and finance will take that meeting.
A bundled tool is paid for with gravity. The question that cuts through: if you stop paying, or the bundle changes, what do you still own? Definitions, decisions, and models you keep are the counterweight.
Buyers feel the pull daily. A layer that works across whatever platforms they run and moves nothing is a relief, and that is exactly the posture of the CodeMettle + DataOS seam.
The hyperscalers, tool by tool
Snowflake and Databricks are guests in the hyperscalers' house. Here is what each landlord sells, by name, so the logos in a customer's slide make sense.
The everything store. S3 is the object storage under most lakes anywhere; S3 Tables builds Iceberg tables right into that storage, no assembly required. Redshift is the warehouse, Glue the pipelines and technical catalog, SageMaker Unified Studio the workbench stitching it together, and Bedrock the model platform. Strength: every part exists. Weakness: the parts feel like separate products, because they were.
The bundler. Fabric is one SaaS covering warehouse, lakehouse, pipelines, and BI on a single copy of data in OneLake; Fabric IQ is its meaning layer for agents; Purview the governance catalog; Azure AI Foundry the model platform. Strength: it rides the Office/Azure agreement the company already signed. Weakness: good-enough everywhere, best-in-class rarely.
The analytics-first one. BigQuery is the warehouse analysts love because there is nothing to size or manage; the catalog relaunched as Knowledge Catalog, pitched as a context graph that grounds agents; Vertex AI is the model platform with Gemini woven through. Strength: the cleanest analytics experience. Weakness: smallest enterprise footprint of the three.
Salesforce (with Informatica and Agentforce) and SAP hold the systems where business events are born, and they are running the same play from above: keep the data, bundle the meaning, sell the agents. Worth naming in any account map, because their gravity competes with everyone below.
How to read a hyperscaler in a deal
Three tells. Everything is good-enough rather than best, because the bundle is the product. The price advantage is real but it is gravity, paid for in future optionality. And neutrality between clouds is structurally impossible for them, which is why a layer that spans clouds, platforms, and the network estate cannot come from this tier.
Iceberg, and the two kinds of catalog
This is the most load-bearing piece of plumbing in the 2026 market, in plain terms.
What Iceberg actually is
Data in a lake is just files (usually Parquet, a compressed file format built for fast analysis). Apache Iceberg is a layer of metadata over those files: it records which files make up a table right now, what the schema is, and every change as a snapshot. That's what gives a pile of files database manners: changes that either fully happen or don't, the ability to look at the table as it was last Tuesday, and column changes that don't break anything. Think of the files as pages and Iceberg as the binding and index that make them a book.
Why everyone builds on the standard
Customers refused to keep paying storage lock-in, and the vendors converged. Databricks paid roughly $2B for Tabular, the company founded by Iceberg's creators, and aligned Delta with it. Snowflake embraced Iceberg and gave away its Polaris catalog as open source. AWS built Iceberg straight into S3. Once the table format is a commodity everyone shares, no one can win there, so each vendor competes one layer up: in the catalog, the meaning, and the agents.
The two kinds of catalog: don't mix them up
A machine's system of record for tables: it tracks table state and authorizes every read and write. Unity Catalog (Databricks), Apache Polaris / Horizon (Snowflake), AWS Glue, and Google's equivalent, BigLake. Because whoever runs it controls how every engine reaches the data, this is where lock-in moved when storage went open.
The one this reader has been teaching: the searchable inventory of what data exists, what it means, and who owns it, for humans and agents. Atlan, Collibra, Alation, Purview. It describes; it doesn't commit table changes.
The one-line separation: the technical catalog answers "which files are table X, and who may write them?" The governance catalog answers "what does table X mean, and who owns it?" A vendor saying "we have the catalog" may mean either. Ask which.
Palantir and the meaning movers
The most interesting tier of the market: companies whose product is agreement about what data means.
Palantir: the existence proof
For a decade Palantir sold something analysts struggled to categorize: an "Ontology," a decision-and-action layer over whatever data a customer has. In 2026 the market caught up with the vocabulary. Q1 revenue grew 85% year over year, the fastest since its IPO, with commercial customers (Airbus, GE Aerospace, Stellantis) joining the defense franchise. Every "context layer" announcement from the platforms is, in effect, an imitation of what Palantir proved: the meaning-and-decision seat monetizes. The open question buyers now ask is whether that seat must be a closed platform, or something they can own themselves.
The catalogs' second act
The fast riser: named a Leader in Gartner's 2026 governance quadrant, selling an "AI-native context layer" over 100+ connectors. The bet: the independent catalog becomes the cross-platform memory agents rely on.
The governance incumbent for regulated estates, banks, insurers, pharma. Less AI flash, deep policy machinery. The safe choice where the auditor is in the room.
Bought an AI startup (Numbers Station) and shipped agent products: chat with your data, an agent studio, a data products builder. Racing to stay the front door.
The graph database leader, now moving up-stack: its GraphAware acquisition targets intelligence analysis, marketed openly as an alternative to Palantir Gotham. GraphRAG is its AI wedge.
The plumbing consolidates, and legacy giants buy back in
The 2020 "modern data stack" was thirty venture-funded point tools. The 2026 version is a short list of big companies.
The signature merger, closed June 2026. Ingestion (Fivetran), transformation (dbt), plus earlier buys covering reverse-ETL and pipeline tooling: one neutral company owning data movement end to end, pitched as "the data infrastructure for trusted AI agents," 100,000+ data teams.
$11B for the commercial home of Kafka. IBM's thesis in one line: agents act in real time, so real-time data becomes the engine of enterprise AI. Rivals (Redpanda, Aiven, AWS) court customers wary of Big Blue.
~$8B, closed late 2025. The CRM giant bought thirty years of integration, master data, and governance machinery to feed its Agentforce ambitions. Even application vendors now need a governed data layer.
Databricks bought Neon (~$1B); Snowflake bought Crunchy Data (~$250M); Snowflake open-sourced a Postgres-to-Iceberg bridge. The humble open-source database became the standard home for agent state and memory.
What survives independently
Orchestration (Airflow, Dagster), observability (Monte Carlo and rivals), semantic layers (Cube, AtScale), and the vector specialists (Pinecone, now squeezed as every database bundles vector search). The pattern: a point tool survives if it is either genuinely neutral across platforms or genuinely deep in a job the platforms do shallowly.
How to read any vendor pitch: the five jobs
Vendors blur categories on purpose. Five verbs cut through it. (The napkin map told you where a vendor lives; these verbs are what it actually does there.)
Tells you what data exists and what it means, for people to find and govern. Catalogs and glossaries: Collibra, Atlan, Alation. Necessary, passive.
Connects and moves data between systems: pipelines, streaming, integration fabrics, "customer 360" projects. Linked is real progress, and still not the same as agreed meaning.
Encodes the meaning: semantic layers, knowledge graphs, ontologies. This is where "which of five revenue numbers is real" gets settled, once.
Draws conclusions: ML models, LLMs, reasoning engines. Powerful, and increasingly a commodity; every vendor has access to roughly the same models.
The fifth job, and the least crowded seat: where a person (or an authorized agent) turns all of the above into a committed call the business owns, with governance and an audit trail. Palantir sells this closed; DataOS sells it open and owned by the customer. Inference is cheap and getting cheaper; the owned, compounding decision record is not.
Why AI pilots stall
The most repeated finding in enterprise AI, from MIT to the vendors' own post-mortems: the model was fine.
The stall pattern is consistent across industries. The pilot works, because someone hand-fed it clean data from one system. Production fails, because production spans five systems that disagree about what the data means, two of which nobody fully documents, and none of which the AI is allowed to verify. The industry name for this is data debt: fragmented, ungoverned, semantically inconsistent data, accumulated over decades of reasonable local decisions.
The four specific failure modes
- Semantic ambiguity. Five definitions of "customer," "availability," or "revenue." The AI picks one, confidently.
- Quality drift. A pipeline silently breaks; the AI keeps answering from the frozen or corrupted table.
- No lineage. The AI asserts a number and nobody can trace where it came from, so nobody signs off, so it never ships.
- Context that lives in heads. The rules and exceptions that make answers right are institutional knowledge no system records.
Data products: build the context once
The consensus answer to the stall, articulated best by the Modern Data 101 community: package context as a first-class asset.
When OpenAI built its own internal data agent, it had to hand-construct six layers of context: schema notes and lineage, query history, expert descriptions, definitions mined from code, institutional knowledge, and a memory of corrections. Every company wiring an agent to raw tables pays that same tax, per agent, forever.
A data product pays it once. It is a unit of data with an owner, a machine-readable contract (what it guarantees about schema, freshness, quality), named consumers, versioning, and its meaning attached. An agent consuming a data product inherits all six of OpenAI's layers by construction. Build it once, and every future agent, dashboard, and model starts from there instead of from zero.
Why agents made this urgent
- Scale: agents multiply data consumers a hundredfold; improvised context doesn't survive that.
- Safety: agents can't improvise judgment; they need trust signals (contracts, quality checks) they can verify before acting.
- Economics: context built per-project is an expense; context built as products is an asset that compounds.
Knowledge graphs: shared meaning across the products
Data products carry the meaning of one domain each. Real questions span domains.
"Did the Q3 campaign drive revenue?" touches marketing's products and finance's. "Which enterprise customers are affected by this fiber cut?" touches the network domain, the service domain, and the contract domain. Each product knows itself; none of them knows the others. The knowledge graph is the layer that connects them: organizational definitions held once, cross-domain relationships made explicit and typed (this campaign influences that pipeline; this antenna serves that SLA), so a question can travel.
What makes a graph AI-ready rather than a diagram
- Grounding: formal, typed definitions (an ontology) so the model doesn't guess what "conversion" or "outage" means.
- Right-sized retrieval: the ability to pull just the relevant subgraph and hand it to a model in a form that fits its context window.
- Decision-awareness: the graph records decisions and state changes, so answers carry history and an audit trail.
The boundary matters too: the graph owns meaning, the products own data, and the model owns language and inference. As the Modern Data 101 line puts it, the graph supplies the one thing the model cannot infer: the meaning of your world.
Context graphs: what comes after the knowledge graph
The frontier idea of 2026, worth knowing before your customers ask about it.
A knowledge graph models entities: what exists and how it relates. A context graph, the idea pushed by Foundation Capital and developed in Modern Data 101, records decisions: each moment of judgment, with the context that was evaluated, the constraints and precedents that applied, who had authority, and what was committed. The argument: warehouses, catalogs, and governance tools all see outcomes after the fact, so they can say what happened but never why it was allowed to happen. If you want AI autonomy you can inspect and defend, the judgment itself has to be captured at the moment of decision, before state changes, or it is lost.
Why this matters practically, not just philosophically
- Audit: "why did the system approve this?" gets a recorded answer instead of a shrug.
- Precedent: agents (and new employees) can consult how similar calls were made before; judgment stops walking out the door when experts leave.
- Compounding: decision records become training context for the next decision, an asset that grows with use.
The glossary: tools, terms, jargon, and companies
Every term and vendor from this reader (and the ones you'll meet in the wild), each tagged with its seat in the value chain from the stack diagram. Filter by stage, or search. Companies are CO, terms are TERM, and marketing jargon worth decoding is JARGON.
The library
Every source and further read from this reader, in one place, filtered to your track. Switch tracks in the top bar to see the rest.
One seam, three focus areas
The BD reader learned the story and the map. The market reader learned the players. The AI reader learned why pilots stall and what fixes them. All three converge on the same seam.
Data became abundant and cheap. Models became powerful and interchangeable. The scarce thing in between, the thing every acquisition, rebrand, and product launch in this reader is chasing, is trusted, agreed, governed meaning: data an AI can find, believe, interpret, and act on, with a human accountable for the result. Whoever holds that seam for a customer holds the most durable position in the stack.
Modern Data 101, the publication from the team behind DataOS, writes the working-practitioner version of everything in this reader. Four reads that continue the path, in order: