Stitching Records with a Golden ID

In the last chapter, we loaded raw data from multiple business systems into the Bronze layer. The problem we need to tackle next is a big one — the same person or company shows up multiple times with different IDs. Think about it. Your sales team uses a CRM. Your finance team uses a billing platform. Your support team uses a ticketing tool. Jane Smith exists in all three. But in the CRM she's "Jane Smith" with ID number 4501. In billing she's "J. Smith" with account number 88210. And in the support tool she's "jane.smith@company.com" with ticket profile 337. Three systems. Three records. Three different identifiers. As far as your data's concerned, these might as well be three completely different people.

Why does this matter? Because when an AI agent tries to answer a simple question like "what's Jane's current status," it doesn't know which Jane to look at. It might pull her billing record but miss that she has an open support ticket. Or it might see three separate customers and give you triple the revenue count. Fragmented records don't just create confusion — they create wrong answers. And wrong answers from an AI agent erode trust fast. This is the core problem that a Golden ID solves.

A Golden ID is a single, universal unique identifier that you create and assign to represent one real person or one real company across every system. It doesn't replace the IDs that already exist in your CRM or billing tool. It sits above them. Think of it like a passport number. You might have a driver's license, a library card, and a gym membership, each with its own number. But your passport is the one ID that proves you're you no matter where you go. A Golden ID works the same way — one number that says "this is Jane Smith" regardless of which system you're looking at.

But how do you figure out that those three scattered records actually belong to the same Jane? That's where record matching comes in, sometimes called entity resolution. It's the process of comparing records across systems and determining which ones refer to the same real-world person or thing. You set up matching rules — maybe you compare email addresses, maybe you look at combinations of first name, last name, and phone number. Or maybe you use fuzzy matching, which means finding records that are close but not identical, like "Jane Smith" and "J. Smith." When the rules find a match, those records get linked to the same Golden ID. This is where Databricks earns its keep. You can write matching logic that runs across millions of records from every source you ingested into that Bronze layer, all in one place.

Once you've assigned Golden IDs, you need a way to keep track of which system ID maps to which Golden ID. That's what a crosswalk table does. Picture a simple spreadsheet with three columns. Column one is the Golden ID. Column two is the source system name, like "CRM" or "billing." Column three is the original ID from that system. Jane's row might show Golden ID 7001 mapped to CRM ID 4501, billing account 88210, and support profile 337. This crosswalk table becomes your translation dictionary. Any time a system needs to look up a customer, it checks the crosswalk, finds the Golden ID, and instantly connects to every record that person has across your entire organization. That's what people mean when they talk about a single source of truth — one trusted, unified view of each customer or company.

Records are linked. But the data itself? Still messy. Dates are in different formats, fields are missing, and some records are clearly wrong. Time to clean things up.