Ingesting Raw Data into the Bronze Layer

Last time, we looked at what Databricks is and mapped out the three-layer Medallion Architecture. Let's actually start moving data in. The Bronze layer is where everything begins. Think of it as a giant receiving dock at a warehouse. Trucks pull up from different suppliers, and workers unload every single box exactly as it arrives. Nobody opens the boxes. Nobody sorts anything. They just stamp each one with a date and a truck number, then stack it on the shelf. That's exactly what the Bronze layer does with your data. It catches everything coming in from your business systems and stores it in its original, untouched form.

Where does this data come from? Everywhere. Your CRM — Customer Relationship Management — is the tool your sales team uses to track leads and deals. Your ERP, or Enterprise Resource Planning system, handles things like inventory, purchasing, and accounting. Then there's your support desk, your marketing platform, your billing system. Each one holds a piece of the puzzle. The problem we talked about before is that these tools don't naturally talk to each other. Data gets scattered, duplicated, and stuck in silos. Databricks solves this by connecting to all of these sources and pulling their data into one place — the Bronze layer — without forcing you to pick and choose what to keep.

Here's something worth pausing on. Why store raw data? Why not clean it up immediately? Because raw data is your safety net. Imagine you're recording a conversation. You wouldn't edit the recording before you even listen to it. What if you accidentally cut out the most important part? Same principle here. If you transform data before you store it, you might throw away something you didn't realize was valuable. Maybe a field in your CRM that seems useless today becomes critical six months from now when your AI agent needs it. By keeping the original, you can always go back and reprocess it. You never lose information. That's a rule worth remembering — never throw away raw data.

Two main ways. Getting data into the Bronze layer is pretty intuitive once you hear them. The first is batch ingestion. This means collecting a chunk of data on a schedule — maybe every hour, maybe every night. Picture a mail carrier who delivers letters once a day. You get everything at once, on a predictable timetable. The second is streaming ingestion, where data flows in continuously, in real time, like a live news ticker scrolling across your screen. Every new event, every transaction, every click arrives the moment it happens. Databricks handles both approaches and even lets you mix them. Your ERP might send daily batch updates while your website streams customer activity in real time. Both land in Bronze, side by side, ready for the next step.

What makes this even easier is that Databricks provides built-in connectors — pre-built bridges between Databricks and popular business tools. You don't need to write complicated code to pull data from Salesforce or SAP or a cloud storage bucket. You configure a connector, point it at your source, and the data starts flowing. Each record that arrives gets a timestamp so you know exactly when it landed. Nothing is modified. Nothing is filtered. It's a faithful copy of what your source systems produced, sitting safely in your Bronze layer, waiting to be refined.

You've got raw data from five different systems sitting in Bronze. But the same customer might appear in three of them with different names and different IDs. How do you figure out they're all the same person?