Bronze/Silver/Gold built on Databricks, Snowflake and Fabric

    Implement a medallion architecture that survives contact with real data

    Bronze, Silver and Gold is a good idea that goes wrong in predictable ways — layers that blur together, a Silver tier nobody agrees on, Gold tables built straight off raw. We implement it so the boundaries hold under production pressure.

    No commitment · 30 minutes · Zero pressure

    How medallion implementations go wrong

    Nobody can say what belongs in Silver

    Without a written contract for each layer, Silver becomes a dumping ground — some tables are lightly cleaned, others carry business logic. The layer stops meaning anything and consumers start reaching past it.

    Gold tables read straight from Bronze

    A deadline hits, someone shortcuts a layer, and the shortcut becomes permanent. Now raw-data quirks are baked into an executive dashboard with no cleaning step to fix them in.

    Every run reprocesses all of history

    Full refreshes that were fine at 10GB become a cost and latency problem at 10TB. The pipeline gets slower every month and the fix keeps getting deferred.

    Bronze has been silently mutated

    Once someone updates raw in place, you've lost the ability to replay history. Every downstream bug becomes unreproducible, and you can never fully trust a backfill again.

    The same business logic exists in four models

    Copy-paste beats abstraction under deadline pressure. Then the definition of 'active customer' changes and only three of the four get updated.

    A quality problem is found in Gold, by a human

    If the first detector of bad data is an executive looking at a chart, the architecture is doing nothing for you. Checks belong at each boundary, not at the end.

    What we actually implement

    Layers with enforced contracts

    Bronze is immutable and append-only — raw, with lineage and ingestion metadata, never edited in place. Silver is conformed, deduplicated and typed. Gold is business-facing and aggregated. Each boundary has a written contract and automated checks that enforce it.

    Tests at every boundary

    Uniqueness, referential integrity, accepted values, freshness and volume anomalies run on every build. Bad data fails the pipeline at the layer it entered, instead of surfacing three layers downstream where the cause is hard to find.

    Incremental by default

    Merge and change-tracking strategies so runs process new and changed records, not the entire history. Late-arriving and out-of-order data is handled explicitly, and backfills are a routine operation rather than an incident.

    Lineage you can actually follow

    Column-level lineage from a Gold metric back to the raw record that produced it. When someone asks why the number moved, the answer takes minutes and comes with evidence.

    How a typical engagement runs

    1. Week 1

      Define the layer contracts

      Before any code, we write down what each layer guarantees for your domain — what Bronze retains, what Silver conforms and deduplicates on, what Gold exposes and to whom. Most failed implementations skipped exactly this step.

      • Written contract per layer, agreed with data consumers
      • Naming, partitioning and retention standards
      • Grain and key definitions for the core entities
    2. Week 2–6

      Build one domain end to end

      A complete Bronze → Silver → Gold path for a single real domain, with incremental logic, tests and alerting. It becomes the reference every subsequent domain is copied from.

      • Immutable Bronze ingestion with full lineage metadata
      • Conformed, deduplicated, tested Silver models
      • Gold tables serving a real consumer
      • CI running the full test suite on every pull request
    3. Week 6–12

      Scale the pattern

      Remaining domains follow the reference implementation. This is deliberately the boring phase — if the first slice was designed properly, each new domain is mostly mechanical.

      • Additional domains built to the same contracts
      • Shared macros and models for cross-cutting logic
      • Orchestration, retries and failure alerting
      • Cost and runtime baselines per layer
    4. Ongoing

      Hand over and get out of the way

      Your engineers pair with ours throughout so the patterns transfer to the people who'll live with them. We leave documentation, standards and a team that doesn't need us.

      • Team trained on the toolchain and layer contracts
      • Runbooks for failures, backfills and schema changes
      • Review standards so the layers don't erode

    What each layer is actually for

    Bronze is the landing zone and it has exactly one job: preserve what the source system sent, exactly as it sent it, with enough metadata to know when it arrived and where it came from. It is append-only. The moment you update Bronze in place you lose the ability to replay history, and replay is the whole reason the layer exists.

    Silver is where data becomes trustworthy. Types are enforced, duplicates resolved, keys conformed across systems, and the messy realities — the same customer arriving from three systems with three spellings — are reconciled. Silver should be queryable by an analyst who knows the domain but not the source quirks.

    Gold is shaped for consumption: aggregated, denormalized, named in business language, and built for specific consumers. A Gold table should be readable by someone who has never heard of your source systems. If your Gold tables still leak source-system column names, the Silver layer isn't finished.

    Medallion isn't only for Databricks

    The pattern originated in the Databricks ecosystem and is often described in Delta Lake terms, but nothing about it is vendor-specific. We have implemented the same layer contracts on Snowflake with dbt, on Microsoft Fabric, and on open Iceberg tables queried by several engines.

    What matters is not the storage format but whether the boundaries are enforced. A medallion architecture on Delta with no tests and a mutable Bronze layer is worse than a well-governed three-schema warehouse, because it carries the vocabulary of rigor without the substance.

    That said, the transformation tool does matter for how enforcement feels day to day. dbt and SQLMesh both give you version-controlled models, tests and lineage; they differ in how they handle incrementality, environments and change detection — a distinction worth understanding before you commit.

    Do you need three layers?

    Sometimes not. For a small estate with a handful of clean sources and one consuming team, three layers can be ceremony that slows you down without buying much. Two layers, or one well-tested transformation step, may be the honest answer.

    The layers earn their cost when you have multiple sources that disagree, multiple teams consuming the same entities, regulatory pressure to show where a number came from, or a history of quality incidents nobody could trace. If two or more of those apply, the structure pays for itself quickly.

    We'll say so if your situation doesn't warrant it. Recommending architecture you don't need is how consultancies get a bad name.

    What you'll get from the call

    • A clear read on whether medallion is right for your estate — or overkill
    • Written layer contracts your team and consumers actually agree on
    • An implementation plan starting with one real domain, not a sandbox
    • The incremental and quality strategy decided before code gets written

    30-minute session

    Focused, no-obligation conversation about your situation.

    Senior engineers

    Talk directly with architects, not a sales rep.

    Schedule a Consultation

    Frequently asked questions

    Related reading and demos