dbt Contracts Check Your Models. What Checks the Data on the Way In?

Picture a nightly dbt build failing at 02:40. The orders model fct_orders has an enforced contract, and a column that should be an integer arrived as a string. Nothing downstream is built, which is the point. But the bad rows are already in raw.orders, loaded from an API the evening before, and the fix now means finding which rows turned the column into a string, deciding what to do with them and rerunning everything.

It follows from a common pattern: the quality checks sit after the data has landed. dbt checks the shape of the models it builds. Ingestion tools mostly check that the data fits a schema. The question "is this particular row acceptable, and if not, where does it go?" often has no owner until something fails.

This article maps where each common tool checks data, what it does with a bad row, and where a row-level contract at ingestion fills a gap. It also covers when you do not need one. It does not argue that you should replace dbt contracts; they do a different job well.

Why bad rows travel so far

Two habits push quality checks downstream.

Checks are attached to the transformation tool. dbt is where many teams first write tests, so that is where the rules live. By the time a test runs, the data has been loaded, and often transformed.

Ingestion is treated as plumbing. A loader that copies an API into a table is judged on whether it ran, not on what it loaded. A schema check may exist, but value rules ("amount is not negative", "country is one of five codes") usually do not.

The result is that a single bad value can fail a whole build, or worse, pass a schema check and reach a dashboard.

The ways teams check data today

Each of these works. They guard different points and treat a bad row differently.

Some details do not fit in a table. dbt's constraint enforcement varies by platform (several warehouses enforce only not_null), and contracts do not cover Python models, ephemeral models or sources. dlt can add value validation through Pydantic models. Platform expectations can keep dropped rows only if you add a quarantine step yourself.

A common combination is a schema guard at ingestion plus tests or contracts on the published models.

The question these leave open is the one from the opening: when one row fails a value rule on the way in, where does it go, and who can see why? Dropping it loses evidence. Failing the whole load punishes every good row with it. That is the gap LakeLogic is designed for.

How LakeLogic approaches it

LakeLogic is an open-source runtime that executes data contracts written in the Open Lakehouse Contract format. A contract declares fields, types and quality rules. When the runtime processes a batch, rows that pass every row rule go to the target, and rows that fail go to a quarantine output with the name of the rule they failed. Dataset rules, such as uniqueness, are evaluated on the whole batch; a failure is recorded and alerted, and stops the run only if the contract sets quality.fail_pipeline_on_dataset_error: true.

The same contract runs on Polars, DuckDB or Spark, so it can sit in front of whatever loads your data. Three entry points are relevant here.

From an existing dbt schema file. The dbt importer reads a schema.yml or sources.yml and produces a contract, so rules you have already written can run earlier.

lakelogic import-dbt --schema models/schema.yml --model customers --output contracts/

Given this dbt definition:

models:
  - name: customers
    columns:
      - name: customer_id
        tests: [not_null, unique]
      - name: country
        tests:
          - accepted_values:
              values: ['US', 'GB', 'DE', 'FR', 'JP']

the importer maps not_null to a required field, unique to a dataset uniqueness rule, and accepted_values to a row rule (country IN ('US', 'GB', 'DE', 'FR', 'JP')). A row with country = 'XX' is quarantined with country_accepted_values as its reason; the rest of the batch continues. You can also build a processor directly with DataProcessor.from_dbt(...).

From a dlt source. A contract can name a dlt verified source or a declarative REST API as its source. LakeLogic runs the dlt extraction, then validates the extracted rows with the contract before anything is written. Credentials are referenced as environment variables, not stored in the contract.

From a stream. LakeLogic includes connectors for Kafka (including Event Hubs over the Kafka protocol), Server-Sent Events, WebSockets, Azure Event Grid and Service Bus, AWS SQS, GCP Pub/Sub and webhooks. The stream sink collects events into micro-batches, runs each batch through the same contract, writes good rows and quarantined rows, and only then commits a checkpoint. Delivery is at least once, so a crash between the write and the commit replays the batch; use a merge with a primary key, not a bare append, on resumable sources.

The Integrations notebook runs all three in Google Colab, using generated data and, for the streaming section, a public live feed.

Where it sits next to dbt

A sensible split for many teams:

• At ingestion: a LakeLogic contract per source table. Row rules quarantine bad values with reasons; dataset rules flag a batch that is wrong as a whole, or stop it if you opt in.
• In the warehouse: dbt model contracts on the models other teams consume, so their published shape cannot change by accident, and dbt tests for rules that depend on joins across models.

The two do not need to agree line for line. Ingestion rules describe what a source must deliver; dbt contracts describe what a model promises.

When you might not need this

• One trusted source and one team. If your data comes from a well-governed internal database and the dbt build is the first consumer, dbt contracts and tests are usually enough.
• Failing the whole load is acceptable. If a bad batch should simply stop and be reloaded, dlt's freeze mode or a platform expectation set to fail does that with less machinery.
• You are already on one platform's pipelines. Databricks expectations or Fabric materialized lake view constraints keep rules next to the pipeline, which is simpler if you never need them elsewhere.
• You only need schema drift protection. dlt schema contracts handle new columns and type changes well on their own.

LakeLogic earns its place when several sources feed the same estate, bad rows need to be kept and reviewed rather than dropped, or the same rules must run in more than one engine.

Limits worth knowing

• The dbt import is partial. relationships becomes a soft "is not null" rule, not a referential check against the parent table. Custom generic tests are not translated. The importer reads the tests: key; tests written under dbt's newer data_tests: key are not picked up, so check the generated contract.
• Quarantine is a place, not a process. Someone still has to own the quarantined rows and decide whether to fix, replay or discard them.
• At-least-once streaming. Replays are possible after a crash. Idempotent writes are your responsibility in the contract's materialisation settings.
• Contracts do not replace platform permissions. Who can read the target or the quarantine is set by your platform's access controls.

Key takeaways

1. dbt model contracts guard the shape of models at build time; they do not check data on the way in.
2. Decide per source what should happen to one bad row: drop it, fail the load, or keep it with a reason.
3. Keep row rules near ingestion and published-shape promises on the warehouse models; the two complement each other.
4. Reuse rules you have already written, but read what an importer produced before trusting it.
5. For streams, the checkpoint order (write, then commit) decides whether a crash loses or replays data; design the write to tolerate replays.

Our post on quarantining bad rows and gating bad batches goes deeper into the row and dataset rules used here.

Try it

Open the Integrations notebook in Google Colab. It imports a dbt schema, runs a dlt-style source and validates a live public stream against contracts, with nothing to install locally.