Skip to main content
Source freshness lets SQLBuild observe whether external source data has actually changed between runs. Models downstream of unchanged sources are skipped automatically.

​
Configuration

Source freshness is configured per source in sources/*.yml with a freshness: block. See Sources: Source freshness for the full configuration reference.

​
How observations work

During planning, SQLBuild observes the current data version of each source that has freshness configured (or that the adapter can observe automatically):
  1. Observe - query the source’s current data version using the configured strategy.
  2. Compare - compare the observed version against the last recorded observation from _sqlbuild_source_freshness in the target schema.
  3. Propagate - walk the DAG downstream from changed or unknown sources to identify which models are affected.
Sources without explicit freshness: config are auto-observed using the adapter strategy if the adapter supports table metadata and the source has a physical table (not an expression source, not a managed source).

​
Lag tolerance

For timestamp-based freshness, lag_tolerance controls how much the observed value can drift before being considered a real change. If the current timestamp is within the tolerance of the previous observation, the source is treated as unchanged. This is useful for sources where the freshness timestamp moves by seconds or minutes on every query but the underlying data hasn’t meaningfully changed.

​
State storage

Source freshness observations are stored in _sqlbuild_source_freshness in each target schema. Records are appended only after the affected downstream models build successfully. If a build fails, the previous observation is preserved so the next run still sees the source as changed. Observations are resolved across all target schemas in the project, so a source referenced by models in different schemas is tracked consistently. Use sqb freshness to observe source freshness on demand without triggering a build.