ingestr

ingestr is an open-source CLI tool by Bruin that copies data from any source to any destination using a single command. SQLBuild integrates with ingestr as a declarative source loader - you configure the ingestion directly in your source YAML, and SQLBuild handles execution as part of the build lifecycle.

Install

pip install 'sqlbuild[ingestr]'
# or
uv pip install 'sqlbuild[ingestr]'

This installs ingestr alongside SQLBuild. The ingestr CLI must be available on PATH.

How it works

Declare an ingestr block on a source in sources/*.yml
SQLBuild generates a synthetic loader that calls ingestr ingest as a subprocess
ingestr reads from the configured source and writes directly to the SQLBuild adapter’s database
The destination URI is built automatically from your SQLBuild connection config - no manual destination setup

No Python loader code is needed. The YAML declaration is the entire configuration.

sqb build loading three external (ingestr) sources (raw_customers, raw_orders, raw_payments); the raw_customers step shows the ingestr ingest command, pipeline source/destination, and an ingestion metrics table before the models build

Example: PostgreSQL to DuckDB

Replicate a table from PostgreSQL into your local DuckDB project:

# sources/raw.yml
sources:
  - name: raw_orders
    table: orders
    ingestr:
      source_uri: "postgresql://user:pass@host:5432/mydb"
      source_table: "public.orders"

That’s it. Run sqb load or sqb build and ingestr copies the orders table into your project.

Example: Stripe with incremental merge

Load Stripe charges with incremental merge on a primary key:

sources:
  - name: raw_stripe_charges
    table: charges
    ingestr:
      source_uri: "stripe://${stripe_api_key}"
      source_table: "charges"
      strategy: merge
      primary_key: id
      incremental_key: created

On subsequent runs, ingestr merges new and updated records based on the id column, using created to detect changes.

Configuration reference

The ingestr block on a source supports the following fields:

Field	Required	Description
`source_uri`	Yes	ingestr source connection URI (e.g. `postgresql://...`, `stripe://...`, `shopify://...`)
`source_table`	Yes	Source table or resource name to ingest
`strategy`	No	Ingestion strategy: `replace`, `append`, `merge`, `delete+insert`, or `truncate+insert`
`incremental_key`	No	Column used for incremental change detection
`primary_key`	No	Primary key column(s) for merge strategy (string or list)
`columns`	No	Comma-separated column list to select from the source
`extra_args`	No	Additional CLI arguments passed to `ingestr ingest` (list of strings)

Strategy mapping

Strategy	Behavior
`replace`	Drop and recreate the destination table (default when no strategy is set)
`append`	Insert all rows without deduplication
`merge`	Upsert based on `primary_key`, using `incremental_key` for change detection
`delete+insert`	Delete matching rows by `incremental_key` range, then insert replacements
`truncate+insert`	Truncate destination table, then insert all rows

Template variables

All string fields in the ingestr block support SQLBuild project variable substitution and context templates:

sources:
  - name: raw_orders
    ingestr:
      source_uri: "postgresql://${pg_user}:${pg_password}@${pg_host}:5432/${pg_database}"
      source_table: "public.orders"

Variables are resolved from the project’s merged variable config (project + target + local). Set sensitive values in sqlbuild_local.toml (gitignored):

[vars]
pg_user = "readonly"
pg_password = "secret"
pg_host = "prod-db.example.com"
pg_database = "analytics"
stripe_api_key = "sk_live_..."

Destination URI

SQLBuild automatically builds the ingestr destination URI from your adapter connection config. All supported adapters work without manual destination configuration:

Adapter	Destination URI format
DuckDB	`duckdb:///path/to/db.duckdb`
MotherDuck	`motherduck://database`
PostgreSQL	`postgresql://user:pass@host:port/db`
Snowflake	`snowflake://user:pass@account/db/schema`
BigQuery	`bigquery://project`
Databricks	`databricks://token:...@host`
SQL Server	`mssql://user:pass@host:port/db`

Reload

When --reload is passed, ingestr runs with --full-refresh, forcing a complete reload regardless of the configured strategy:

sqb load --reload
sqb build --reload

Build integration

ingestr sources are managed sources - they participate in the same lifecycle as Python loaders:

# ingestr runs automatically before dependent models
sqb build

# run ingestr sources standalone
sqb load

# skip loading
sqb build --no-load

See Loaders for details on auto-load behavior, source deferral, and the --load / --no-load / --reload flags.

Supported sources

ingestr supports 50+ sources including databases, SaaS APIs, and file systems. See the ingestr documentation for the full list of supported sources and their URI formats.

Getting Started

dbt Compatibility

Concepts

Python Nodes

Virtual Environments (Alpha)

Integrations

CLI Reference

Install

How it works

Example: PostgreSQL to DuckDB

Example: Stripe with incremental merge

Configuration reference

Strategy mapping

Template variables

Destination URI

Reload

Build integration

Supported sources

​Install

​How it works

​Example: PostgreSQL to DuckDB

​Example: Stripe with incremental merge

​Configuration reference

​Strategy mapping

​Template variables

​Destination URI

​Reload

​Build integration

​Supported sources

Install

How it works

Example: PostgreSQL to DuckDB

Example: Stripe with incremental merge

Configuration reference

Strategy mapping

Template variables

Destination URI

Reload

Build integration

Supported sources