> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sqlbuild.com/llms.txt
> Use this file to discover all available pages before exploring further.

# ingestr

> Declarative data ingestion from 50+ sources using ingestr.

[ingestr](https://github.com/bruin-data/ingestr) is an open-source CLI tool by [Bruin](https://github.com/bruin-data) that copies data from any source to any destination using a single command. SQLBuild integrates with ingestr as a declarative source loader - you configure the ingestion directly in your source YAML, and SQLBuild handles execution as part of the build lifecycle.

## Install

```bash theme={null}
pip install 'sqlbuild[ingestr]'
# or
uv pip install 'sqlbuild[ingestr]'
```

This installs `ingestr` alongside SQLBuild. The `ingestr` CLI must be available on `PATH`.

## How it works

1. Declare an `ingestr` block on a source in `sources/*.yml`
2. SQLBuild generates a synthetic loader that calls `ingestr ingest` as a subprocess
3. ingestr reads from the configured source and writes directly to the SQLBuild adapter's database
4. The destination URI is built automatically from your SQLBuild connection config - no manual destination setup

No Python loader code is needed. The YAML declaration is the entire configuration.

<Frame>
  <img src="https://mintcdn.com/mimo-9eaac99f/fKTqkb5cke_PLqGK/ingestr-example.png?fit=max&auto=format&n=fKTqkb5cke_PLqGK&q=85&s=16829a0dccf4a5afcbde0294b262a360" alt="sqb build loading three external (ingestr) sources (raw_customers, raw_orders, raw_payments); the raw_customers step shows the ingestr ingest command, pipeline source/destination, and an ingestion metrics table before the models build" width="1549" height="1015" data-path="ingestr-example.png" />
</Frame>

## Example: PostgreSQL to DuckDB

Replicate a table from PostgreSQL into your local DuckDB project:

```yaml theme={null}
# sources/raw.yml
sources:
  - name: raw_orders
    table: orders
    ingestr:
      source_uri: "postgresql://user:pass@host:5432/mydb"
      source_table: "public.orders"
```

That's it. Run `sqb load` or `sqb build` and ingestr copies the `orders` table into your project.

## Example: Stripe with incremental merge

Load Stripe charges with incremental merge on a primary key:

```yaml theme={null}
sources:
  - name: raw_stripe_charges
    table: charges
    ingestr:
      source_uri: "stripe://${stripe_api_key}"
      source_table: "charges"
      strategy: merge
      primary_key: id
      incremental_key: created
```

On subsequent runs, ingestr merges new and updated records based on the `id` column, using `created` to detect changes.

## Configuration reference

The `ingestr` block on a source supports the following fields:

| Field             | Required | Description                                                                              |
| ----------------- | -------- | ---------------------------------------------------------------------------------------- |
| `source_uri`      | Yes      | ingestr source connection URI (e.g. `postgresql://...`, `stripe://...`, `shopify://...`) |
| `source_table`    | Yes      | Source table or resource name to ingest                                                  |
| `strategy`        | No       | Ingestion strategy: `replace`, `append`, `merge`, `delete+insert`, or `truncate+insert`  |
| `incremental_key` | No       | Column used for incremental change detection                                             |
| `primary_key`     | No       | Primary key column(s) for merge strategy (string or list)                                |
| `columns`         | No       | Comma-separated column list to select from the source                                    |
| `extra_args`      | No       | Additional CLI arguments passed to `ingestr ingest` (list of strings)                    |

### Strategy mapping

| Strategy          | Behavior                                                                    |
| ----------------- | --------------------------------------------------------------------------- |
| `replace`         | Drop and recreate the destination table (default when no strategy is set)   |
| `append`          | Insert all rows without deduplication                                       |
| `merge`           | Upsert based on `primary_key`, using `incremental_key` for change detection |
| `delete+insert`   | Delete matching rows by `incremental_key` range, then insert replacements   |
| `truncate+insert` | Truncate destination table, then insert all rows                            |

## Template variables

All string fields in the `ingestr` block support SQLBuild project variable substitution and context templates:

```yaml theme={null}
sources:
  - name: raw_orders
    ingestr:
      source_uri: "postgresql://${pg_user}:${pg_password}@${pg_host}:5432/${pg_database}"
      source_table: "public.orders"
```

Variables are resolved from the project's merged variable config (project + target + local).

Set sensitive values in `sqlbuild_local.toml` (gitignored):

```toml theme={null}
[vars]
pg_user = "readonly"
pg_password = "secret"
pg_host = "prod-db.example.com"
pg_database = "analytics"
stripe_api_key = "sk_live_..."
```

## Destination URI

SQLBuild automatically builds the ingestr destination URI from your adapter connection config. All supported adapters work without manual destination configuration:

| Adapter    | Destination URI format                    |
| ---------- | ----------------------------------------- |
| DuckDB     | `duckdb:///path/to/db.duckdb`             |
| MotherDuck | `motherduck://database`                   |
| PostgreSQL | `postgresql://user:pass@host:port/db`     |
| Snowflake  | `snowflake://user:pass@account/db/schema` |
| BigQuery   | `bigquery://project`                      |
| Databricks | `databricks://token:...@host`             |
| SQL Server | `mssql://user:pass@host:port/db`          |

## Reload

When `--reload` is passed, ingestr runs with `--full-refresh`, forcing a complete reload regardless of the configured strategy:

```bash theme={null}
sqb load --reload
sqb build --reload
```

## Build integration

ingestr sources are managed sources - they participate in the same lifecycle as Python loaders:

```bash theme={null}
# ingestr runs automatically before dependent models
sqb build

# run ingestr sources standalone
sqb load

# skip loading
sqb build --no-load
```

See [Loaders](/concepts/python-nodes/loaders) for details on auto-load behavior, source deferral, and the `--load` / `--no-load` / `--reload` flags.

## Supported sources

ingestr supports 50+ sources including databases, SaaS APIs, and file systems. See the [ingestr documentation](https://bruin-data.github.io/ingestr/) for the full list of supported sources and their URI formats.