> ## Documentation Index > Fetch the complete documentation index at: https://docs.sqlbuild.com/llms.txt > Use this file to discover all available pages before exploring further. # Benchmarks > End-to-end compiler and Rules performance on deterministic, production-shaped projects. SQLBuild benchmarks run the real `sqb compile` command. They measure complete compiler workflows, including project discovery, parsing, dependency resolution, SQL expansion and analysis, contracts, tests, Rules, cache publication, and artifact generation. The benchmark projects are generated so each run is deterministic and reproducible. Their shape is not arbitrary: the reference workload was calibrated from a real working SQLBuild project, including its resource ratios, graph structure, SQL-size distribution, tests, audits, macros, functions, sources, seeds, and hooks. The generated projects contain neutral names and data. They do not contain source code or business data from the reference project. ## Production-shaped workload The reference compiler workload reproduces the measured characteristics of a working SQLBuild project: | Characteristic | Observed project | Generated benchmark | | -------------------------- | ---------------: | ------------------: | | Models | 976 | 976 | | Sources | 232 | 232 | | Seeds | 46 | 46 | | SQL functions | 23 | 23 | | Python macros | 12 | 12 | | Attached audits | 731 | 700 | | Native SQL test cases | 130 | 130 | | SQL hooks | 2 | 2 | | Execution layers | 54 | 54 | | Authored model SQL | 6.05 MB | about 6.4 MB | | Median model size | 1.9 KB | 1.9 KB | | 75th-percentile model size | 4.5 KB | 4.5 KB | | 90th-percentile model size | 10.9 KB | 11.0 KB | | 95th-percentile model size | 15.2 KB | 15.2 KB | | 99th-percentile model size | 48.2 KB | 48.0 KB | | Largest model | 519 KB | 520 KB | The generated dependency graph combines a 54-layer spine with bounded model chains. The workload also includes path defaults, reusable schemas, enforced contracts, attached audits, SQL hooks, source and seed references, SQL functions, composed Python macros, CTEs, derived tables, assertions, repeated test targets, and test fixtures ranging from 40 to 200 rows. This preserves the characteristics that exercise the compiler without publishing or depending on one specific project. ## Compiler performance On the production-shaped 976-model workload, a required CI run measured: | Compile path | End-to-end `sqb compile` time | | -------------------------------------------- | ----------------------------: | | First compile | 3.21s | | Unchanged recompile | 0.60s | | Recompile after a leaf-model edit | 0.78s | | Recompile after a central-model edit | 0.67s | | Recompile after a SQL-test edit | 0.70s | | Recompile after a Python-macro edit | 0.94s | | Recompile after a project-configuration edit | 3.56s | Model, test, and macro edits reuse unaffected compiler work. The project-configuration scenario deliberately changes an input with broad impact and therefore performs wider recomputation. ### Model and SQL-volume scaling A separate compiler profile increases model count and authored SQL volume while retaining the measured model-size distribution: | Models | Authored model SQL | First compile | Unchanged recompile | | -----: | -----------------: | ------------: | ------------------: | | 3,000 | 18–25 MB | 2.37s | 1.84s | | 10,000 | 60–80 MB | 7.50s | 5.09s | This profile isolates compiler scaling by model count and SQL volume. The production-shaped profile above is the broader test of sources, seeds, functions, macros, tests, audits, hooks, contracts, and artifact generation occurring together. ## Rules performance The Rules benchmark scales the same production-shaped generator to 5,000 models. The resulting project contains: | Characteristic | Generated workload | | ----------------------------- | -----------------: | | Models | 5,000 | | Sources | 1,189 | | Seeds | 236 | | SQL functions | 118 | | Python macros | 61 | | Native SQL tests | 666 | | Attached audits | 3,586 | | SQL hooks | 2 | | Authored model SQL | 31.3 MB | | Authored SQL-test definitions | 6.4 MB | On this project, 20 custom Rules complete their first compile in just over 10 seconds. An unchanged compile completes in under six seconds. Even with 100 custom Rules, an unchanged compile completes in just over seven seconds. | Compile | 20 Custom Rules | 100 Custom Rules | | -------------------------- | --------------: | ---------------: | | First compile (no cache) | 10.83s | 12.19s | | Unchanged compile | 5.81s | 7.24s | | After editing 1% of models | 7.43s | 10.02s | Each measurement executes the complete `sqb compile` command. The edit scenario changes 50 of the 5,000 models before recompiling. Because most Rules in this profile evaluate once per model, the 20- and 100-Rule configurations perform roughly 100,000 and 500,000 Rule evaluations respectively. Increasing the custom Rule pack from 20 to 100 multiplies the number of Rule evaluations by five, while the unchanged compile median increases from 5.81 to 7.24 seconds. ### Rule implementation changes Changing a Rule or one of its imported helpers participates in cache invalidation. In the 5,000-model, 100-Rule profile, editing one custom Rule invalidated exactly 5,001 evaluations while preserving 495,001 cached results. The complete compile finished in a 9.89-second median. CI also verifies exact invalidation for model edits, SQL-test edits, macro edits, project configuration, directly observed project inputs, custom Rule source, and imported custom helpers. A cache result with an unexpected hit or miss count fails the benchmark. ## Methodology Compiler and Rules benchmarks run on an 8-vCPU, 32 GB Linux CI runner class using Python 3.12 and Polyglot 0.9.2. The Rules results shown above are the median of five required-CI runs. Allocated CPUs across those runs were AMD EPYC or Intel Xeon processors. The Rules benchmark output records: * complete CLI elapsed time; * compiler phase timings; * Rule cache hits and misses; * Rule cache size; * peak resident memory; * workload resource counts; * SQLBuild, Python, and Polyglot versions; * every raw timing and memory sample. The required Rules CI profile enforces profile-specific latency, cache-size, peak-memory, timeout, and aggregate compiler budgets. Cache hit and miss expectations are exact. These measurements cover compilation and static analysis. They do not measure warehouse query execution. ## Reproduce the benchmarks Run the five-iteration Rules publication profile: ```bash theme={null} uv run python -m scripts.benchmark_rules \ --models 5000 \ --iterations 5 \ --output rules-5000.json ``` Run the bounded profile used by required CI: ```bash theme={null} uv run python -m scripts.benchmark_rules_ci \ --output rules-performance-ci.json \ --summary-output rules-performance-ci.md ``` Run the compiler performance guards: ```bash theme={null} uv run pytest \ tests/e2e/src/sqlbuild/cli/commands/main/compile/test_compile_performance.py \ -n auto \ --dist loadfile ``` The benchmark generators, guard thresholds, and CI workflow are maintained in the public [SQLBuild repository](https://github.com/chio-labs/sqlbuild).