Skip to main content

dbt Core

Prerequisites

Connecting dbt Core

  • To set up dbt Core begin by navigating to Admin -> Settings -> Orchestration. Here you will click on Add New Integration to enter your dbt Core details.

Configure dbt Core on Datafold

After selecting dbt Core from the available options, complete configuration with the following information:

Complete the configuration by specifying the following fields:

Field NameDescription
RepositorySelect the repository that generates the webhooks and where pull / merge requests will be raised.
Data SourceSelect the data source where the code that is changed in the repository will run.
NameAn identifier used in Datafold to identify this CI configuration.
Primary key tagSee dbt Integration.
Sync metadata on every push to productionWhen selected, will sync the metadata from the dbt run with Datafold every time a push happens on the default branch.
Files to ignoreIf defined, the files matching the pattern will be ignored in the PRs. The pattern uses the syntax of .gitignore. Excluded files can be re-included by using the negation; re-included files can be later re-excluded again to narrow down the filter. For example, to exclude everything except the /dbt folder, but not the dbt .md files, do:*!dbt/*dbt/*.md.
CI Status ReportingIf the checkbox is disabled, the errors in the CI runs will be reported back to GitHub/GitLab as successes, to keep the check "green" and not block the PR/MR. By default (enabled), the errors are reported as failures and may prevent PR/MRs from being merged.
Slim CIIf this box is checked, data diffs will be run only for models changed in a pull request. You'll be able to automatically diff downstream models within your PR.
Require the datafold label to start CIWhen this is selected, the Datafold CI process will only run when the 'datafold' label has been applied. This label needs to be created manually in GitHub or GitLab and the title or name must match 'datafold' exactly.
Sampling toleranceThe tolerance to apply in sampling for all data diffs.
Sampling confidenceThe confidence to apply when sampling.
Sampling ThresholdSampling will be disabled automatically if tables are smaller than specified threshold. If unspecified, default values will be used depending on the Data Source type.
info

Using dbt Core requires use of the Datafold SDK. Continue with setup by generating a Datafold API key.

Next Steps

  • Save
  • Note the CI config id:

Helpful Guides for dbt Core