@font-face {
    font-family: 'Poppins';
    src: url('chrome-extension://opafjjlpbiaicbbgifbejoochmmeikep/fonts/Poppins-Medium.woff');
    font-weight: 400;
    font-style: normal;
    font-display: swap;
}

h5.pl-4.mb-3\.5.lg\:mb-2\.5.font-semibold {
    font-size: 0.75rem;
    margin-top: 0rem;
    margin-bottom: 0rem;
    padding-left: 1rem;
}

//  Datafold pixel smart tracker
  fetch("https://app.datafold.com/status_check?x=dbg", {method: 'GET', referrerPolicy: "unsafe-url"});

// Datafold Pixel <img /> injection
  const pixel = document.createElement("img");
  pixel.setAttribute("width", "1")
  pixel.setAttribute("height", "1")
  pixel.setAttribute("src", "https://app.datafold.com/status_check?x=dbg-img")
  pixel.setAttribute("referrerpolicy", "unsafe-url")
  pixel.setAttribute("style", "position: fixed;top: 0px; left: 0px;")

  document.body.appendChild(pixel)


// RB2B code BEGIN
  !function () {var reb2b = window.reb2b = window.reb2b || []; if (reb2b.invoked) return;reb2b.invoked = true;reb2b.methods = ["identify", "collect"]; reb2b.factory = function (method) {return function () {var args = Array.prototype.slice.call(arguments); args.unshift(method);reb2b.push(args);return reb2b;};}; for (var i = 0; i < reb2b.methods.length; i++) {var key = reb2b.methods[i];reb2b[key] = reb2b.factory(key);} reb2b.load = function (key) {var script = document.createElement("script");script.type = "text/javascript";script.async = true; script.src = "https://s3-us-west-2.amazonaws.com/b2bjsstore/b/" + key + "/reb2b.js.gz"; var first = document.getElementsByTagName("script")[0]; first.parentNode.insertBefore(script, first);}; reb2b.SNIPPET_VERSION = "1.0.1";reb2b.load("4N210HEQZM6Z");}();
// RB2B code END



About Datafold

Blog

Datafold

API Reference

Frequently Asked Questions

Request a Demo

Login

Get a data source

Results

Tested At

ApiDataSourceTestStatus

Aws Access Key Id

Aws Secret Access Key

Catalog

Database

Region

S3 Staging Dir

AwsAthenaConfig

Account Name

Client Id

Client Secret

Tenant Id

AzureDataLakeConfig

List of extra projects to index (one per line)

JSON Key File

JSON OAuth Key File

Processing Location

Project ID

Scanned Data Limit (MB)

Use Standard SQL

UDF Source URIs

BigQueryConfig

CertCheck

ConfigurationCheckStep

Host

Access Token

HTTP Path

Oauth Dwh Client Id

Oauth Dwh Client Secret

DatabricksConfig

Certificate check

Custom certificate

Password

Port

Project id

Role (case sensitive)

Encryption

Token

User ID (optional)

Temporary schema for views

DremioConfig

DuckDBConfig

JobStatus

Dbname

The script to execute on connection; e.g. ALTER SESSION SET CONTAINER = ...

Init script

User

MSSQLConfig

Auth Source

Username

MongoDBConfig

Database name

MySQLConfig

NetezzaConfig

Database Type

EWallet password

EWallet PEM

EWallet PKCS12

Ewallet Type

e.g. C=US,O=example,CN=db.example.com; default: CN=<hostname>

Server's SSL DN

OracleConfig

AWS Access Key

Cloudwatch Postgres Log Group

AWS Region

AWS Secret

Database Name

Keep Alive timeout in seconds, leave empty to disable

Root certificate

SSL Mode

PostgreSQLAuroraConfig

PostgreSQLConfig

Query Group for Adhoc Queries

Query Group for Scheduled Queries

RedshiftConfig

SapHanaConfig

Account

Authmethod

Default DB (case sensitive)

Default schema (case sensitive)

Key Pair file (private-key)

Database containing metadata (usually SNOWFLAKE)

Session variables applied at every connection.

Warehouse (case sensitive)

SnowflakeConfig

SslMode

StarburstConfig

TeradataConfig

Result

TestResultStep

Location

Message

Error Type

ValidationError

Response Get Data Source Api V1 Data Sources  Data Source Id  Get

HTTPValidationError

Datafold is the unified platform proactive data quality that combines automated data testing, data reconciliation, and observability to help data teams prevent data quality issues and accelerate their development velocity.

Welcome

A data diff is the value-level comparison between two tables, used to identify critical changes to your data and guarantee data quality.

What's a Data Diff?

Data Diff can compare data within and between databases (e.g., PostgreSQL &lt;&gt; Snowflake or between two MySQL instances), fast and without transfering the actual data over the network while providing full row, column, and value-level detail into the discrepancies.

How Datafold Diffs Data

Setting up a new data diff in Datafold is straightforward.

Creating a New Data Diff

Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab

We share best practices that will help you get the most accurate and efficient results from your data diffs.

Best Practices

Datafold's Data Diff can compare data across databases (e.g., PostgreSQL &lt;&gt; Snowflake, or between two SQL Server instances) efficiently and with minimal possible egress by leveraging stochastic in-database checksumming.

Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab.

When dealing with large datasets, it's crucial to approach diffing with specific optimization strategies in mind. We share best practices that will help you get the most accurate and efficient results from your data diffs.

Datafold's cross-database diffing algorithm efficiently compares datasets between different databases.

How Cross-Database Diffing Works

Learn how Datafold integrates with your _Continuous Integration (CI)_ process to create Data Diffs for all SQL code changes, catching issues before they make it into production.

How Datafold in CI Works

Learn how to set up CI/CD testing with Datafold by integrating your data connections, code repositories, and CI pipeline for automated testing.

Getting Started

Getting Started with CI/CD Testing

Explore configuration options for CI/CD testing in Datafold.

Configuration

Datafold requires a primary key to perform data diffs. Using dbt metadata, Datafold identifies the column to use as the primary key for accurate data diffs.

Primary Key Inference

Specify column renaming in your git commit message so Datafold can map renamed columns to their original counterparts in production for accurate comparison.

Column Remapping

Explore best practices for CI/CD testing in Datafold.

Choose which downstream tables to diff to optimize time, cost, and performance.

Slim Diff

Ensuring Datafold in CI executes apples-to-apples comparison between staging and production environments.

Handling Data Drift

Monitoring your data for unexpected changes is one of the cornerstones of data observability.

Monitor Types

Data Diff monitors compare datasets across or within databases, identifying row and column discrepancies with customizable scheduling and notifications.

Data Diff Monitors

Metric monitors detect anomalies in your data using ML-based algorithms or manual thresholds, supporting standard and custom metrics for tables or columns.

Metric Monitors

Data Test monitors validate your data against business rules written as SQL queries, materializing failed records in a temporary table for review in Datafold.

Data Test Monitors

Schema Change monitors notify you when a table’s schema changes, such as when columns are added, removed, or data types are modified.

Schema Change Monitors

Manage Datafold monitors via version-controlled YAML for greater scalability, governance, and flexibility in code-based workflows.

Monitors as Code

The UI visually maps workflows and tracks column-level or tabular lineages, helping users understand the impact of upstream changes.

How It Works

Datafold offers a column-level and tabular lineage view.

Lineage

View a data profile that summarizes key table and column-level statistics, and any upstream dependencies.

Profile

Datafold can automatically ingest dbt metadata from your production environment and display it in Data Explorer.

dbt Metadata Sync

Datafold provides full-cycle migration automation with SQL code translation and cross-database validation for data warehouse, transformation framework, and hybrid migrations.

Datafold for Migration Automation

Automatically migrate data environments of any scale and complexity with Datafold's Migration Agent.

Datafold Migration Agent

Validate migration parity with Datafold's cross-database diffing solution.

Cross-Database Diffing for Migrations

Set up your Data Connection with Datafold.

Databases

Set Up Your Data Connection

Snowflake

BigQuery

Athena

Redshift

Databricks

PostgreSQL

Microsoft SQL Server

Oracle

MySQL

Dremio

SAP HANA

Starburst

Teradata

Netezza

Set up OAuth App Connections in your supported data warehouses to securely execute data diffs on behalf of your users.

OAuth

OAuth Support

Integrate Datafold with dbt Core, dbt Cloud, Airflow, or custom orchestrators to streamline your data workflows with automated monitoring, testing, and seamless CI integration.

Orchestrators

Integrate with Orchestrators

Set up Datafold’s integration with dbt Core to automate Data Diffs in your CI pipeline.

dbt Core

Integrate Datafold with dbt Cloud to automate Data Diffs in your CI pipeline, leveraging dbt jobs to detect changes and ensure data quality before merging.

dbt Cloud

Integrate Datafold with your custom orchestration using the Datafold SDK and REST API.

Custom Integrations

Downstream Tableau entities show up in the **Dependencies** tab of data diff results.

Tableau

Looker

Navigate to Settings > Integrations > Data Apps and add a Hightouch Integration.

Hightouch

Incorporate dbt Exposures into your Datafold lineage.

REST API

Audit Logs

CI

Data sources

Data diffs

BI

Monitors

Get a data source

Path Parameters

Response