Data Connections
Databricks
Steps to complete:
- Generate a Personal Access Token
- Retrieve SQL warehouse settings
- Create schema for Datafold
- Configure your data connection in Datafold
Generate a Personal Access Token
Visit Settings → User Settings, and then switch to Personal Access Tokens tab.
Then, click Generate new token. Save the generated token somewhere, you’ll need it later on.
Retrieve SQL warehouse settings
In SQL mode, navigate to SQL Warehouses.
Choose the preferred warehouse and copy the following fields values from its Connection Details tab:
- Server hostname
- HTTP path
Create schema for Datafold
Datafold utilizes a temporary dataset to materialize scratch work and keep data processing in the your warehouse.
Configure in Datafold
| Field Name | Description |
|---|---|
| Name | A name given to the data connection within Datafold |
| Host | The hostname retrieved in the Connection Details tab |
| HTTP Path | The HTTP Path retrieved in the Connection Details tab |
| Access Token | The token retrieved in Generate a Personal Access Token |
| Catalog | The catalog and schema name of your Databricks account. Formatted as catalog_name.schema_name (In most cases, catalog_name is hive_metastore.) |
| Dataset for temporary tables | Certain operations require Datafold to materialize intermediate results, which are stored in a dedicated schema. The input for this field should be in the catalog_name.schema_name format. (In most cases, catalog_name is hive_metastore.) |
Click Create. Your data connection is ready!