Skip to main content

Looker

Create a Code Repositories Integration​

Create a Code Repositories integration that connects Datafold to your Looker repository.

Create a Looker Integration​

Navigate to Settings > Integrations > Data Apps and add a Looker Integration.

Complete the configuration by specifying the following fields:

Field NameDescription
Integration nameAn identifier used in Datafold to identify this Data App configuration.
Project RepositorySelect the same repository as used in your Looker project.
API Host URLThe Looker API Host URL. It has the following format: https://<instance_name>.cloud.looker.com:<port>. The port defaults are 19999 (legacy) and 443 (new), see the Looker Docs for hints. Examples: Legacy (https://datafold.cloud.looker.com:19999), New (https://datafold.cloud.looker.com:443)
Client IDFollow these steps to generate Client ID and Client Secret. These are always user specific. We recommend using a group email for continuity. See Looker User Minimum Access Policy for the required permissions.
Client SecretSee Client ID
Data source mappingWhen the correct credentials are entered we will begin to populate data sources in Looker (on the left side) that will need to be mapped to data sources configured in Datafold (on the right side). See image below.

When completed, click Submit.

It may take some time to sync all the Looker entities to Datafold and for Lineage to populate. When completed, your Looker assets will appear in Lineage as search results.

tip

Tracking Jobs explains how to find out when your data app integration is ready.

Looker User Minimum Access Policy​

The user linked to the API credentials needs the predefined Developer role or you can create a custom role with these permissions:

  • access_data
  • download_without_limit
  • explore
  • login_special_email
  • manage_spaces
  • see_drill_overlay
  • see_lookml
  • see_lookml_dashboards
  • see_looks
  • see_pdts
  • see_sql
  • see_user_dashboards
  • send_to_integration

Providing Context for Connecting Looker to Your Database and Schema​

Using the Fully Qualified Names in your Looker view files is not always possible. There are 3 different ways Datafold can construct the Fully Qualified Names.

  1. Using the (default) database configured in the Data Source configuration in the Datafold Console.
  2. Providing it through the Looker API. Unfortunately, this requires the Looker user to have the Admin role attached.
  3. Adding User Attributes to the Looker user.
    1. Setting datafold_default_database to the database name you want to connect Looker Lineage to.
    2. (BigQuery only) Setting datafold_default_host to the project id.

Order of Precedence:​

  1. The Fully Qualified names defined in your Looker view files (if any) take precedence over
  2. The User Attributes set for the Looker user, which take precedence over
  3. What we get from the Looker API, which takes precedence over
  4. The (default) database configured in the Data Source configuration.

Current Limitations of Datafold Looker Integration​

Datafold lets you connect to Looker and extend our capabilities to your Looker Views, Explores, Looks, and Dashboards. But this is a new feature, so there are some things we don’t support yet:

  • PDT/Derived Tables:Datafold only works with the tables that come from your data sources, but not with the tables that Looker makes from your SQL queries.
  • Merge Queries: Datafold supports the Queries and Looks that make up your Dashboards, but Merge Queries are not one of them. For some use cases you could achieve the same by joining the underlying views with an explore.
  • Usage metrics and popularity: Datafold shows you your Looker objects - such as dashboards, looks, and fields - but not how much you use or like them.

We are improving our Looker integration and adding more features soon. We welcome your feedback and suggestions.