Google BigQuery

Connects to Google Cloud BigQuery to extract metadata from datasets, handle nested schemas (STRUCT/ARRAY), and document partitioned tables.

Connection Parameters

ParameterDescriptionRequiredExample
Project IDGCP Project IDYesmy-project-12345
Service Account JSONAuthentication fileYesUpload .json key file
DatasetBigQuery datasetOptionalproduction_data

Required Permissions

The Service Account requires the following IAM roles or specific permissions:

iam_roles.txt
# Recommended Roles
roles/bigquery.dataViewer (on datasets)
roles/bigquery.jobUser (on project)

# Or Granular Permissions
bigquery.datasets.get
bigquery.tables.list
bigquery.tables.get
bigquery.tables.getData

How to Create Service Account

  1. Go to GCP Console → IAM & Admin → Service Accounts
  2. Create new service account
  3. Grant `BigQuery Data Viewer` role
  4. Create JSON key and download it

Special Features

Multi-Project Support

Connect to and document datasets across multiple Google Cloud projects.

Partitioned Table Support

Identifies date/time partitioned tables and ingests partition metadata.

Clustered Table Support

Extracts clustering information to help optimize query performance.

Nested Schema Support

Native handling of complex STRUCT and ARRAY data types.

Common Issues

Permission denied on dataset

Ensure the service account has the `bigquery.dataViewer` role specifically on the target dataset.

Invalid credentials

Re-download the Service Account JSON key. Keys can expire or be revoked.

Project not found

Verify the Project ID is correct and the project is active in GCP.

Quota exceeded

Check if you have hit BigQuery API quota limits. Contact GCP support for increases if necessary.