Use customer-managed storage

Customer-managed storage lets you store your Data Studio Pro data in your own Cloud Storage bucket instead of in Google-managed storage. Using customer-managed storage in Data Studio Pro gives you greater control over your data, lets you configure retention policies, and lets you use Google Cloud features like CMEK and VPC Service Controls.

When you enable customer-managed storage for your Data Studio Pro subscription, Customer Core Content is stored in the following locations:

  • Data extracts: BigQuery
  • Excel file uploads: Cloud Storage and BigQuery
  • CSV file uploads: Cloud Storage

When customer-managed storage is enabled, the Data Studio service agent writes data extracts in a team workspace to the BigQuery dataset and reads them from it, instead of using individual user credentials. As a result, the service agent needs permissions on the customer-managed storage BigQuery dataset, but individual workspace users don't.

Before you begin

In order to enable and use customer-managed storage, you must meet the following prerequisites:

  • You can enable customer-managed storage only for a new Data Studio Pro subscription.
  • You must be an administrator in your Data Studio Pro organization.
  • You must create a Cloud Storage bucket and BigQuery dataset created in your Data Studio Pro project. If you selected a data region other than 'global' when setting up data residency, you must set up your bucket and dataset in the same data region.
  • You must grant the Data Studio Pro service account the following IAM roles:
    • roles/datastudio.managedStorageServiceAgent (Data Studio Managed Storage Service Agent) on the Cloud Storage bucket
    • roles/datastudio.managedStorageServiceAgent (Data Studio Managed Storage Service Agent) on the BigQuery dataset
    • roles/datastudio.serviceAgent (Data Studio Service Agent) on your project

If you want to create a custom role to use customer-managed storage, you need the following permissions:

  • bigquery.datasets.get
  • bigquery.jobs.create
  • bigquery.tables.create
  • bigquery.tables.delete
  • bigquery.tables.get
  • bigquery.tables.getData
  • bigquery.tables.list
  • bigquery.tables.update
  • bigquery.tables.updateData
  • serviceusage.services.use
  • storage.buckets.get
  • storage.objects.create
  • storage.objects.get
  • storage.objects.delete
  • storage.objects.list

Enable customer-managed storage

To enable customer-managed storage, follow these steps while creating a new Data Studio Pro subscription:

To complete the customer-managed storage setup step during subscription creation:

  1. Navigate to the Customer-managed storage section of the setup flow.
  2. Enter your GCS bucket name and BigQuery dataset id in the corresponding fields.
  3. Click Next.

As a Google Cloud administrator, you can also enforce that all new Data Studio Pro subscriptions are set up with customer-managed storage.

Disable customer-managed storage

You can disable customer-managed storage only when you create a new subscription.

To disable customer-managed storage for an existing subscription, you must cancel the subscription.

Limitations of customer-managed storage

Customer-managed storage has the following limitations:

  • If you downgrade from Data Studio Pro to the no-cost version of Data Studio, or if your subscription is canceled, customer-managed storage features will continue to work for a 30-day grace period. After that, data sources created using customer-managed storage will be deleted.

Troubleshoot customer-managed storage

Server error when saving an extracted data source

When you create an extracted data source in a team workspace and click Save and extract, you might see a generic server error similar to the following:

Your request could not be processed due to a server error.

This error occurs when the Data Studio service agent lacks permission to retrieve table metadata in the customer-managed storage BigQuery dataset. When you create an extract, the service agent creates the table and then uses the bigquery.tables.get permission to read the table schema. If the service agent doesn't have this permission, the operation fails with a server error.

To resolve this issue, grant the Data Studio service agent the roles/datastudio.managedStorageServiceAgent role on the BigQuery dataset. If you use a custom role, add the bigquery.tables.get permission to the role. Alternatively, grant the service agent the BigQuery Metadata Viewer role (roles/bigquery.metadataViewer) on the dataset.