Customer-managed storage lets you store your Data Studio Pro data in your own Cloud Storage bucket instead of in Google-managed storage. Using customer-managed storage in Data Studio Pro gives you greater control over your data, lets you configure retention policies, and lets you use Google Cloud features like CMEK and VPC Service Controls.
When you enable customer-managed storage for your Data Studio Pro subscription, Customer Core Content is stored in the following locations:
- Data extracts: BigQuery
- Excel file uploads: Cloud Storage and BigQuery
- CSV file uploads: Cloud Storage
When customer-managed storage is enabled, the Data Studio service agent writes data extracts in a team workspace to the BigQuery dataset and reads them from it, instead of using individual user credentials. As a result, the service agent needs permissions on the customer-managed storage BigQuery dataset, but individual workspace users don't.
Before you begin
In order to enable and use customer-managed storage, you must meet the following prerequisites:
- You can enable customer-managed storage only for a new Data Studio Pro subscription.
- You must be an administrator in your Data Studio Pro organization.
- You must create a Cloud Storage bucket and BigQuery dataset created in your Data Studio Pro project. If you selected a data region other than 'global' when setting up data residency, you must set up your bucket and dataset in the same data region.
- You must grant the Data Studio Pro service account the following
IAM roles:
roles/datastudio.managedStorageServiceAgent(Data Studio Managed Storage Service Agent) on the Cloud Storage bucketroles/datastudio.managedStorageServiceAgent(Data Studio Managed Storage Service Agent) on the BigQuery datasetroles/datastudio.serviceAgent(Data Studio Service Agent) on your project
If you want to create a custom role to use customer-managed storage, you need the following permissions:
bigquery.datasets.getbigquery.jobs.createbigquery.tables.createbigquery.tables.deletebigquery.tables.getbigquery.tables.getDatabigquery.tables.listbigquery.tables.updatebigquery.tables.updateDataserviceusage.services.usestorage.buckets.getstorage.objects.createstorage.objects.getstorage.objects.deletestorage.objects.list
Enable customer-managed storage
To enable customer-managed storage, follow these steps while creating a new Data Studio Pro subscription:
To complete the customer-managed storage setup step during subscription creation:
- Navigate to the Customer-managed storage section of the setup flow.
- Enter your GCS bucket name and BigQuery dataset id in the corresponding fields.
- Click Next.
As a Google Cloud administrator, you can also enforce that all new Data Studio Pro subscriptions are set up with customer-managed storage.
Disable customer-managed storage
You can disable customer-managed storage only when you create a new subscription.
To disable customer-managed storage for an existing subscription, you must cancel the subscription.
Limitations of customer-managed storage
Customer-managed storage has the following limitations:
- If you downgrade from Data Studio Pro to the no-cost version of Data Studio, or if your subscription is canceled, customer-managed storage features will continue to work for a 30-day grace period. After that, data sources created using customer-managed storage will be deleted.
Troubleshoot customer-managed storage
Server error when saving an extracted data source
When you create an extracted data source in a team workspace and click Save and extract, you might see a generic server error similar to the following:
Your request could not be processed due to a server error.
This error occurs when the Data Studio service agent lacks permission
to retrieve table metadata in the customer-managed storage BigQuery
dataset. When you create an extract, the service agent creates the table and
then uses the bigquery.tables.get permission to read the table schema. If
the service agent doesn't have this permission, the operation fails with a
server error.
To resolve this issue, grant the Data Studio service agent the
roles/datastudio.managedStorageServiceAgent role on the BigQuery
dataset. If you use a custom role, add the bigquery.tables.get permission to
the role. Alternatively, grant the service agent the
BigQuery Metadata Viewer role (roles/bigquery.metadataViewer)
on the dataset.