Skip to main content

Privacera Documentation

Table of Contents

BigQuery connector properties for PolicySync on Privacera Platform

These BigQuery connector properties can be set for PolicySync in Privacera Platform.

The properties are grouped by general function, such as JDBC connection properties, properties for user, group, and role management, and other functions.

The properties are also categorized as BASIC or ADVANCED:

  • BASIC pertains to the most fundamental aspects of the connector, such as authentication.

  • ADVANCED indicates additional features beyond the BASICs, such as row-filtering or group member handling.

Start by setting the BASIC properties and then examine the ADVANCED properties to determine which of these features you might want to enable.

For a general process to migrate values from old YAML files to the new YAML files, see Migration to PolicySync v2 on Privacera Platform 7.2.

Category

Property name

Description

Default

Allowable values

JDBC configuration properties

BASIC

CONNECTOR_BIGQUERY_PROJECT_LOCATION

The BigQuery data location. We can check any of the table details and we can see its data location value. https://cloud.google.com/bigquery/docs/locations?hl=en&_ga=2.44580199.-217615852.1620270192

us

BASIC

CONNECTOR_BIGQUERY_PROJECT_ID

This property is used to set the project id which can be used for initial interaction with BigQuery APIs to explore the other projects and datasets available. Example: privacera-demo-project

CONNECTOR_BIGQUERY_JDBC_URL

This property is used to set JDBC url which can be used to connect to BigQuery server.

jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443

CONNECTOR_BIGQUERY_USE_VM_CREDENTIALS

This is property is used to set whether if you want to use service account attached to your VM for getting credentials to connect to BigQuery to get metadata and to apply grants. when this property is used to set there is no need to set BIGQUERY_OAUTH_PRIVATE_KEY_PATH.

FALSE

BASIC

CONNECTOR_BIGQUERY_OAUTH_SERVICE_ACCOUNT_EMAIL

This property is used to set service account email address which is to be used for PolicySync to connect to BigQuery.

CONNECTOR_BIGQUERY_OAUTH_PRIVATE_KEY_PATH

This property is used to set the path of service account credential json file downloaded from google service account keys section. This property value is needed when BIGQUERY_USE_VM_CREDENTIALS is set to false.

/workdir/connector/{{connector}}/cust_conf/{{ CONNECTOR_BIGQUERY_OAUTH_PRIVATE_KEY_FILE_NAME }}

BASIC

CONNECTOR_BIGQUERY_OAUTH_PRIVATE_KEY_FILE_NAME

This property is used to set the credential JSON file name which you have copied inside your connector instance configuration folder.

policysync-gbq-service-account.json

Custom IAM Roles

ADVANCED

CONNECTOR_BIGQUERY_CREATE_CUSTOM_IAM_ROLES

Enable this property if you want PolicySync to create custom IAM roles automatically in your GCP project or organization for fine grained access control. If you keep this disable, then you have to create all custom IAM roles manually in your GCP project or organization.

true

ADVANCED

CONNECTOR_BIGQUERY_CUSTOM_IAM_ROLES_SCOPE

BigQuery PolicySync used custom IAM roles for fine grained access control. These roles can be created at each individual project level IAM or at organization level IAM. This can be configurable by setting this property value to project or org. project - create/use custom IAM roles from each individual project level. org - create/use custom IAM roles from organization level.

project

project - create/use custom iam roles from each individual project level. org - create/use custom iam roles from organization level.

ADVANCED

CONNECTOR_BIGQUERY_ORGANIZATION_ID

This is property is used to set your GCP organization id if you decide to use the custom IAM roles at the organization level.

ADVANCED

CONNECTOR_BIGQUERY_CUSTOM_IAM_ROLES_NAME_MAPPING

BigQuery PolicySync used custom IAM roles for fine grained access control. These roles are created with some default Privacera defined names, but if you want to change those names as per your organization standards then you can update those using this property and PolicySync will continue to use those customized role names. It should be specified in syntax like Example value can look like below.

PrivaceraGBQProjectListRole:CustomBigQueryProjectListRole,PrivaceraGBQJobListRole:CustomBigQueryJobListRole,PrivaceraGBQJobListAllRole:CustomBigQueryJobListAllRole,PrivaceraGBQJobCreateRole:CustomBigQueryJobCreateRole,PrivaceraGBQJobGetRole:CustomBigQueryJobGetRole,PrivaceraGBQJobUpdateRole:CustomBigQueryJobUpdateRole,.... Below are the list of default custom role names which we have - PrivaceraGBQProjectListRole - PrivaceraGBQJobListRole - PrivaceraGBQJobListAllRole - PrivaceraGBQJobCreateRole - PrivaceraGBQJobGetRole - PrivaceraGBQJobUpdateRole - PrivaceraGBQJobDeleteRole - PrivaceraGBQDatasetCreateRole - PrivaceraGBQDatasetGetMetadataRole - PrivaceraGBQDatasetUpdateRole - PrivaceraGBQDatasetDeleteRole - PrivaceraGBQTableListRole - PrivaceraGBQTableCreateRole - PrivaceraGBQTableGetMetadataRole - PrivaceraGBQTableQueryRole - PrivaceraGBQTableExportRole - PrivaceraGBQTableUpdateMetadataRole - PrivaceraGBQTableUpdateRole - PrivaceraGBQTableSetCategoryRole - PrivaceraGBQTableDeleteRole - PrivaceraGBQTransferUpdateRole - PrivaceraGBQTransferGetRole

Load keys and intervals

CONNECTOR_BIGQUERY_AUDIT_SYNC_INTERVAL

This property is used to set the interval in seconds for getting access audits process. Access audits process is the process where get the access audits from the BigQuery which tells us who access what and then we push those audits to SOLR so we can display it in Privacera Access Audit UI Page. This process happens in defined interval time.

30

BASIC

CONNECTOR_BIGQUERY_MANAGE_PROJECT_LIST

This property is used to set comma separated project names for which access control should be managed by PolicySync. If you want to manage all projects then you can skip specifying this property. This supports wildcards as well. The ignore projects list has precedence over manage projects list. For example: testproject1,testproject2,sales_project.

Note

Values for this property are case-sensitive.

ADVANCED

CONNECTOR_BIGQUERY_MANAGE_DATASET_LIST

This property is used to set comma separated dataset fqdn for which access control should be managed by PolicySync. If you want to manage all datasets then you can skip specifying this property. This supports wildcards as well. The ignore dataset list has precedence over manage dataset list. For example: testproject1.dataset1,testproject2.dataset2,sales_project*.sales* .

Note

Values for this property are case-sensitive.

ADVANCED

CONNECTOR_BIGQUERY_MANAGE_TABLE_LIST

This property is used to set comma separated table/view fqdn for which access control should be managed by PolicySync. If you want to manage all tables/views then you can skip specifying this property. This supports wildcards as well. The ignore table list has precedence over manage table list. For example: testproject1.dataset1.table1,testproject2.dataset2.view2,sales_project*.sales*.*.

Note

Values for this property are case-sensitive.

ADVANCED

CONNECTOR_BIGQUERY_IGNORE_PROJECT_LIST

This property is used to set comma separated project names for which you don't want access control to be managed by PolicySync. If you don't want to ignore any project then you can skip specifying this property. This supports wildcards as well. This has precedence over manage project list. For example: testproject1,testproject2,sales_project*.

Note

Values for this property are case-sensitive.

ADVANCED

CONNECTOR_BIGQUERY_IGNORE_DATASET_LIST

This property is used to set comma separated dataset fqdn for which you don't want access control to be managed by PolicySync. If you don't want to ignore any dataset then you can skip specifying this property. This supports wildcards as well. This has precedence over manage dataset list. For example: testproject1.dataset1,testproject2.dataset2,sales_project*.sales*.

Note

Values for this property are case-sensitive.

ADVANCED

CONNECTOR_BIGQUERY_IGNORE_TABLE_LIST

This property is used to set comma separated table/view fqdn for which you don't want access control to be managed by PolicySync. If you don't want to ignore any tables/views then you can skip specifying this property. This supports wildcards as well. This has precedence over manage table list. For example: testproject1.dataset1.table1,testproject2.dataset2.view2,sales_project*.sales*.*

Note

Values for this property are case-sensitive.

Users/Groups/Roles management

ADVANCED

CONNECTOR_BIGQUERY_MANAGE_USER_LIST

This property is used to set comma separated user names for which access control should be managed by PolicySync. If you want to manage all users then you can skip specifying this property. This supports wildcards as well. The ignore users list has precedence over manage users list. For example: user1,user2,dev_user*.

ADVANCED

CONNECTOR_BIGQUERY_MANAGE_GROUP_LIST

This property is used to set comma separated group names for which access control should be managed by PolicySync. If you want to manage all group then you can skip specifying this property. This supports wildcards as well. The ignore group list has precedence over manage group list. For example: group1,group2,dev_group*.

ADVANCED

CONNECTOR_BIGQUERY_IGNORE_USER_LIST

This property is used to set comma separated user names for which you don't want access control to be managed by PolicySync. If you don't want to ignore any users then you can skip specifying this property. This supports wildcards as well. This has precedence over manage users list. For example: user1,user2,dev_user*.

ADVANCED

CONNECTOR_BIGQUERY_IGNORE_GROUP_LIST

This property is used to set comma separated group names for which you don't want access control to be managed by PolicySync. If you don't want to ignore any groups then you can skip specifying this property. This supports wildcards as well. This has precedence over manage groups list. For example: group1,group2,dev_group*.

BASIC

CONNECTOR_BIGQUERY_NATIVE_PUBLIC_GROUP_IDENTITY_NAME

Set this property to your preferred value, PolicySync uses this native public group for access grants whenever there is policy created referring to public group inside it.

ALL_AUTHENTICATED_USERS - all gcp project authenticated users. ALL_USERS - all google authenticated users.

ADVANCED

CONNECTOR_BIGQUERY_MANAGE_USER_FILTERBY_GROUP

Set this property to true, if you want to manage only the users who belongs the the groups defined in manage groups list property.

false

Access control management

ADVANCED

CONNECTOR_BIGQUERY_COLUMN_ACCESS_CONTROL_TYPE

This property is used to set the method of column level access control to be used by PolicySync.

view

view - This supports view based column level access control, which means whatever the columns users not having the access they will see those columns as null in the secure view of table or secure view of native view.

ADVANCED

CONNECTOR_BIGQUERY_POLICY_NAME_SEPARATOR

This property is used to set separator, this separator is used while creating name for native row filter policy

_

ADVANCED

CONNECTOR_BIGQUERY_ROW_FILTER_POLICY_NAME_TEMPLATE

This property is used to set template, which will be used to create native row filter policy names. For example, multiple native row filters added on table will look like tr_filter_item_1, tr_filter_item_2 etc.

tr_filter_item_

ADVANCED

CONNECTOR_BIGQUERY_ENABLE_ROW_FILTER

Set this property to true, if you want to enable native row filter functionality. This is not recommended to use, since the native row filters can only be created on tables, they can't be created on views.

false

ADVANCED

CONNECTOR_BIGQUERY_ENABLE_VIEW_BASED_MASKING

Set this property to true, if you want to enable secure view based masking in BigQuery PolicySync.

Note

BigQuery don't support native masking yet, thus recommended to use view based masking.

true

ADVANCED

CONNECTOR_BIGQUERY_ENABLE_VIEW_BASED_ROW_FILTER

Set this property to true, if you want to enable secure view based row filter in BigQuery PolicySync.

Note

BigQuery support native row filters, but due to its some limitations we recommended to use view based row filter.

true

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_CREATE_FOR_ALL

Set this property to true, if you want to create secure view for all tables as well all views which were created by end users. This will create secure view for resource regardless whether there is any masking/row filter policy exists in ranger.

true

CONNECTOR_BIGQUERY_MASKING_FUNCTIONS_DATASET

This property is used to set the name of the dataset in the gcp project which can be used to create custom masking functions required by PolicySync if any.

privacera_dataset

ADVANCED

CONNECTOR_BIGQUERY_MASKED_NUMBER_VALUE

This property is used to specify the default masking value for numeric columns

0

ADVANCED

CONNECTOR_BIGQUERY_MASKED_TEXT_VALUE

This property is used to specify the default masking value for text/string columns

<MASKED>'

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_NAME_PREFIX

By default view-based row filter and masking related secure views have the same name as the table name. If you want to change the secure view name prefix and postfix, that can be done with these properties. After prefix and postfix is specified the view name will be in this format : {prefix}{table_name}{postfix}

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_NAME_POSTFIX

By default view-based row filter and masking related secure views have the same name as the table name. If you want to change the secure view name prefix and postfix, that can be done with these properties. After prefix and postfix is specified the view name will be in this format : {prefix}{table_name}{postfix}

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_DATASET_NAME_PREFIX

By default view-based row filter and masking related secure views have the same schema name as the table schema name postfixed by _secure. If you want to change the secure view schema name prefix and postfix, that can be done with these properties. After prefix and postfix is specified the view schema name will be in this format : {prefix}{view_schema_name}{postfix}

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_DATASET_NAME_POSTFIX

By default view-based row filter and masking related secure views have the same schema name as the table schema name postfixed by _secure. If you want to change the secure view schema name prefix and postfix, that can be done with these properties. After prefix and postfix is specified the view schema name will be in this format : {prefix}{view_schema_name}{postfix}

_secure

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_NAME_REMOVE_SUFFIX_LIST

You can remove any unwanted suffix attached at the end of a table/view name. For example, if the table name is some_name_table, you can remove the suffix, _table. and then your secure name will be {prefix}some_name{postfix} Enter a suffix string or a comma-separated list of suffix strings.

ADVANCED

CONNECTOR_BIGQUERY_SECURE_VIEW_DATASET_NAME_REMOVE_SUFFIX_LIST

You can remove any unwanted suffix attached at the end of a schema name. For example, if the schema is some_name_schema, you can remove the suffix, _schema. and then your secure schema name will be {schema_prefix}some_schema{schema_postfix} Enter a suffix string or a comma-separated list of suffix strings.

CONNECTOR_BIGQUERY_ENABLE_AUTHORIZED_VIEW_ACL_UPDATER

This property is used to enable unsynchronized authorized view ACLs updater thread, it updates the dataset ACLs with authorized secure view names, It also does it periodically with by batching the requests for one or more views

true

ADVANCED

CONNECTOR_BIGQUERY_GRANT_UPDATES

This property controls whether actual grant/revoke should be run on BigQuery.

true

ADVANCED

CONNECTOR_BIGQUERY_ENABLE_DATA_ADMIN

This property is used to enable data admin feature, with data admin feature enabled you can create all the policies on table/native view and by default perspective grants will be made on secure view of table, table or native view. And the secure views will have row filter and masking capability as well. In case if you need permission on table then you can select the permission you want plus DataAdmin in the policy, In this case that selected permissions will be granted on both, the table/native view and its secure view as well.

true

ADVANCED

CONNECTOR_BIGQUERY_RANGER_SERVICE_NAME

If you have created a custom policy repository for this connector, as described in PolicySync design and configuration on Privacera Platform, set this property to the name of that custom policy repository. For example, if you have created a policy repository named postgresql_qa_instance, the value of this property should be set to postgresql_qa_instance.PolicySync design and configuration on Privacera Platform

Access audits management

BASIC

CONNECTOR_BIGQUERY_AUDIT_ENABLE

This property is used to enable access audit fetching from BigQuery.

false

ADVANCED

CONNECTOR_BIGQUERY_AUDIT_EXCLUDED_USERS

This property is used to set the list of users whose access audits should be ignored by PolicySync. It takes list of comma separated email addresses of the users.

BIGQUERY_OAUTH_SERVICE_ACCOUNT_EMAIL

ADVANCED

CONNECTOR_BIGQUERY_AUDIT_PROJECT_ID

This property is used to set the project id which should be used when running query to fetch the audits from BigQuery.

CONNECTOR_BIGQUERY_PROJECT_ID

ADVANCED

CONNECTOR_BIGQUERY_AUDIT_DATASET_NAME

This property is used to set the dataset name which should be used when running query to fetch the audits from BigQuery.

bigquery_audits