Skip to main content

Privacera Platform master publication

Privacera Release 6.5

:

This document contains information about the new features and enhancements in Privacera products and services, updates to supported third-party systems, and important announcements for this release. Release notes are available with every new version of the Privacera software package.

Each release comes with product documentation that explains any new features or enhancements.

Access Management

What’s new

Certify Dremio versions 20.0 and 21.0 for FGAC support

Dremio versions 20.0 and 21.0 have been certified to include FGAC support.

Attribute-based access control for PolicySync connectors

Attribute-based access control (user/group) is supported for Azure SQL Database, Databricks SQL, AWS Redshift, AWS Snowflake, AWS RDS PostgresSQL, Amazon Aurora PostgresSQL, CloudSQL PostgresSQL and Google BigQuery.

Support access control for Varada Trino connector

The Varada Trino connector now supports access control at the schema/table/row/column level.

Support of JWT token authorization in EMR FGAC

JWT token authorization is supported in EMR FGAC.

Preview: Sync PingFederate and OneLogin identities to Privacera via UserSync

PingFederate and OneLogin identities can now be synced to Privacera via UserSync.

Support of LDAP/AD to the Privacera UserSync for case-insensitive group mapping option.

Privacera UserSync supports LDAP/AD and offers a case-insensitive group mapping option.

Support of OLAC for Databricks with DBX PrivateLink enabled environments

Databricks PrivateLink enabled environments support OLAC for Databricks (AWS).

Cost optimization in Snowflake PolicySync

By forgoing the cost of cloud services, Snowflake PolicySync lowers expenses. We have modified our metadata-loading query strategy in order to cut costs.

Support of DataAdmin Permission and Secure View Concept for the Microsoft SQL/Synapse

By default, Microsoft SQL/Synapse now supports DataAdmin Permission and Secure View Concept. If you have installed the MSSQL connector prior to the 6.4 release, contact the Privacera support team for the necessary configuration.

Introducing new macros for ABAC

New macros have been added to improve support for attribute-based access control. For more information, see Use Macros with Attribute-Based Access Control.

Enhancements

Support of column masking or row-level filtering policy for the PUBLIC group in RedShift PolicySync connector

The RedShift PolicySync connector supports column masking or row-level filtering policies for the PUBLIC group.

CREATE DATABASE privilege for a specific user and the PUBLIC group in the RedShift PolicySync connector

Granting the PUBLIC group and a specific user the CREATE DATABASE privilege in a policy now works perfectly with the RedShift PolicySync connector.

New masking function in Snowflake PolicySync for numeric dataType columns

Previously, PolicySync used 0 as the default masking value for numeric dataType columns; now, PolicySync will use the HASH() function from Snowflake to mask numeric dataType columns. As a result, existing policies that were previously created with 0 as the masking value will need to be disabled and enabled to reflect the new masking function, i.e., HASH().

Privacera Discovery

What's new

Support of Discovery offline and real-time scans by Databricks v10.4 LTS

Databricks version 10.4 LTS supports Discovery offline and real-time scans.

Privacera Platform 6.5.1.1: necessary Privacera Discovery settings

If you are running Privacera Platform version 6.5.1.1, use these settings for Privacera Discovery.

In Solr, for starting a separate consumer pod to write classification and scan summary information, enable the following property in any Discovery .yml file in the directory ~/privacera/privacera-manager/config/custom-vars/:

DISCOVERY_CONSUMER_ENABLE: "true"

For enabling multithreading for different consumers in the Discovery driver pod or Discovery consumer pod, refer to Configure system properties and follow these steps:

  1. For the Discovery driver, create the property file discovery-custom.properties.

  2. For the Discovery consumer, create the property file discovery-consumer-custom.properties.

  3. Add all of the following properties in both of the above files.

    #privacera_offline_scan_topic privacera.discovery.cloud.consumer.config.offline.scan.summary.max.poll.records=1
    privacera.discovery.cloud.consumer.config.offline.scan.max.poll.records=1
    
    #this is the timeout for offline scan job for each batch file
    privacera.discovery.cloud.consumer.config.offline.scan.summary.task.timeout.ms=172800000
    privacera.discovery.cloud.consumer.config.offline.scan.task.timeout.ms=172800000
    
    #privacera_scan_resource_info_topic privacera.discovery.cloud.consumer.config.ow.solr.scan.resource.info.max.poll.records=10000
    privacera.discovery.cloud.consumer.config.ow.solr.resource.max.poll.records=10000
    privacera.discovery.cloud.consumer.config.ow.solr.scan.resource.meta.max.poll.records=10000
    
    privacera.discovery.cloud.consumer.config.ow.solr.scan.resource.info.task.timeout.ms=172800000
    privacera.discovery.cloud.consumer.config.ow.solr.resource.task.timeout.ms=172800000
    privacera.discovery.cloud.consumer.config.ow.solr.scan.resource.meta.task.timeout.ms=172800000
    
    privacera.discovery.cloud.consumer.config.ow.solr.scan.resource.info.parallel.size=50
    privacera.discovery.cloud.consumer.config.ow.solr.scan.resource.meta.parallel.size=50
    privacera.discovery.cloud.consumer.config.ow.solr.resource.parallel.size=50
    
    #privacera_classification_topic privacera.discovery.cloud.consumer.config.ow.solr.classifications.max.poll.records=10000
    privacera.discovery.cloud.consumer.config.ow.resource.workflow.max.poll.records=10000
    privacera.discovery.cloud.consumer.ow.ranger.rest.classifications.max.poll.records=10000
    
    privacera.discovery.cloud.consumer.config.ow.solr.classifications.task.timeout.ms=86400000
    privacera.discovery.cloud.consumer.config.ow.resource.workflow.task.timeout.ms=86400000
    privacera.discovery.cloud.consumer.ow.ranger.rest.classifications.task.timeout.ms=86400000
    
    privacera.discovery.cloud.consumer.config.ow.solr.classifications.parallel.size=50
    privacera.discovery.cloud.consumer.ow.ranger.rest.classifications.parallel.size=50
    

For configuring different memory parameters for the Discovery driver, consumer, and executor pods, the following variables need to be set in any Discovery .yml file in the directory ~/privacera/privacera-manager/config/custom-vars/:

DISCOVERY_DRIVER_K8S_MEM_LIMITS
DISCOVERY_DRIVER_K8S_CPU_LIMITS
DISCOVERY_DRIVER_K8S_MEM_REQUESTS
DISCOVERY_DRIVER_K8S_CPU_REQUESTS
DISCOVERY_K8S_SPARK_DRIVER_MEMORY
DISCOVERY_K8S_SPARK_DRIVER_CORES
DISCOVERY_K8S_SPARK_EXECUTOR_MEMORY
DISCOVERY_K8S_SPARK_EXECUTOR_CORES
DISCOVERY_K8S_SPARK_DRIVER_LIMIT_CORES
DISCOVERY_K8S_SPARK_EXECUTOR_REQUEST_CORES
DISCOVERY_K8S_SPARK_EXECUTOR_LIMIT_CORES
DISCOVERY_EXECUTOR_K8S_MEM_LIMITS
DISCOVERY_EXECUTOR_K8S_CPU_LIMITS
DISCOVERY_EXECUTOR_K8S_MEM_LIMITS
DISCOVERY_EXECUTOR_K8S_CPU_REQUESTS
DISCOVERY_CONSUMER_K8S_MEM_LIMITS
DISCOVERY_CONSUMER_K8S_MEM_REQUESTS
DISCOVERY_CONSUMER_K8S_CPU_LIMITS
DISCOVERY_CONSUMER_K8S_CPU_REQUESTS

To enable autoscaling for Discovery, the following properties need to be set in any Discovery .yml file in the directory ~/privacera/privacera-manager/config/custom-vars/:

DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_ENABLED: "true"
DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_SHUFFLE_TRACKING_ENABLED: "true"
DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_EXECUTOR_IDLE_TIMEOUT: "60s"
DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_CACHED_EXECUTOR_IDLE_TIMEOUT: "120s"
DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_MAX_EXECUTORS: "5"
DISCOVERY_K8S_SPARK_MEMORY_OVERHEAD_FACTOR: "0.3"
DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_SHUFFLE_TRACKING_TIMEOUT: "300s"

The following properties need to be set in any Discovery .yml file in the directory ~/privacera/privacera-manager/config/custom-vars/:

DISCOVERY_SCAN_HIVE_MAX_COLS: 200
DISCOVERY_SCAN_HIVE_MAX_ROWS: 500
DISCOVERY_SCAN_MAX_LINES: 500
DISCOVERY_MAX_SAMPLE_VALUES: 5
DISCOVERY_SAMPLE_VALUES_MAX_LENGTH: 50
DISCOVERY_MAX_TAG_SNIPPET_SAMPLE_VALUES: 3
DISCOVERY_SPARK_JOB_MAX_TIME_MS: 86400000

For each S3 application defined in Privacera:

  1. Go to Data Source Registration.

  2. Edit each S3 application and change the following properties to the indicated values:

    • record.max.fields = 200

    • offline_scan_cleanup_enable = false

Privacera Encryption

What's new

Introducing masking

Support for Nullify and Redact as a masking option.

Deprecation of older version of PolicySync

Installation support for PolicySync V1 has been removed starting with the Privacera Manager and Documentation 6.5.0.1 release. Manual configuration for PolicySync V1 is still available between the 6.3 and 6.4 releases.

Upgrade Prerequisites

For version 6.5 the AWS CLI Version must be upgraded to Version 2, see 6.5 Platform Installation fails with invalid apiVersion to learn more.

Supported versions of third-party systems

For more information about the versions of third-party systems that Privacera Platform supports, see "Platform 6.x" table in Platform - Supported Versions of Third-Party Systems

Documentation changelog

For documentation updates in this release, see PrivaceraCloud documentation changelog.PrivaceraCloud documentation changelog