- PrivaceraCloud Release 7.4
- Enhancements and updates in PrivaceraCloud release 7.4
- Known Issues in PrivaceraCloud 7.4
- PrivaceraCloud User Guide
- Overview of PrivaceraCloud
- Connect applications with the setup wizard
- Connect applications
- About applications
- Connect Azure Data Lake Storage Gen 2 (ADLS) to PrivaceraCloud
- Connect Amazon Textract to PrivaceraCloud
- Athena
- Privacera Discovery with Cassandra
- Connect Databricks to PrivaceraCloud
- Databricks SQL
- Databricks SQL Overview and Configuration
- Planning and general process
- Prerequisites
- Databricks SQL with Privacera Hive
- Connect Databricks SQL application
- Grant Databricks SQL permissions to PrivaceraCloud users
- Define a resource policy
- Test the policy
- Databricks SQL PolicySync fields
- Configuring column-level access control
- View-based masking functions and row-level filtering
- Create an endpoint in Databricks SQL
- Databricks SQL Fields
- Databricks SQL Hive Service Definition
- Databricks SQL Masking Functions
- Databricks SQL Encryption
- Use a custom policy repository with Databricks
- Connect Databricks SQL to Hive policy repository on PrivaceraCloud
- Databricks SQL Overview and Configuration
- Connect Databricks Unity Catalog to PrivaceraCloud
- Connect S3 to PrivaceraCloud
- Prerequisites in AWS console
- Connect S3 application to PrivaceraCloud
- Enable Privacera Access Management for S3
- Enable Data Discovery for S3
- S3 AWS Commands - Ranger Permission Mapping
- S3
- AWS Access with IAM
- Access AWS S3 buckets from multiple AWS accounts
- Add UserInfo in S3 Requests sent via Dataserver
- Control access to S3 buckets with AWS Lambda function on PrivaceraCloud
- Dremio Plugin
- DynamoDB
- Connect Elastic MapReduce from Amazon application to PrivaceraCloud
- Connect EMR application
- EMR Spark access control types
- PrivaceraCloud configuration
- AWS IAM roles using CloudFormation setup
- Create a security configuration
- Create EMR cluster
- How to configure multiple JSON Web Tokens (JWTs) for EMR
- EMR Native Ranger Integration with PrivaceraCloud
- Connect EMRFS S3 to PrivaceraCloud
- Files
- GBQ
- Google Cloud Storage
- Connect Glue to PrivaceraCloud
- Google BigQuery for PolicySync
- Connect Kinesis to PrivaceraCloud
- Connect Lambda to PrivaceraCloud
- Microsoft SQL Server
- MySQL for Discovery
- Open Source Apache Spark
- Oracle for Discovery
- PostgreSQL
- Connect Power BI to PrivaceraCloud
- Presto
- Redshift
- Snowflake
- Starburst Enterprise with PrivaceraCloud
- Starburst Enterprise Presto
- Trino
- Connect users
- Data access Users, Groups, and Roles
- UserSync
- Portal user LDAP/AD
- Datasource
- Okta Setup for SAML-SSO
- Azure AD setup
- SCIM Server User-Provisioning
- User Management
- Identity
- Access Manager
- Access Manager
- Resource Policies
- Tag Policies
- Scheme Policies
- Service Explorer
- Reports
- Audit
- About data access users, groups, and roles resource policies
- Security zones
- Discovery
- Classifications via random sampling
- Privacera Discovery scan targets
- Propagate Privacera Discovery Tags to Ranger
- Enable offline scanning on Azure Data Lake Storage Gen 2 (ADLS)
- Enable Real-time Scanning of S3 Buckets
- Enable Real-time Scanning on Azure Data Lake Storage Gen 2 (ADLS)
- Enable Discovery Realtime Scanning Using IAM Role
- Encryption
- Overview of Privacera Encryption
- Encryption schemes
- Presentation schemes
- Masking schemes
- Create scheme policies
- Privacera-supplied encryption schemes for the Privacera API
- Privacera-supplied encryption schemes for the Bouncy Castle API
- API date input formats
- Deprecated encryption formats, algorithms, and scopes
- Privacera Encryption REST API
- PEG API endpoint
- PEG REST API encryption endpoints
- Prerequisites
- Common PEG REST API fields
- Construct the datalist for the /protect endpoint
- Deconstruct the response from the /unprotect endpoint
- Example data transformation with the /unprotect endpoint and presentation scheme
- Example PEG API endpoints
- Make encryption API calls on behalf of another user
- Privacera Encryption UDF for masking in Databricks on PrivaceraCloud
- Privacera Encryption UDFs for Trino on PrivaceraCloud
- Syntax of Privacera Encryption UDFs for Trino
- Prerequisites for installing Privacera Crypto plug-in for Trino
- Download and install Privacera Crypto jar
- Set variables in Trino etc/crypto.properties
- Restart Trino to register the Privacera encryption and masking UDFs for Trino
- Example queries to verify Privacera-supplied UDFs
- Privacera Encryption UDF for masking in Trino on PrivaceraCloud
- Encryption UDFs for Apache Spark on PrivaceraCloud
- Launch Pad
- Settings
- Dashboard
- Usage statistics
- Operational status of PrivaceraCloud and RSS feed
- How to Get Support
- Coordinated Vulnerability Disclosure (CVD) Program of Privacera
- Shared Security Model
- PrivaceraCloud Previews
- Preview: File Explorer for S3
- Preview: File Explorer for Azure
- Preview: File Explorer for GCS
- Preview: Scan Generic Records with NER Model
- Preview: Scan Electronic Health Records with NER Model
- Preview: OneLogin setup for SAML-SSO
- Preview: Azure Active Directory SCIM Server UserSync
- Preview: OneLogin UserSync
- Preview: PingFederate UserSync
- Quickstart for Databricks Unity Catalog on PrivaceraCloud
- What do I need to do in my Databricks Workspace?
- Where is the sample dataset in my Databricks Workspace?
- What should I do in the PrivaceraCloud web portal?
- Access use-case - How do I give a user access to a table or restrict from running a SQL select query?
- Access use-case - How do I restrict a user from seeing contents of a column in the result of a SQL select query?
- Column masking use-case - How do I restrict a user from seeing contents of a column by masking the values in the result of a SQL select query?
- Access use-case - How do I disallow a user from seeing certain rows of a table?
- PrivaceraCloud documentation changelog
Starburst Enterprise Presto
Starburst Enterprise platform (SEP) is a commercial distribution of PrestoSQL. It includes additional security features, more connectors, and a cost-based query optimizer not available in the open source version.
As with open source PrestoSQL, SEP is designed to support an external Apache Ranger. This can be configured in the following independent ways:
System-level: Configure SEP so that resource policies defined in PrivaceraCloud under the
privacera_starburstenterprisepresto
resource service control access to Starburst resources.System-Plus-Hive: Configure SEP so that resource policies defined in PrivaceraCloud under both the privacera_starburstenterprisepresto and privacera_hive resource services control access to Starburst resources;
This configuration requires two additional configuration files.
Create a SEP service user
Create a service-user identity that will be used authenticate to your PrivaceraCloud account from the SEP
Go to Access Manager > Users/Groups/Roles, and then create a user. Record the user name. This will be referred to as "${RANGER_API_USERNAME}" in the SEP configuration steps.
Set the Role to Admin and record the password. This will be referred to as "${RANGER_API_PSWD}" in the SEP configuration steps.
Get the account specific API URL
Go to Settings > API Key, and then click GENERATE API KEY.
In the Generate Api Key dialog, set the purpose to REST API Access or similar, and then click the Never Expires check box.
Click the GENERATE API KEY* button.
Click the Copy Url button, and then click Close. Paste and store the URL value. This will be referred to as variable "${RANGER_URL} in the steps that follow.
The API Key page will display the added Api Key.
The Ranger Admin URL (${RANGER_URL}) will look similar to:
https://api.privaceracloud.com/api/13afxxxxxx6b981fxxxxxx2dc7cdd7xxxxxxa921636xxxxxx2d189d425b5f01
A full URL Ranger API service URI is:
<RangerAdminURL>/service/<Ranger API Resource Path>
.
Connect Starburst Enterprise Presto application
Use the following steps to connect Starburst Enterprise Presto application to the PrivaceraCloud for Privacera Access Management.
Go to Settings > Applications.
On the Applications screen, select StarburstStarburst Presto.
Enter the application Name and Description, and then click Save.
Click the toggle button to enable the Privacera Access Management for Starburst Enterprise Presto.
You will see this message: Save the setting to start controlling access on Starburst Enterprise Presto.
Click Save.
The starburst-enterprise-presto service will be available in the Access Manager > Resource Policies section.
Configure Starburst Enterprise (SEP) to use your Account PrivaceraCloud Ranger
SSH to the Hadoop cluster.
Use the following sequence of commands, and using wget, download and extract
starburst presto v350 jar
.mkdir downloads cd downloads wget https://s3.us-east-2.amazonaws.com/software.starburstdata.net/350e/350-e.3/starburst-presto-server-350-e.3.tar.gz -O presto-server.tar.gz presto-server.tar.gz tar zxvf presto-server.tar.gz mv presto-server-350-e.3 presto-server cd presto-server
Create a folder etc in which you need to create files and edit them to add the necessary properties.
mkdir etc cd etc/
Create an SSL truststore to communicate with Apache Ranger. The
chmod
command is used to change permission of theranger.jceks
file.hadoop credential create sslTrustStore -value changeit -provider localjceks://file/home/hadoop/downloads/presto-server/etc/ranger.jceks chmod a+r /home/hadoop/downloads/presto-server/etc/ranger.jceks
Create a catalog directory in which you need to create a
hive.properties
, so that you can use hive as a catalog for query.mkdir catalog
Change the default java interpreter on your cluster. By default it will be set to java 8, change it to java 11.
sudo update-alternatives --config java ---- Select one with Java 11
Now, in the folder etc, you can start configuring properties.
All the following files must be configured:
File | Standard location | Use |
---|---|---|
hive.properties | etc/catalog | Global Hive properties |
config.properties | etc | Points to plugin configuration files |
access-control-privacera.properties | etc | Values for Privacera access control |
ranger-policymgr-ssl.xml | etc | Values for Ranger Policy Manager |
ranger-hive-audit.xml | etc | Values for Ranger Hive and Audit |
access-control-priv-hive.properties | etc | Values for Hive Policies (used only for "System-Plus-Hive" configuration) |
Create the
hive.properties
file.a. Use the following command to create
hive.properties
file in the${PRESTO_CONFIG_PATH}/etc/catalog/
folder:vi hive.properties
b. Add the following content and save this file.
hive.metastore=glue hive.security=allow-all
Create an
access-control-privacera.properties
file.a. Use the following command to create
access-control-privacera.properties
in the${PRESTO_CONFIG_PATH}/etc/
folder:vi access-control-privacera.properties
b. Add the following content, substituting the values for
${RANGER_URL}
, ${RANGER_API_USERNAME}
, and${RANGER_API_PSWD}
, as they are referenced in the text below.Substitute values for
${PRESTO_CONFIG_PATH}
and${PRESTO_TEMP_DIRECTORY}
should correct for your environment.access-control.name=privacera-starburst ranger.policy-rest-url=https://${RANGER_URL} ranger.service-name=privacera_starburstenterprisepresto ranger.presto-plugin-username=${Ranger API username} ranger.presto-plugin-password=${Ranger API user password} ranger.policy-refresh-interval=3s # Example: ranger.config-resources=/usr/presto-server-341-e/etc/ranger-hive-audit.xml ranger.config-resources=${PRESTO_CONFIG_PATH}/etc/ranger-hive-audit.xml # Example: ranger.policy-cache-dir=/tmp/ranger ranger.policy-cache-dir=${PRESTO_TEMP_DIRECTORY} ranger.plugin-policy-ssl-config-file=${PRESTO_CONFIG_PATH}/etc/ranger-policymgr-ssl.xml
c. Save this file.
Create a
ranger-policymgr-ssl.xml
file.a. Use the following command to create
ranger-policymgr-ssl.xml
file in the${PRESTO_CONFIG_PATH}/etc/
folder.vi ranger-policymgr-ssl.xml
b. Add the following XML tags:
```xml <?xml version="1.0" encoding="UTF-8"?> <configuration> <property> <name>xasecure.policymgr.clientssl.truststore</name> <value>${JAVA_PATH}/lib/security/cacerts</value> </property> <property> <name>xasecure.policymgr.clientssl.truststore.password</name> <value>crypted</value> </property> <property> <name>xasecure.policymgr.clientssl.truststore.credential.file</name> <value>jceks://file/home/hadoop/downloads/presto-server/etc/ranger.jceks</value> </property> </configuration> ```
Create
ranger-hive-audit.xml
file.a. Use the following command to create
ranger-hive-audit.xml
file in the${PRESTO_CONFIG_PATH}/etc/
folder.vi ranger-hive-audit.xml
b. Add the following XML tags and substitute
${RANGER_URL}
where used.```xml <?xml version="1.0" encoding="UTF-8"?> <configuration> <property> <name>ranger.plugin.hive.service.name</name> <value>privacera_hive</value> </property> <property> <name>ranger.plugin.hive.policy.pollIntervalMs</name> <value>5000</value> </property> <property> <name>ranger.service.store.rest.url</name> <value> https://${RANGER_URL} </value> </property> <property> <name>ranger.plugin.hive.policy.rest.url</name> <value> https://${RANGER_URL} </value> </property> <property> <name>ranger.plugin.hive.policy.source.impl</name> <value>org.apache.ranger.admin.client.RangerAdminRESTClient</value> <description> Class to retrieve policies from the source </description> </property> <property> <name>ranger.plugin.hive.policy.rest.ssl.config.file</name> <value>/home/hadoop/downloads/presto-server/etc/ranger-policymgr-ssl.xml</value> <description> Path to the file containing SSL details to contact Ranger Admin </description> </property> <property> <name>ranger.service.store.rest.ssl.config.file</name> <value>/home/hadoop/downloads/presto-server/etc/ranger-policymgr-ssl.xml</value> </property> <property> <name>ranger.plugin.hive.policy.cache.dir</name> <value>/tmp/ranger</value> <description> Directory where Ranger policies are cached after successful retrieval from the source </description> </property> <property> <name>ranger.plugin.starburst-enterprise-presto.policy.cache.dir</name> <value>/tmp/ranger</value> <description> Directory where Ranger policies are cached after successful retrieval from the source </description> </property> <property> <name>xasecure.audit.destination.solr</name> <value>true</value> </property> <property> <name>xasecure.audit.destination.solr.batch.filespool.dir</name> <value>presto temp file location</value> </property> <property> <name>xasecure.audit.destination.solr.urls</name> <value> https://${RANGER_AUDIT_URL} </value> </property> <property> <name>xasecure.audit.is.enabled</name> <value>true</value> </property> <property> <name>xasecure.audit.solr.is.enabled</name> <value>true</value> </property> <property> <name>xasecure.audit.solr.async.max.queue.size</name> <value>1</value> </property> <property> <name>xasecure.audit.solr.async.max.flush.interval.ms</name> <value>1000</value> </property> </configuration> ```
Create access-control-priv-hive.properties files.
a. Use the following command to create access-control-priv-hive.properties in the
${PRESTO_CONFIG_PATH}/etc/
folder:vi access-control-priv-hive.properties
b. Add the following content, substituting the values for
${RANGER_URL}
,${RANGER_API_USERNAME}
, and${RANGER_API_PSWD}
, as they are referenced in the text below.access-control.name=privacera ranger.policy-rest-url=https://${RANGER_URL} ranger.service-name=privacera_hive privacera.catalogs=hive ranger.presto-plugin-username=${RANGER_API_USERNAME} ranger.presto-plugin-password=${RANGER_API_PSWD} ranger.policy-refresh-interval=3s # Example: ranger.config-resources=/usr/presto-server-341-e/etc/ranger-hive-audit.xml ranger.config-resources={PRESTO_CONFIG_PATH}/etc/ranger-hive-audit.xml # Example: ranger.policy-cache-dir=/tmp/ranger ranger.policy-cache-dir=${PRESTO_TEMP_DIRECTORY} # Fallback allow-all allows privacera_starburst catalog-level permissions as fallback privacera.fallback-access-control=allow-all ranger.plugin-policy-ssl-config-file={PRESTO_CONFIG_PATH}/etc/ranger-policymgr-ssl.xml ranger.enable-row-filtering=true
If configuring for System-Level only, do not create this file, because you have already done the "System-Level" configuration in access-control-privacera.properties file.
Create
config.properties
file.a. Use the following command to create
config.properties
file in the${PRESTO_CONFIG_PATH}/etc/
folder:vi config.properties
If configuring for System-Level, add the following to this file:
access-control.config-files=etc/access-control-privacera.properties
If configuring for System-Plus-Hive, add the following to this file (note that this is a single line):
access-control.config-files=etc/access-control-privacera.properties,etc access-control-priv-hive.properties
Restart Starburst.