Skip to main content

Privacera Documentation

Configure AWS Lake Formation connector on Privacera Platform

This topic describes how to enable the AWS Lake Formation connector with Privacera Platform.

icon-aws-lake-formation.png

There are two ways to connect to the AWS Lake Formation application:

  • Push mode

  • Pull mode

Implement PolicySync

Overview of steps of PolicySync Setup
  1. Understand how PolicySync works and how it is configured. See PolicySync design and configuration on Privacera Platform to learn more.PolicySync design and configuration on Privacera Platform

  2. Decide which PolicySync topology best suits your needs:

    • Required basic PolicySync topology: always at least one connector instancePolicySync design and configuration on Privacera Platform

    • Optional topology: multiple connector instances for Kubernetes pods and Docker containersPolicySync design and configuration on Privacera Platform

    • Recommended PolicySync topology: individual policy repositories for individual connectorsPolicySync design and configuration on Privacera Platform

  3. Create the PolicySync configuration of at least one connector.

  4. Review the BASIC and ADVANCED properties and decide which features you want to implement. Then set the values in the YAML property file.

Note

When you create the connector, use lakeformation as the connector name. This is a reserved term.

In formal syntax (shown in the Connector instance directory/file structure) replace <ConnectorName> with lakeformation .PolicySync design and configuration on Privacera Platform

Prerequisites

Ensure following prerequisites before configuring AWS Lake Formation connector using Push mode:

  • The AWS Account ID.

  • IAM Role. For more information, see Create IAM Role for AWS Lake Formation connector.

  • In AWS, ensure you have databases and tables in AWS Glue managed by the AWS Lake Formation.

  • To sync permissions for IAM Users/Groups, then these users/groups should be present in Privacera. Ideally, these are synchronized from AD/LDAP or Okta into Privacera, but can also be added manually in Privacera. If the users/groups are not in Privacera, then these permissions will not be synchronized.

    Note

    For IAM Roles, Privacera will automatically sync the IAM Roles as Apache Ranger Roles into Privacera.

Procedure

Perform following steps to configure AWS Lake Formation connector using Push mode:

  1. SSH to the instance where Privacera is installed.

  2. Run the following command to navigate to the /config directory.

    cd ~/privacera/privacera-manager/config
  3. Run the following command to create a new directory:

    mkdir -p custom-vars/connectors/lakeformation/instance1
  4. Run the following command to copy the sample vars:

    cp sample-vars/vars.connector.lakeformation.push.yml custom-vars/connectors/lakeformation/instance1/
  5. Run the following command to open the .yml file to be edited.

    vi custom-vars/connectors/lakeformation/instance1/vars.connector.lakeformation.push.yml
  6. Modify the following properties:

    • CONNECTOR_LAKEFORMATION_AWS_ACCOUNT_ID - Enter the AWS Account ID of the account you will be running the AWS Lake Formation connector.

    • CONNECTOR_LAKEFORMATION_AWS_REGION - Set AWS region to connect to your AWS Lake Formation instance.

    • CONNECTOR_LAKEFORMATION_ENABLE_PUSH_POLICIES_TO_RANGER - Set this to true, if you want to push policies to other policy repositories.

    • CONNECTOR_LAKEFORMATION_SINK_HIVE_SERVICE_APP_ID - Set the policy repository name where you want the connector to push policies for the hive.

    Note

    The # (hash) symbol before any property indicates it is commented out. You need have to remove the # sign and space if you want to uncomment that property. Click here to see more on property details and description.

  7. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    cd ~/privacera/privacera-manager
    ./privacera-manager.sh update

The following points are to be considered when synchronizing policies in Ranger for AWS Lake Formation:

Synchronizing Ranger Column Exclude Policies to AWS Lake formation Column Exclude Policies

When the ranger policies with exclude columns synchronized into AWS Lake formation, it converts it into permissions with include column policies.

For example:

Consider a table with the columns country, id, region, sales_amount, city, and name. Ranger policy with SELECT permission on columns excluding city and name columns is converted into AWS Lake Formation policy with SELECT permission on columns including country, id, region, and sales_amount.

Synchronizing Ranger Tag Policies to AWS Lake formation Tag Policies

For the tag policy created in Ranger, it internally fetches the resource attached to that tag and applies the permissions on the actual resource in AWS Lake Formation.

For example

You have attached the PII tag to a table resource called sales_data, and then you created a tag based policy with SELECT permission for user emily on the PII tag. This internally gets the resource (i.e., table resource sales_data) attached to the PII tag and applies SELECT permission to user emily on the actual table sales_data in the AWS Lake Formation.

Multiple Row Filter Policy Items Enforcement Behavior difference in AWS Lake Formation

When you add a row filter policy in Ranger with a row filter condition, then it creates a data filter with the same row filter condition inside the AWS Lake Formation. If you add multiple row filter items with different row filter conditions inside the row filter policy in Ranger, it creates those many data filters inside the lake formation and applies permissions on top of that.

For example:

If you create a Row-Level Filter policy on the sales_data table with two row filter items as below:

  • SELECT permission with row filter condition country='US' to user emily

  • SELECT permission with row filter condition id=4 to user emily

When the AWS Lake Formation engine enforces these permissions, when the user emily queries the sales_data table from Athena, it gets the result that is the intersection of both data filters. That is, it only gets one row with id=4 and country='US' in the result.

select * from sales_data;

4,'US','Mountain','Palmertown','Sarah','50771.9'

No Access control on IAM Groups

AWS Lake Formation does not support IAM groups, but it does support AD groups, and you can add policies to AD groups.

When granting access to IAM Role, it assumes the IAM role is present on the AWS console

When you create a policy for a role in Ranger, you assume that the IAM role is present on the AWS console with the same name as the role name in Ranger and just try to assign permissions to that role. It doesn’t create any role if it's not present in the AWS Lake Formation.

Prerequisites

Ensure following prerequisites before configuring AWS Lake Formation connector using Pull mode:

  • The AWS Account ID.

  • The AWS IAM Role, For more information see Create IAM Role for AWS Lake Formation connector

  • In AWS, ensure that databases and tables are in AWS Glue managed by Lake Formation.

  • To sync permissions for IAM Users/Groups, then these users/groups should be present in Privacera. Ideally, these are synchronized from AD/LDAP or Okta into Privacera, but can also be added manually in Privacera. If the users/groups are not in Privacera, then these permissions will not be synchronized.

    Note

    For IAM Roles, Privacera will automatically sync the IAM Roles as Apache Ranger Roles into Privacera.

Procedure

Perform following steps to configure AWS Lake Formation connector using Pull mode:

  1. SSH to the instance where Privacera is installed.

  2. Run the following command to navigate to the /config directory.

    cd ~/privacera/privacera-manager/config
  3. Run the following command to create a new directory:

    mkdir -p custom-vars/connectors/lakeformation/instance1
  4. Run the following command to copy the sample vars:

    cp sample-vars/vars.connector.lakeformation.pull.yml custom-vars/connectors/lakeformation/instance1
  5. Run the following command to open the .yml file to be edited:

    vi custom-vars/connectors/lakeformation/instance1/vars.connector.lakeformation.pull.yml
  6. Modify the following properties:

    • CONNECTOR_LAKEFORMATION_AWS_ACCOUNT_ID - Enter the AWS Account ID of the account you will be running the AWS Lake Formation connector.

    • CONNECTOR_LAKEFORMATION_AWS_REGION - Set AWS region to connect to your AWS Lake Formation instance.

    Note

    The # (hash) symbol before any property indicates it is commented out. You need have to remove the # sign and space if you want to uncomment that property. Click here to see more on property details and description.

  7. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    cd ~/privacera/privacera-manager
    ./privacera-manager.sh update