Skip to content

Databricks Policy Management#

Privacera Manager#

  • Databricks Integration - AWS can now be done directly using Privacera Manager. See topic: Databricks - AWS.

  • Databricks Integration - Azure can now be done directly using Privacera Manager. See topic: Databricks - Azure.

Create Policy in Portal#

To create a policy in Privacera Portal, use the following steps:

  1. Login to Privacera Portal.

  2. On the Privacera home page, expand the Settings menu and click on Databricks Policies from left menu.

  3. Click the +Create Policy.

  4. Enter the Policy Name. (Mandatory)

  5. Select the Users, Groups, IAM Role from the drop-down.

    Note

    You can select multiple Users and Groups.

  6. Enter the Additional JSON (If any). This will append with the existing JSON which will be fetched from backend.

  7. Click Save.

    The policy is created successfully.

Note

By default, Admin groups will have permission to all the policies. If you have not configured Databricks properties in Privacera Portal properties file then you will get the following error.

  • The Token should be generated from a user who is an Admin.

  • Additional JSON that can be used to create policy.

    {
    "autoscale.min_workers": {
            "type": "range",
            "minValue": 1,
            "hidden": false
        },
        "autoscale.max_workers": {
            "type": "range",
            "maxValue": 2
        },
        "cluster_name": {
            "type": "fixed",
            "value": "secured"
        },
        "spark_version": {
            "type": "regex",
            "pattern": "5.5.x-scala2.11"
        },
        "spark_conf.spark.hadoop.hadoop.security.credential.provider.path": {
            "type": "fixed",
            "value": "jceks://dbfs@/${JCEKS_FILE_PATH}",
            "hidden": true
        },
        "spark_conf.spark.databricks.delta.formatCheck.enabled": {
            "type": "fixed",
            "value": "false",
            "hidden": true
        },
        "spark_conf.spark.databricks.delta.preview.enabled": {
            "type": "fixed",
            "value": "true",
            "hidden": true
        },
        "node_type_id": {
            "type": "regex",
            "pattern": "m4.*"
        },
        "autotermination_minutes": {
            "type": "unlimited",
            "defaultValue": 50
        }
    }
    

Create Cluster in Databricks#

To create a Cluster in Databricks through policy, use the following steps:

  1. Login to Databricks.

  2. Click on Clusters from left menu.

  3. Click on Create Cluster.

  4. Select the Policy from the drop down.

  5. Enter the the required details.

  6. Click on Create Cluster.

The Cluster is created successfully.

Supported Actions#

Policy

  • Create: Setting users and group permissions

  • Update:Setting users and group permissions

  • Delete:

Form elements

  • Ranger Enabled

    • True: Compulsory JSON will be added from backend

    • False:Compulsory JSON will not be added from the backend.

IAM role (Optional)

If selected then the below JSON value will be added from backend.

{
   "aws_attributes.instance_profile_arn":{
      "type":"fixed",
      "value":{SELECTED_VALUE},
      "hidden":false
   }
}

Last update: August 24, 2021