Skip to content

Qubole Cluster Setup

Connect Presto on Qubole Cluster PrivaceraCloud#

PrivaceraCloud uses a plug-in to integrate with your Qubole Presto cluster.

Connecting your Qubole Presto cluster to PrivaceraCloud consists of the following general steps.

  • Create a service user on PrivaceraCloud for data user access control call-in from Presto to PrivaceraCloud.

  • Create, or identify and use an existing, unique call-in authentication (access control) and audit URLs from your Qubole Presto cluster to PrivaceraCloud.

  • Configure your Qubole Presto cluster to first load the necessary Privacera hosted Apache Ranger Plug in components (on boot), and execute the call-in for access control and audit.

Those steps are detailed below.

PrivaceraCloud Steps#

  1. Create a new data access service user for interaction with Qubole.

    1. Open Access Manager: Users/Groups/Roles and Click on + Add.
    2. Create a new service data access user. Assign it an Admin role. Record the User Name and Password. These are referred to as ADMIN_ROLE_USER and ADMIN_ROLE_PASSWORD in the following steps and will be substituted in configuration properties.
  2. Obtain Api Key associated "Ranger" URLs for call back from Qubole Cluster to Privacera.

    1. Open Settings: Api Key.
    2. You can use an existing Active Api Key or create a new one. Expiry = Never Expires is recommended.
    3. Open the Api Key Info box (click the (i) in the key row).
    4. Copy and store the values for each of the Ranger Admin URL and Ranger Audit URL. These will be referenced as RANGER_ADMIN_URL and RANGER_AUDIT_URL in the following steps.

Presto Qubole Console Steps#

  1. Open or create a new Presto Cluster.

  2. Proceed to "Advanced Configuration".

  3. In the "PRESTO SETTINGS" section override the Presto Configuration with the following changes. Substitute values obtained above for ADMIN_ROLE_USER, ADMIN_ROLE_PASSWORD, RANGER_ADMIN_URL, and RANGER_AUDIT_URL.

         bootstrap.properties:
         mkdir -p /media/ephemeral0/rangerssl/
         hadoop credential create sslTrustStore -value changeit -provider localjceks://file/media/ephemeral0/rangerssl/ranger.jceks
         chmod a+r /media/ephemeral0/rangerssl/ranger.jceks
         wget https://privacera-public1.s3.amazonaws.com/0001-httpcore-4.4.14.jar -P /usr/lib/presto/plugin/ranger
    
         access-control.properties:
         access-control.name=ranger-access-control
         ranger.username=<ADMIN_ROLE_USER>
         ranger.password=<ADMIN_ROLE_USER_PASSWORD>
         ranger.hive.security-config-xml=/usr/lib/presto/etc/ranger-hive-security.xml
         ranger.hive.audit-config-xml=/usr/lib/presto/etc/ranger-hive-audit.xml
    
         ranger-hive-security.xml:
         <configuration>
         <property>
              <name>ranger.plugin.hive.service.name</name>
              <value>privacera_hive</value>
         </property>
         <property>
              <name>ranger.plugin.hive.policy.pollIntervalMs</name>
              <value>5000</value>
         </property>
         <property>
              <name>ranger.service.store.rest.url</name>
              <value>
                   <RANGER_ADMIN_URL>
              </value>
         </property>
         <property>
              <name>ranger.plugin.hive.policy.rest.url</name>
              <value>
                   <RANGER_ADMIN_URL>
              </value>
         </property>
         <property>
              <name>ranger.service.store.rest.ssl.config.file</name>
              <value>/usr/lib/presto/etc/ranger-ssl.xml</value>
         </property>
         <property>
              <name>ranger.plugin.hive.policy.rest.ssl.config.file</name>
              <value>/usr/lib/presto/etc/ranger-ssl.xml</value>
         </property>
         </configuration>
    
    ranger-ssl.xml:
         <configuration>
         <property>
              <name>xasecure.policymgr.clientssl.truststore</name>
              <value>/etc/pki/ca-trust/extracted/java/cacerts</value>
         </property>
         <property>
              <name>xasecure.policymgr.clientssl.truststore.password</name>
              <value>crypted</value>
         </property>
         <property>
              <name>xasecure.policymgr.clientssl.truststore.credential.file</name>
              <value>jceks://file/media/ephemeral0/rangerssl/ranger.jceks</value>
         </property>
         </configuration>
    
    ranger-hive-audit.xml:
         <configuration>
         <property>
              <name>xasecure.audit.is.enabled</name>
              <value>true</value>
         </property>
         <property>
              <name>xasecure.audit.solr.is.enabled</name>
              <value>true</value>
         </property>
         <property>
              <name>xasecure.audit.solr.async.max.queue.size</name>
              <value>1</value>
         </property>
         <property>
              <name>xasecure.audit.solr.async.max.flush.interval.ms</name>
              <value>1000</value>
         </property>
         <property>
              <name>xasecure.audit.solr.solr_url</name>
              <value>
                   <RANGER_AUDIT_URL>
              </value>
         </property>
         </configuration>
    

  4. Click on Update/Update and Push.

  5. Start/Stop and Start the cluster.


Last update: August 20, 2021