Skip to main content

Privacera Documentation

Access Firehose with Data Access Server on Privacera Platform

Create a Firehose delivery stream and set up access control for it.

Firehose delivery stream queries

  1. From a terminal prompt, create a new delivery datastream 'SalesDataDeliveryStream'.

    • (–delivery-stream-type : KinesisStreamAsSource)

      aws firehose create-delivery-stream --delivery-stream-name SalesDataDeliveryStream --delivery-stream-type KinesisStreamAsSource --kinesis-stream-source-configuration "KinesisStreamARN=arn:aws:kinesis:us-east-1:857494200836:stream SalesDataStream,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role" --extended-s3-destination-configuration "BucketARN=arn:aws:s3:::sales-data-stream-bucket,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role" --region us-east-1
      
    • Options:

      • -delivery-stream-name: delivery datastream name

      • -delivery-stream-type: input source type (Kinesis DataStream or DirectPut)

      • -kinesis-stream-source-configuration (Source Kinesis DataStream ARN)

      • -extended-s3-destination-configuration (Destination S3 bucket ARN)

      • -region

    It will show the following result An error occurred (403)

    • (–delivery-stream-type : DirectPut)3 nju88ik;

      aws firehose create-delivery-stream --delivery-stream-name SalesDataDeliveryStream --delivery-stream-type DirectPut --extended-s3-destination-configuration "BucketARN=arn:aws:s3:::sales-data-stream-bucket,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role” --region us-east-1

    It will show the following result An error occurred (403).

    This indicates that the current user does not have permission to perform this operation.

  2. Check the audit log for the related event (Access Manager > Audit).

  3. Create three Ranger policies for this scenario.

    • Firehose Policy to have CreateDeliveryStream on SalesDataDeliveryStream.

    • S3 Policy to provide on input-data bucket location and for OutputLocation to query.

    • Kinesis Policy required for source input (Required if –delivery-stream-type : KinesisStreamAsSource).

Create Firehose policy

  1. From the navigation menu, select Access Management > Resource Policies.

  2. On the Resource Policies page, go to privacera_kinesis and then select Firehose to create policy for Firehose.

  3. Enter the following details:

    • Policy Name: SalesDataDeliveryStreamPolicy

    • kinesis_firehose: SalesDataDeliveryStream (Firehose datastream which you want to allow the user to create)

  4. Under Allow Conditions, click the '+' icon and select the below:

    • User: User’s username to which you want to allow access.

    • Add Permission as: CreateDeliveryStream

Create S3 policy

  1. On the Resource Policies page, go to privacera_kinesis and then click Add New Policy to create policy for S3.

  2. Enter the following details:

    • Policy Name: SalesDataDeliveryStreamPolicy

    • Bucket: sales-data-stream-bucket (destination S3 bucket)

    • Object: *

  3. Under Allow Conditions, click '+' icon and select the below:

    • User: username to which you want to allow access.

    • Permission: metadata read, metadata write, write

Create Kinesis policy

  1. On the Resource Policies page, go to privacera_kinesis and click Add New Policy and then select Firehose to create policy for Kinesis.

  2. Enter the following details:

    • Policy Name: SalesDataStreamPolicy

    • kinesis_datastream: SalesDataStream

    • Under Allow Conditions, click '+' icon and select the below:

      • User: User’s username to which you want to allow access.

      • Permission: GetRecords, GetShardIterartor, DescribeStream

  3. Run the query from step 1.

    aws firehose create-delivery-stream --delivery-stream-name SalesDataDeliveryStream --delivery-stream-type KinesisStreamAsSource --kinesis-stream-source-configuration "KinesisStreamARN=arn:aws:kinesis:us-east-1:857494200836:stream/SalesDataStream,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role" --extended-s3-destination-configuration "BucketARN=arn:aws:s3:::sales-data-stream-bucket,RoleARN=arn:aws:iam::857494200836:role/user_role" --region us-east-1