Behavior of trailing `/` on Datasources

If Datasources have trailing /, then the folders in the bucket are scanned individually otherwise they are scanned together.

For example, with S3 buckets:

The following three folders are in a S3 bucket:
A, A_1, A_1_2

If these three folders need to be scanned individually, then the URL/URI in datasource should be listed as:

  • s3://bucket/A/
  • s3://bucket/A_1/
  • s3://buket/A_1_2/

or

If the three folders need to be scanned together, then the URL/URI in datasource should be listed as:

s3://buket/A or if you want to scan A_1, A_1_2 then, use s3://bucket/A_1. The s3://bucket/A_1 will scan s3://bucket/A_1 and s3://buket/A_1_2 as well.