How to list all files in an s3 folder using AWS-SD

Question

I wanted to show a list of all files in an s3 folder so I can get all the last modified dates so I can determine what files has been changed.

I tried using objects.with_prefix('Folder1') it give me a full list but also contain Folder1.1 key

I don't know if i needed to use delimiter but I couldn't find anything how to use delimiter in aws sdk.

Thanks so much in advance!

I'm using 'aws-sdk' gem

Here is my bucket structure -Folder1 -File1 -File2 -Folder.1.1

Here is my code

bucket = s3.buckets[bucket_name]
data = bucket.objects.with_prefix('Folder1/')
data.each do |object|
    puts "#{object.key}\t#{object.last_modified}";
end

Answer 1

Currently I am also stuck with this problem. So far solution is to fetch all the objects and to filter them later:

data = bucket.objects(bucketname, prefix: 'Folder1')

data_without_folders = data.select { |obj| !(obj.key =~ /\/$/) }

For delimiter, you just have to pass it in bucket.objects call like:

data = bucket.objects(bucketname, prefix: 'prefix', delimiter: 'delimiter') 

If better solution is available, I will let you know.

Answer 2

You can use this small piece of code for getting list of files for a specific folder.

 s3 = Aws::S3::Resource.new(region: 'ap-southeast-1', access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'] )
 data_files = s3.bucket(bucket_name).objects(prefix: 'prefix/', delimiter: 'delimiter').collect(&:key)

Answer 3

https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#list_objects_v2-instance_method

SDK V3 has the prefix option for client!

resp = client.list_objects_v2({ bucket: "BucketName", # required prefix: "FolderName", })

Answer 4

Too late answer but better than never.

You can do

s3_bucket.objects.with_prefix('folder_name').collect(&:key)

According to official documentation here