如何使用 python 列出 S3 存储桶文件夹中的文件

发布于 2025-01-11 11:55:54 字数 1025 浏览 0 评论 0原文

我尝试列出存储桶中的所有文件。这是我的代码，

import boto3
s3 = boto3.resource('s3')
my_bucket = s3.Bucket('my_project')

for my_bucket_object in my_bucket.objects.all():
    print(my_bucket_object.key)

它可以工作。我得到了所有文件的名称。但是，当我尝试对文件夹执行相同的操作时，代码会引发错误

import boto3
s3 = boto3.resource('s3')
my_bucket = s3.Bucket('my_project/data/') # add the folder name

for my_bucket_object in my_bucket.objects.all():
    print(my_bucket_object.key)

，错误如下：

botocore.exceptions.ParamValidationError: Parameter validation failed:

Invalid bucket name "carlos-cryptocurrency-research-project/data/": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"

我确信文件夹名称是正确的，我尝试将其替换为 Amazon 资源名称 (ARN) 和 S3 URI，但是仍然收到错误。

原文

I tried to list all files in a bucket. Here is my code

import boto3
s3 = boto3.resource('s3')
my_bucket = s3.Bucket('my_project')

for my_bucket_object in my_bucket.objects.all():
    print(my_bucket_object.key)

it works. I get all files' names. However, when I tried to do the same thing on a folder, the code raise an error

import boto3
s3 = boto3.resource('s3')
my_bucket = s3.Bucket('my_project/data/') # add the folder name

for my_bucket_object in my_bucket.objects.all():
    print(my_bucket_object.key)

Here is the error:

botocore.exceptions.ParamValidationError: Parameter validation failed:

Invalid bucket name "carlos-cryptocurrency-research-project/data/": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}
quot; or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}quot;

I'm sure the folder name is correct and I tried replacing it with Amazon Resource Name (ARN) and S3 URI, but still get the error.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

那支青花 2025-01-18 11:55:54

您不能在 Bucket 构造函数中指定前缀/文件夹。请改用客户端级 API 并调用 list_objects_v2 类似这样的内容：

import boto3

client = boto3.client('s3')

response = client.list_objects_v2(
    Bucket='my_bucket',
    Prefix='data/')

for content in response.get('Contents', []):
    print(content['Key'])

请注意，这最多会产生 1000 个 S3 对象。您可以使用如果需要，可以使用分页器，或者考虑使用更高级别的存储桶资源及其对象集合根据此问题的另一个答案，它可以为您处理分页。

You can't indicate a prefix/folder in the Bucket constructor. Instead use the client-level API and call list_objects_v2 something like this:

import boto3

client = boto3.client('s3')

response = client.list_objects_v2(
    Bucket='my_bucket',
    Prefix='data/')

for content in response.get('Contents', []):
    print(content['Key'])

Note that this will yield at most 1000 S3 objects. You can use a paginator if needed, or consider using the higher-level Bucket resource and its objects collection which handles pagination for you, per another answer to this question.

回复收藏 0 原文

初懵 2025-01-18 11:55:54

获取s3 Bucket中特定文件夹中的所有文件列表

import boto3

s3 = boto3.resource('s3')
myBucket = s3.Bucket('bucketName')


for object_summary in myBucket.objects.filter(Prefix="path/"):
    print(object_summary.key)

Get all the list of files in specific folder in s3 Bucket

import boto3

s3 = boto3.resource('s3')
myBucket = s3.Bucket('bucketName')


for object_summary in myBucket.objects.filter(Prefix="path/"):
    print(object_summary.key)

回复收藏 0 原文

~没有更多了~