从Google驱动器到Google Cloud Storage,一次上传多个文件

发布于 2025-02-08 10:15:52 字数 3036 浏览 2 评论 0原文

我的目标是在Google驱动器中识别一些文件,并将它们顺序上传到Google存储存储桶中。我正在使用Google的“云功能”来执行此操作,并已经进行了测试以确认连接正常工作。

我似乎与我如何获取文件的名称有关的问题 - 它正在返回“无类型”值。请在下面查看我的代码

    import os.path
    from google.auth.transport.requests import Request
    from google.oauth2.credentials import Credentials
    from google_auth_oauthlib.flow import InstalledAppFlow
    from googleapiclient.discovery import build
    from googleapiclient.errors import HttpError
    from google.cloud import storage
    import io
    from googleapiclient import discovery
    from pandas.io.json import json_normalize
    import google.auth
    import re
    import logging
    
    # Confirming Oauth for Drive API# 
    SCOPES = [ 'https://www.googleapis.com/auth/drive']
    creds, project = google.auth.default(scopes=SCOPES)
    service = build('drive', 'v3', credentials=creds)
    
    # Confirming Oauth #
    storage_client = storage.Client()
    ## target Bucket #
    bucket_name = 'my-bucket'
    bucket = storage_client.bucket(bucket_name)

    get_bucket = storage_client.get_bucket(bucket_name)
    team_drive_loc = 
    'https://drive.google.com/drive/u/0/folders/xxxxxxxxxxx'
    team_drive_parent_folder ='xxxxxxxxxxxxxxxxxA'
    #bucket = storage_client.bucket(bucket_name)
    
    
    query= "name contains 'Customer' and name contains '2022'    "
    drive_files = service.files().list(q= query,
                                 driveId =team_drive_parent_folder,
                                 supportsAllDrives= True,
                                 includeItemsFromAllDrives= True,
                                 corpora ='drive',fields="files(id,name)").execute()

    for file in drive_files:

            source_file_name =service.get(fileId=fileId, fields="files(name)").execute()["files(name)"]
            bucket = storage_client.get_bucket(bucket_name)
            blob = bucket.blob("incoming/iri/IRI_Updates/Ongoing_Sales_Data/2022/" + 
            source_file_name)
            blob.upload_from_filename(source_file_name)
            logging.info('Uploaded {} ...'.format(source_file_name))

...这是我遇到的错误。如果有人可以帮助我正确地源源并上传到GCS存储桶,那将是

/[post] trackback上的例外很棒的exceptions(最新呼叫最后一个):文件“/layers/google.python.pip/pip/pip/pip/pip /lib/python3.7/site-packages/flask/app.py”,第2073行,在WSGI_APP响应中-packages/flask/app.py”,第1518行,在full_dispatch_request rv = self.handle_user_expection(e)文件“/layers/google.python.pip/pip/pip/pip/lib/python3.7/site-packages/site-packages/site-packages/flask/app/app-app。 py”,第1516行,在full_dispatch_request rv = self.dispatch_request()文件“/layers/google.python.pip/pip/pip/pip/lib/python3.7/site-packages/flask/flask/app.py返回self.ensure_sync(self.view_functions [rule.endpoint])(** req.view_args)文件“/layers/google.python.pip/pip/pip/pip/pip/lib/python3.7/site-packages/site-packages/functions_framework/functions_framework/__init__.init__.py _.y.py” ,第171行,在view_func function(数据,上下文)中文件“/workspace/main.py”,第86行,在iri_data_sync源[“ files(name)”] attributeError:'资源'对象没有属性'get'

my objective is to identify some files in google drive and upload them sequentially to a google storage bucket. I am using a google 'cloud function' to do this and have already done tests to confirm that the connection is working properly.

The issue I have seems to relate to how I get the name of the file - it is returning a 'none type' value. Please see my code below

    import os.path
    from google.auth.transport.requests import Request
    from google.oauth2.credentials import Credentials
    from google_auth_oauthlib.flow import InstalledAppFlow
    from googleapiclient.discovery import build
    from googleapiclient.errors import HttpError
    from google.cloud import storage
    import io
    from googleapiclient import discovery
    from pandas.io.json import json_normalize
    import google.auth
    import re
    import logging
    
    # Confirming Oauth for Drive API# 
    SCOPES = [ 'https://www.googleapis.com/auth/drive']
    creds, project = google.auth.default(scopes=SCOPES)
    service = build('drive', 'v3', credentials=creds)
    
    # Confirming Oauth #
    storage_client = storage.Client()
    ## target Bucket #
    bucket_name = 'my-bucket'
    bucket = storage_client.bucket(bucket_name)

    get_bucket = storage_client.get_bucket(bucket_name)
    team_drive_loc = 
    'https://drive.google.com/drive/u/0/folders/xxxxxxxxxxx'
    team_drive_parent_folder ='xxxxxxxxxxxxxxxxxA'
    #bucket = storage_client.bucket(bucket_name)
    
    
    query= "name contains 'Customer' and name contains '2022'    "
    drive_files = service.files().list(q= query,
                                 driveId =team_drive_parent_folder,
                                 supportsAllDrives= True,
                                 includeItemsFromAllDrives= True,
                                 corpora ='drive',fields="files(id,name)").execute()

    for file in drive_files:

            source_file_name =service.get(fileId=fileId, fields="files(name)").execute()["files(name)"]
            bucket = storage_client.get_bucket(bucket_name)
            blob = bucket.blob("incoming/iri/IRI_Updates/Ongoing_Sales_Data/2022/" + 
            source_file_name)
            blob.upload_from_filename(source_file_name)
            logging.info('Uploaded {} ...'.format(source_file_name))

...And this is the error i get. If anyone can help me source the file name correctly and upload to the gcs bucket, that would be great

Exception on / [POST] Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app response = self.full_dispatch_request() File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request rv = self.handle_user_exception(e) File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request rv = self.dispatch_request() File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) File "/layers/google.python.pip/pip/lib/python3.7/site-packages/functions_framework/__init__.py", line 171, in view_func function(data, context) File "/workspace/main.py", line 86, in iri_data_sync source_file_name =service.get(fileId=fileId, fields="files(name)").execute()["files(name)"] AttributeError: 'Resource' object has no attribute 'get'

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

多情癖 2025-02-15 10:15:52

错误消息告诉您问题

'资源'对象没有属性'get'

而不是service.get service.files()。get

source_file_name =service.files().get(fileId=fileId, fields="files(name)").execute()["files(name)"]
        

您实际上使用service.files( ).LIST您刚刚删除了使用Get请求忘记它。

The error message is telling you the issue

'Resource' object has no attribute 'get'

its not service.get its service.files().get

source_file_name =service.files().get(fileId=fileId, fields="files(name)").execute()["files(name)"]
        

you actually had it right with service.files().list you just removed forgot it with the get request.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文