AppEngine：从获取的 URL 上传到 S3 时仅获取 1KB

发布于 2024-10-20 17:38:56 字数 1332 浏览 3 评论 0原文

我正在尝试上传从服务器获取的文件。但当获取的文件大于 1MB 时，S3 上将仅保存一个 1KB 的小文件。当它小于1MB时，文件将被正确保存。

我搜索并尝试了不同的方法，但没有一个有效。这是由用户单击按钮触发的类。

` class Fetch_by_button(webapp.RequestHandler)：

def get(self):
    #1. fetch the file
    try: 
        result = urlfetch.fetch('http://someurl.com/music.mp3') # file with 2MB
        #result = urlfetch.fetch('http://anotherurl.com/document.pdf') #file with 900KB is working
    except DownloadError:
        print("URL fetch doesnt work properly")

    #2. search datastore UserData-model by userobject to get s3 credentials
    usr = users.get_current_user()
    query = UserData.gql("WHERE account =:1", usr)
    qr = query.get()

    #3. establish connection to s3
    conn = S3Connection(qr.s3_accesskey, qr.s3_secret_accesskey)

    #4. create bucket
    ak_lowercase = str.lower(str(qr.s3_accesskey))
    individual_bucket_name = ak_lowercase + '_' + usr.nickname()
    bucket = conn.create_bucket(individual_bucket_name)

    #5. generate key and assign to already created bucket
    k = Key(bucket)

    #6. name the key by some examplary filename
    k.key = 'justsomefilename'

    #7. save the correspondent value to the created key
    k.set_contents_from_string(result.content)`

我只是不知道为什么会发生这种情况。我的代码、一些 AppEngine 限制、boto 的任何内容，...有什么建议吗？

提前致谢。

原文

I´m trying to upload a file which is fetched from a server. But when the fetched file is bigger than 1MB there will be only a 1KB small file saved at S3. When it´s smaller than 1MB, the file will be saved properly.

I ´ve searched and tried different ways, but none off them is working. Here is the class which is triggered by the user clicking a button.

`
class Fetch_by_button(webapp.RequestHandler):

def get(self):
    #1. fetch the file
    try: 
        result = urlfetch.fetch('http://someurl.com/music.mp3') # file with 2MB
        #result = urlfetch.fetch('http://anotherurl.com/document.pdf') #file with 900KB is working
    except DownloadError:
        print("URL fetch doesnt work properly")

    #2. search datastore UserData-model by userobject to get s3 credentials
    usr = users.get_current_user()
    query = UserData.gql("WHERE account =:1", usr)
    qr = query.get()

    #3. establish connection to s3
    conn = S3Connection(qr.s3_accesskey, qr.s3_secret_accesskey)

    #4. create bucket
    ak_lowercase = str.lower(str(qr.s3_accesskey))
    individual_bucket_name = ak_lowercase + '_' + usr.nickname()
    bucket = conn.create_bucket(individual_bucket_name)

    #5. generate key and assign to already created bucket
    k = Key(bucket)

    #6. name the key by some examplary filename
    k.key = 'justsomefilename'

    #7. save the correspondent value to the created key
    k.set_contents_from_string(result.content)`

I just don´t have any clue why this happens. Something with my code, some AppEngine restrictions, anything with boto, ... Any suggestions?

Thanks in advance.

分享到QQ

分享到微博