我正在研究一个项目,以允许用户在我们的存储帐户中将BLOB上传到BLOB容器中。我使用Azure App Service开发了一个简单的UI(烧瓶),以允许用户选择文件上传,然后要将这些文件上传到Blob容器。
我最初的设计是UI - > Python存储SDK的BLOB容器:
containerClient.upload_blob(filename, file)
但是,在上传大文件时,由于Azure App Service,我正面临超时问题。
因此,我使用dropzone.js更改上传UI,然后在块中启用上传,以便服务器可以始终接收响应以防止超时。
而另一个问题是,在每个块上执行上传过程,并且Blob容器仅接收我上传的数据的最后一部分。 (从文档中,我知道块自动用于blob上传,我想知道我们是否能够跟踪上传的进度?块)。
我还通过创建Azure App功能(HTTPS触发器)尝试了另一种方法,然后将HTTP触发器发送到该端点以启动Blob上传。
for file in files:
fileToSend = {'file': (f.filename, f.stream, f.content_type, f.headers)}
r = requests.post('https://myazurefunctionapp.azurewebsites.net/api/funcName', files=fileToSend)
在Azure函数中,我使用Python Storage SDK连接到容器,然后上传Blob,
container = ContainerClient.from_connection_string(conn_str, container_name)
for k, f in req.files.items():
container.upload_blob(f.filename, f)
但我注意到该函数是由块(请求)触发的,我最终还只能在容器中接收最后一块数据。
我想知道更好的工作流程会怎样?或者,如果有任何方法确保上传完成(以Azure函数为单位),然后启动上传到BLOB容器。
非常感谢
I am working on a project to allow users to upload blob into blob container in our storage account. I developed a simple UI (flask) using Azure App Service to allow user choose files to upload, and then want to upload these files to the blob container.
My original design is UI -> Blob Container by Python Storage SDK:
containerClient.upload_blob(filename, file)
But I am facing the timeout issue due to Azure App Service when uploading large files.
So I change the upload UI with dropzone.js, and enable uploading in chunk, so that the server can consistently receive response to prevent timeout.
And another issue coming up is that upload process is executed for every piece of chunk, and blob container only receives the last chunk of the data that I upload. (From the document, I know that the chunking is automatically used in blob upload, I wonder if we are able to track the progress of the upload??? if so, I probably don't need to use dropzone.js for uploading in chunk).
I also tried another approach by creating Azure App Function (HTTPS trigger), and then send an http trigger to that endpoint to start the blob upload.
for file in files:
fileToSend = {'file': (f.filename, f.stream, f.content_type, f.headers)}
r = requests.post('https://myazurefunctionapp.azurewebsites.net/api/funcName', files=fileToSend)
In the azure function, I use Python Storage SDK to connect to container and then upload blob
container = ContainerClient.from_connection_string(conn_str, container_name)
for k, f in req.files.items():
container.upload_blob(f.filename, f)
But I notice that the function is triggered by piece of chunk (request), and I also end up with only receiving the last chunk of data in the container.
I wonder what would be the better workflow? or if there any way that makes sure the upload is completed (in azure function) and then start the upload to blob container.
Many Thanks,
发布评论
评论(1)
•存储客户端默认为32 MB最大单个块上传。当块斑点上传大于中的值时,'singleblobuploploploadthresholdinbytes' 属性时,存储客户端将文件分解为最大允许大小的块,并尝试将其上传。由于您试图上传的块斑点大小大于32 MB,因此会引发异常,并将文件分解为允许较小的块。另外,您可能不会使用正确的 'BLOB服务客户端',该客户端与资源相互作用,即,即存储帐户,BLOB存储容器和Blobs 。
以下是客户端对象创建的代码的示例,该代码需要存储帐户的BLOB服务帐户URL和凭据,该凭据允许您访问存储帐户: -
•因此,同样,您正在使用该帐户。 python中的代码上面的代码创建一个blob服务客户端以与存储帐户进行交互,请参考下面的文档链接,该链接详细描述了如何开发 python代码,以将其与blob存储集成在一起,以存储大量的非结构性信息数据,例如文本或二进制数据。
您可以 您的应用程序服务或功能中的代码,并相应地设置触发器,以从存储帐户上传和下载BLOB 。它还描述了如何为此过程配置身份验证,以确保给出正确的用户和文件访问。
并请参阅文档链接,以获取有关如何在任何用户通过IT启动任何交易 启动任何交易时,在Azure中配置Blob触发函数的详细信息。
• Storage clients default to a 32 MB maximum single block upload. When a block blob upload is larger than the value in ‘SingleBlobUploadThresholdInBytes’ property, storage clients break the file into blocks of maximum allowed size and try to upload it. Since the block blob size that you are trying to upload is greater than 32 MB, it throws an exception and breaks the file into allowed smaller chunks. Also, you might not be using the correct ‘Blob service client’ which interacts with the resources, i.e., storage account, blob storage containers and blobs.
Below is an example of the code for client object creation which requires a storage account’s blob service account URL and a credential that allows you to access a storage account: -
• Thus, similarly, as you are using the above code in python to create a blob service client for interacting with storage accounts, kindly refer to the below documentation link that describes in detail as in how to develop a python code to integrate it with blob storage for storing massive amounts of unstructured data, such as text or binary data.
https://learn.microsoft.com/en-us/python/api/overview/azure/storage-blob-readme?view=azure-python
You can deploy this code in your app service or function and set the trigger accordingly for uploading and downloading blobs from the storage account. It also describes as in how you can configure authentication for this process to ensure that the correct user and files are being given access.
And refer to the documentation link for details on how to configure a blob trigger function in Azure for various interactions with the storage account when any users initiate any transaction through it.
https://learn.microsoft.com/en-us/azure/storage/blobs/blob-upload-function-trigger?tabs=azure-portal