GZZ Memorystream到S3桶散列错误

发布于 2025-02-10 07:30:59 字数 1699 浏览 2 评论 0原文

我正在尝试将一个对象序列化为JSON(Newtonsoft.json),并将其存储在GZIP压缩形式的S3存储桶中。我想使用putobjectRequest的MD5Digest属性来确认成功的传输,但是我很难从内存流中正确计算哈希。我猜这与编码有关。

BasicAWSCredentials credentials = new BasicAWSCredentials(<access key>, <secret key>);
AmazonS3Client s3Client = new AmazonS3Client(credentials, <region>);

AmazonS3Client s3Client 
string md5Hash = null;
//Serialize JSON package to be uploaded to AWS
JsonSerializer serializer = new JsonSerializer();
serializer.NullValueHandling = NullValueHandling.Include;
serializer.TypeNameHandling = TypeNameHandling.Auto;
using (MemoryStream memoryStream = new MemoryStream())
{
    //Write JSON to memory stream through compression layer
    using (GZipStream zipStream = new GZipStream(memoryStream, CompressionLevel.Fastest, true))
    using (StreamWriter sw = new StreamWriter(zipStream))
    using (JsonWriter writer = new JsonTextWriter(sw))
    {
        serializer.Serialize(writer, <object to serialize>);
        writer.Flush();
    }
    //Store gzipped JSON package hash to confirm successful upload later
    using (MD5 md5 = MD5.Create())
    {
        byte[] hash = md5.ComputeHash(memoryStream);
        md5Hash = string.Concat(hash.Select(x => x.ToString("x2")));
    }
    //Reset memoryStream to start because ComputeHash has consumed it
    memoryStream.Position = 0;
    //Upload to AWS
    PutObjectRequest request = new PutObjectRequest();
    request.InputStream = memoryStream;
    request.Key = <file name>;
    request.BucketName = <bucket>;
    request.ContentType = "application/gzip";
    request.MD5Digest = md5Hash;
    PutObjectResponse response = s3Client.PutObject(request);
}

I'm trying to serialize an object as JSON (newtonsoft.json), and store it in an S3 bucket in gzip compressed form. I want to use the MD5Digest property of PutObjectRequest to confirm successful transfer, but I'm having trouble getting the hash to calculate properly from the MemoryStream. I'm guessing it has to do with encoding.

BasicAWSCredentials credentials = new BasicAWSCredentials(<access key>, <secret key>);
AmazonS3Client s3Client = new AmazonS3Client(credentials, <region>);

AmazonS3Client s3Client 
string md5Hash = null;
//Serialize JSON package to be uploaded to AWS
JsonSerializer serializer = new JsonSerializer();
serializer.NullValueHandling = NullValueHandling.Include;
serializer.TypeNameHandling = TypeNameHandling.Auto;
using (MemoryStream memoryStream = new MemoryStream())
{
    //Write JSON to memory stream through compression layer
    using (GZipStream zipStream = new GZipStream(memoryStream, CompressionLevel.Fastest, true))
    using (StreamWriter sw = new StreamWriter(zipStream))
    using (JsonWriter writer = new JsonTextWriter(sw))
    {
        serializer.Serialize(writer, <object to serialize>);
        writer.Flush();
    }
    //Store gzipped JSON package hash to confirm successful upload later
    using (MD5 md5 = MD5.Create())
    {
        byte[] hash = md5.ComputeHash(memoryStream);
        md5Hash = string.Concat(hash.Select(x => x.ToString("x2")));
    }
    //Reset memoryStream to start because ComputeHash has consumed it
    memoryStream.Position = 0;
    //Upload to AWS
    PutObjectRequest request = new PutObjectRequest();
    request.InputStream = memoryStream;
    request.Key = <file name>;
    request.BucketName = <bucket>;
    request.ContentType = "application/gzip";
    request.MD5Digest = md5Hash;
    PutObjectResponse response = s3Client.PutObject(request);
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

も星光 2025-02-17 07:30:59

我的示例中有2个问题。

Anon Coward正确地确定需要重置内存的位置。

另一个问题是putobjectRequest.md5digest期望base64表示(即使AWS在ETAG中显示MD5 Hash作为HEX)。

代码的哈希部分应该看起来像这样,两个问题已固定

memoryStream.Position = 0;
using (MD5 md5 = MD5.Create())
{
    byte[] hash = md5.ComputeHash(memoryStream);
    md5Hash = Convert.ToBase64String(hash);
}

There were 2 issues in my example.

Anon Coward correctly identified that the position of the MemoryStream needed to be reset.

The other problem is that PutObjectRequest.MD5Digest expects a base64 representation (even though AWS shows MD5 hash in Etag as hex).

The hash section of the code should look like this with both issues fixed

memoryStream.Position = 0;
using (MD5 md5 = MD5.Create())
{
    byte[] hash = md5.ComputeHash(memoryStream);
    md5Hash = Convert.ToBase64String(hash);
}
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文