GZZ Memorystream到S3桶散列错误
我正在尝试将一个对象序列化为JSON(Newtonsoft.json),并将其存储在GZIP压缩形式的S3存储桶中。我想使用putobjectRequest的MD5Digest属性来确认成功的传输,但是我很难从内存流中正确计算哈希。我猜这与编码有关。
BasicAWSCredentials credentials = new BasicAWSCredentials(<access key>, <secret key>);
AmazonS3Client s3Client = new AmazonS3Client(credentials, <region>);
AmazonS3Client s3Client
string md5Hash = null;
//Serialize JSON package to be uploaded to AWS
JsonSerializer serializer = new JsonSerializer();
serializer.NullValueHandling = NullValueHandling.Include;
serializer.TypeNameHandling = TypeNameHandling.Auto;
using (MemoryStream memoryStream = new MemoryStream())
{
//Write JSON to memory stream through compression layer
using (GZipStream zipStream = new GZipStream(memoryStream, CompressionLevel.Fastest, true))
using (StreamWriter sw = new StreamWriter(zipStream))
using (JsonWriter writer = new JsonTextWriter(sw))
{
serializer.Serialize(writer, <object to serialize>);
writer.Flush();
}
//Store gzipped JSON package hash to confirm successful upload later
using (MD5 md5 = MD5.Create())
{
byte[] hash = md5.ComputeHash(memoryStream);
md5Hash = string.Concat(hash.Select(x => x.ToString("x2")));
}
//Reset memoryStream to start because ComputeHash has consumed it
memoryStream.Position = 0;
//Upload to AWS
PutObjectRequest request = new PutObjectRequest();
request.InputStream = memoryStream;
request.Key = <file name>;
request.BucketName = <bucket>;
request.ContentType = "application/gzip";
request.MD5Digest = md5Hash;
PutObjectResponse response = s3Client.PutObject(request);
}
I'm trying to serialize an object as JSON (newtonsoft.json), and store it in an S3 bucket in gzip compressed form. I want to use the MD5Digest property of PutObjectRequest to confirm successful transfer, but I'm having trouble getting the hash to calculate properly from the MemoryStream. I'm guessing it has to do with encoding.
BasicAWSCredentials credentials = new BasicAWSCredentials(<access key>, <secret key>);
AmazonS3Client s3Client = new AmazonS3Client(credentials, <region>);
AmazonS3Client s3Client
string md5Hash = null;
//Serialize JSON package to be uploaded to AWS
JsonSerializer serializer = new JsonSerializer();
serializer.NullValueHandling = NullValueHandling.Include;
serializer.TypeNameHandling = TypeNameHandling.Auto;
using (MemoryStream memoryStream = new MemoryStream())
{
//Write JSON to memory stream through compression layer
using (GZipStream zipStream = new GZipStream(memoryStream, CompressionLevel.Fastest, true))
using (StreamWriter sw = new StreamWriter(zipStream))
using (JsonWriter writer = new JsonTextWriter(sw))
{
serializer.Serialize(writer, <object to serialize>);
writer.Flush();
}
//Store gzipped JSON package hash to confirm successful upload later
using (MD5 md5 = MD5.Create())
{
byte[] hash = md5.ComputeHash(memoryStream);
md5Hash = string.Concat(hash.Select(x => x.ToString("x2")));
}
//Reset memoryStream to start because ComputeHash has consumed it
memoryStream.Position = 0;
//Upload to AWS
PutObjectRequest request = new PutObjectRequest();
request.InputStream = memoryStream;
request.Key = <file name>;
request.BucketName = <bucket>;
request.ContentType = "application/gzip";
request.MD5Digest = md5Hash;
PutObjectResponse response = s3Client.PutObject(request);
}
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我的示例中有2个问题。
Anon Coward正确地确定需要重置内存的位置。
另一个问题是putobjectRequest.md5digest期望base64表示(即使AWS在ETAG中显示MD5 Hash作为HEX)。
代码的哈希部分应该看起来像这样,两个问题已固定
There were 2 issues in my example.
Anon Coward correctly identified that the position of the MemoryStream needed to be reset.
The other problem is that PutObjectRequest.MD5Digest expects a base64 representation (even though AWS shows MD5 hash in Etag as hex).
The hash section of the code should look like this with both issues fixed