Google Protocol Buffers - 固定大小的缓冲区?

发布于 2024-09-01 13:20:28 字数 146 浏览 8 评论 0原文

使用 Google Protocol Buffers,我可以为我编码的所有消息设置最大大小吗?

如果我知道我编码的内容永远不会大于 X 字节,那么 Google Protobuffs 总是会生成大小为 Y 的缓冲区,如果我给它较小的数据量,请将其填充到大小 Y?

Using Google Protocol Buffers, can I set a maximum size for all messages I encode?

if I know that what I encode is never larger than X bytes, then Google Protobuffs would always produce a buffer of size Y, and if I give it a smaller amount of data, pad it to size Y?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

渡你暖光 2024-09-08 13:20:29

协议缓冲区的有线格式不会让这变得微不足道;我不知道有什么办法可以做到这一点,但一种选择是将其序列化到带有您自己的长度标头的缓冲区中,并根据需要填充额外的数据。

您需要添加一个长度前缀,因为默认情况下不会添加它,否则它将在缓冲区末尾读取垃圾。即使尾随 0 也是不合法的(它将寻找字段编号)。

我无法评论 C++ 或 Jon 的 C# 版本,但对于我的 C# 版本 (protobuf-net),您应该能够执行类似的操作(未经测试):

using(var  ms = new MemoryStream(fixedLength)) {
     ms.SetLength(fixedLength);
     Serializer.SerializeWithLengthPrefix(ms, obj);
     if(ms.Length > fixedLength) { /* boom */ }
     byte[] arr = ms.ToArray(); // use this
}

如果还使用,这应该可以反序列化DeserializeWithLengthPrefix


回复问题(评论); SerializeWithLengthPrefix 是一个 protobuf-net 特定的方法; C++ 版本中可能有一些东西,但它非常简单。从头开始实现这个的最简单方法是:

  • 假设我们将留下一个固定长度(4 字节)标头来指示我们跳过
  • 4 字节(或写入 00-00-00- 00)
  • 现在 序列化到缓冲区的其余部分
  • 找出您刚刚写入的字节数
  • 将该值反向写回到缓冲区的开头

,显然:

  • 读取 4 个字节并解释为 int
  • 反序列化那么多as data

在 protobuf-net 中稍微复杂一点,因为它提供了更多选项(如何对 int 进行编码,以及是否将其包装起来,以便整个thing 仍然可以被视为 100% 值 protobuf 流 - 特别是我怀疑我刚刚描述了如果我要求 SerializeWithLengthPrefix 使用固定的行为 -宽度编码和“字段 0”)。

The wire format for protocol buffers wouldn't make this trivial; I'm not aware of something to do this, but one option would be to serialize it into a buffer with your own length header and pad with extra data as needed.

You need to add a length prefix because this is not added by default, and otherwise it would be reading garbage at the end of your buffer. Even trailing 0s would not be legal (it would be looking for a field number).

I can't comment on the C++ or Jon's C# version, but for my C# version (protobuf-net), you should be able to do something like (untested):

using(var  ms = new MemoryStream(fixedLength)) {
     ms.SetLength(fixedLength);
     Serializer.SerializeWithLengthPrefix(ms, obj);
     if(ms.Length > fixedLength) { /* boom */ }
     byte[] arr = ms.ToArray(); // use this
}

This should deserialize fine if also using DeserializeWithLengthPrefix.


Re the questions (comments); SerializeWithLengthPrefix is a protobuf-net-specific method; there may be something in the C++ version, but it is pretty simple. The easiest way to implement this from scratch is:

  • assume we will leave a fixed-length (4 byte) header to indicate how much actual data we have
  • skip 4 bytes (or write 00-00-00-00)
  • now serialize to the rest of the buffer
  • find how many bytes you just wrote
  • write that value back at the start of the buffer

in reverse, obviously:

  • read 4 bytes and interpret as an int
  • deserialize that much as data

It is a little bit more complex in protobuf-net, as it offers a few more options (how the int should be encoded, and whether or not to wrap this so that the entire thing can still be treated as a 100% value protobuf stream - in particular I suspect I've just described the behaviour if I asked SerializeWithLengthPrefix to use fixed-width encoding and "field 0").

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文