没有GRPC可以使用协议缓冲区吗?

发布于 2025-02-06 18:13:48 字数 489 浏览 1 评论 0 原文

大家好,我对GRPC和协议缓冲区的双手弄脏,并且遇到了一篇文章,该文章提到消息的二进制Protobuf文件比JSON小5倍,但在该文章中,提到这种压缩水平可以仅当通过GRPC传输时才能实现。这种特殊的评论“通过GRPC传输时可能进行压缩”,我似乎无法理解,因为我了解协议缓冲区是一种序列化格式,无论GRPC如何,都可以起作用,或者这种理解是否存在缺陷?这意味着什么?这是指向文章和屏幕截图的链接。

Hello everyone I am getting my hands dirty with gRPC and protocol buffers and have came across an article which has mentioned that the binary protobuf file for a message is 5 times smaller than Json counterpart but in that article it is mentioned that this level of compression can only be achieved if transmitting via gRPC. This particular comment "compression possible when transmitting via gRPC", I cant seem to understand cause i had an understanding that protocol buffer is a serialization format which can work irrespective of gRPC or is this understanding flawed? what does this means? here is the link to the article and the screen shot.
https://www.datascienceblog.net/post/programming/essential-protobuf-guide-python/
enter image description here

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

辞别 2025-02-13 18:13:49

我同意英国人的回答。

一些单独的点。谈论GPB压缩数据并不准确。它只是某种方法来编码数据(例如整数)。如果您发送一条具有10个字符串的GPB消息,则不会像您期望ZIP相同的方式向下压缩。

GPB并不是其中最有效的数据编码器。 ASN.1 UPER甚至更好,能够利用可以放入ASN.1架构中无法在GPB架构.proto文件中表达的知识。例如,在ASN.1中,您可以将整数的值限制为1000..1003。在ASN.1 UPER中,这是2位(有4个可能的值)。在GPB中,至少是编码数据的两个字节。

I agree with Brits' answer.

Some separate points. It's not quite accurate to talk about GPB compressing data; it merely goes some way to minimally encode data (e.g. integers). If you send a GPB message that has 10 strings all the same, it won't compress that down in the same way you might expect zip to.

GPB is not the most efficent encoder of data out there. ASN.1 uPER is even better, being able to exploit knowledge that can be put into an ASN.1 schema that cannot be expressed in a GPB schema .proto file. For example, in ASN.1 you can constrain the value of an integer to, say, 1000..1003. In ASN.1 uPER that's 2 bits (there are 4 possible values). In GPB, that's two bytes of encoded data at least.

兮颜 2025-02-13 18:13:48

您是正确的 - “ noreferrer”>“协议缓冲区”提供序列化格式大小的数据最多可以使用”,并且具有许多用途(例如,将数据序列化数据,网络传输,应用程序之间传递数据等)。

文章您参考

我们只有使用GRPC协议传输二进制Protobuf

时才能达到这种压缩水平

当您考虑以下段落时,该声明背后的推理变得更加清晰:

如果不是GRPC选项,则常见模式是使用base64编码编码二进制Protobuf数据。尽管这种不可撤销的编码可将有效载荷的大小增加33%,但它仍然比相应的休息有效载荷小得多。

因此,这似乎着重于通过 http 这是基于文本的协议(< a href =“ https://stackoverflow.com/a/58513200/11810946”> http/2 有些改变)。
由于ProfoBuf是一种二进制格式,因此通常在通过HTTP传输之前对其进行编码(从技术上讲,它可以使用 application/opplation/Octet-stream 的二进制格式传输。本文提到了base64编码并使用它增加了其大小。

如果您不使用HTTP(例如,将磁盘编写为Direct TCP/IP链接,Websockets,MQTT等),则不应用。

话虽如此,我相信这个示例可能会夸大了Protobuf的好处,因为很多http流量都被压缩了(本文报告了9%的测试差异)。

You are correct - Protocol buffers "provide a serialization format for packets of typed, structured data that are up to a few megabytes in size" and has many uses (e.g. serialising data to disk, network transmission, passing data between applications etc).

The article you reference is somewhat misleading when it says

we can only achieve this level of compression if we are transmitting binary Protobuf using the gRPC protocol

The reasoning behind the statement becomes a bit clearer when you consider the following paragraph:

If gRPC is not an option, a common pattern is to encode the binary Protobuf data using the base64-encoding. Although this encoding irrevocably increases the size of the payload by 33%, it is still much smaller than the corresponding REST payload.

So this appears to be focusing on transmitting data over HTTP which is a text based protocol (HTTP/2 changes this somewhat).
Because Profobuf is a binary format it is often encoded before being transmitted via HTTP (technically it could be transferred in a binary format using something like application/octet-stream). The article mentions base64-encoding and using this this increases it's size.

If you are not using HTTP (e.g. writing to disk, direct TCP/IP link, Websockets, MQTT etc) then this does not apply.

Having said that I believe that the example may be overstating the benefits of Protobuf a little because a lot of HTTP traffic is compressed (this article reported a 9% difference in their test).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文