对于我正在考虑为即将到来的项目做的一些缓存，我一直在考虑 Java 序列化。 也就是说，应该使用它吗？
现在，我在过去几年中出于各种原因编写了自定义序列化和反序列化（Externalized）。 如今，互操作性已变得更加成为一个问题，我可以预见需要与 .Net 应用程序交互，因此我考虑使用独立于平台的解决方案。
有人有过GPB高性能使用的经验吗？ 与 Java 的原生序列化相比，它的速度和效率如何？ 或者，还有其他值得考虑的方案吗？
For some caching I'm thinking of doing for an upcoming project, I've been thinking about Java serialization. Namely, should it be used?
Now I've previously written custom serialization and deserialization (Externalizable) for various reasons in years past. These days interoperability has become even more of an issue and I can foresee a need to interact with .Net applications so I've thought of using a platform-independant solution.
Has anyone had any experience with high-performance use of GPB? How does it compare in terms of speed and efficiency with Java's native serialization? Alternatively, are there any other schemes worth considering?
发布评论
评论（7）
我没有将 Protocol Buffers 与 Java 的本机序列化在速度方面进行比较，但对于互操作性来说，Java 的本机序列化是一个严重的禁忌。 在大多数情况下，它在空间方面也不会像协议缓冲区那样高效。 当然，它在存储内容和引用等方面更加灵活。Protocol Buffers 非常擅长它的用途，当它满足您的需求时它就很棒 - 但由于互操作性而存在明显的限制（和其他东西）。
我最近发布了 Java 和 .NET 中的 Protocol Buffers 基准测试框架。 Java 版本位于 主要 Google 项目（在 基准目录），.NET 版本位于 我的 C# 端口项目。 如果您想将 PB 速度与 Java 序列化速度进行比较，您可以编写类似的类并对它们进行基准测试。 如果您对互操作感兴趣，我真的不会再考虑本机 Java 序列化（或 .NET 本机二进制序列化）。
除了 Protocol Buffers 之外，还有其他可互操作序列化的选项 - Thrift、JSON 和 YAML 浮现在脑海中，毫无疑问还有其他的。
编辑：好的，由于互操作不是那么重要，值得尝试列出您想要从序列化框架中获得的不同品质。 你应该考虑的一件事是版本控制 - 这是 PB 旨在处理好向后和向前的另一件事（因此新软件可以读取旧数据，反之亦然） - 当然，当你坚持建议的规则时:)
在尝试对 Java 性能与本机序列化保持谨慎之后，我真的不会惊讶地发现 PB 更快。 如果有机会，请使用服务器虚拟机 - 我最近的基准测试显示服务器虚拟机在序列化和反序列化示例数据时速度快了两倍。 我认为 PB 代码非常适合服务器 VM 的 JIT :)
正如示例性能数据一样，序列化和反序列化两条消息（一条 228 字节，一条 84750 字节），我使用服务器 VM 在笔记本电脑上得到了这些结果：
“速度” vs“大小”是生成的代码是否针对速度或代码大小进行了优化。 两种情况下的序列化数据是相同的。“大小”版本是为您定义了很多消息并且不想为代码占用大量内存的情况提供的。）
（这 看，对于较小的消息，它可以非常快 - 每毫秒序列化或反序列化超过 500 个小消息。 即使使用 87K 消息，每条消息花费的时间也不到一毫秒。
I haven't compared Protocol Buffers with Java's native serialization in terms of speed, but for interoperability Java's native serialization is a serious no-no. It's also not going to be as efficient in terms of space as Protocol Buffers in most cases. Of course, it's somewhat more flexible in terms of what it can store, and in terms of references etc. Protocol Buffers is very good at what it's intended for, and when it fits your need it's great - but there are obvious restrictions due to interoperability (and other things).
I've recently posted a Protocol Buffers benchmarking framework in Java and .NET. The Java version is in the main Google project (in the benchmarks directory), the .NET version is in my C# port project. If you want to compare PB speed with Java serialization speed you could write similar classes and benchmark them. If you're interested in interop though, I really wouldn't give native Java serialization (or .NET native binary serialization) a second thought.
There are other options for interoperable serialization besides Protocol Buffers though - Thrift, JSON and YAML spring to mind, and there are doubtless others.
EDIT: Okay, with interop not being so important, it's worth trying to list the different qualities you want out of a serialization framework. One thing you should think about is versioning - this is another thing that PB is designed to handle well, both backwards and forwards (so new software can read old data and vice versa) - when you stick to the suggested rules, of course :)
Having tried to be cautious about the Java performance vs native serialization, I really wouldn't be surprised to find that PB was faster anyway. If you have the chance, use the server vm - my recent benchmarks showed the server VM to be over twice as fast at serializing and deserializing the sample data. I think the PB code suits the server VM's JIT very nicely :)
Just as sample performance figures, serializing and deserializing two messages (one 228 bytes, one 84750 bytes) I got these results on my laptop using the server VM:
The "speed" vs "size" is whether the generated code is optimised for speed or code size. (The serialized data is the same in both cases. The "size" version is provided for the case where you've got a lot of messages defined and don't want to take a lot of memory for the code.)
As you can see, for the smaller message it can be very fast - over 500 small messages serialized or deserialized per millisecond. Even with the 87K message it's taking less than a millisecond per message.
还有一个数据点：该项目：
http://code.google.com/p/ thrift-protobuf-compare/
给出了小对象的预期性能的一些想法，包括 PB 上的 Java 序列化。
根据您的平台，结果会有很大差异，但有一些总体趋势。
One more data point: this project:
http://code.google.com/p/thrift-protobuf-compare/
gives some idea of expected performance for small objects, including Java serialization on PB.
Results vary a lot depending on your platform, but there are some general trends.
您还可以看看 FST，它是内置 JDK 序列化的直接替代品应该更快并且输出更小。
对我近年来所做的频繁基准测试的原始估计：
100% = 基于二进制/结构的方法（例如 SBE、fst-structs）
~10%-35% protobuf & 衍生出
约 10%-30% 的快速序列化器，例如 FST 和 KRYO
~2%-15% JDK 序列化
~1%-15%快速 JSon（例如 Jackson）
0.001-1% 完整图 JSon/XML（例如 JSON.io）
这些数字旨在给出非常粗略的顺序- 规模印象。
请注意，性能很大程度上取决于被序列化/基准测试的数据结构。 因此，单个简单的类基准大多是无用的（但很流行：例如忽略 unicode，没有集合，..）。
另请参见
http ://java-is-the-new-c.blogspot.de/2014/12/a-persistent-keyvalue-server-in-40.html
http://java-is-the-new-c.blogspot.de /2013/10/still-using-externalizable-to-get.html
You might also have a look at FST, a drop-in replacement for built-in JDK serialization that should be faster and have smaller output.
raw estimations on the frequent benchmarking i have done in recent years:
100% = binary/struct based approaches (e.g. SBE, fst-structs)
~10%-35% protobuf & derivates
~10%-30% fast serializers such as FST and KRYO
~2%-15% JDK serialization
~1%-15% fast JSon (e.g. Jackson)
0.001-1% full graph JSon/XML (e.g. JSON.io)
These numbers are meant to give a very rough order-of-magnitude impression.
Note that performance depends A LOT on the data structures being serialized/benchmarked. So single simple class benchmarks are mostly useless (but popular: e.g. ignoring unicode, no collections, ..).
see also
http://java-is-the-new-c.blogspot.de/2014/12/a-persistent-keyvalue-server-in-40.html
http://java-is-the-new-c.blogspot.de/2013/10/still-using-externalizable-to-get.html
你所说的高性能是什么意思？ 如果你想要毫秒级的序列化，我建议你使用最简单的序列化方法。 如果您想要亚毫秒，您可能需要二进制格式。 如果您想要远低于 10 微秒，您可能需要自定义序列化。
我还没有看到很多序列化/反序列化的基准，但很少有支持少于 200 微秒的序列化/反序列化。
平台独立格式是有代价的（您的努力和延迟），您可能必须决定是否需要性能或平台独立性。 但是，您没有理由不能将两者作为配置选项，并根据需要在两者之间进行切换。
What do you means by high performance? If you want milli-second serialization, I suggest you use the serialization approach which is simplest. If you want sub milli-second you are likely to need a binary format. If you want much below 10 micro-seconds you are likely to need a custom serialization.
I haven't seen many benchmarks for serialization/deserialization but few support less that 200 micro-seconds for serialization/deserialization.
Platform independent formats come at a cost (in effort on your part and latency) you may have to decide whether you want performance or platform independence. However, there is no reason you cannot have both as a configuration option which you switch between as required.
如果您对 PB 和 PB 之间感到困惑； 原生java序列化在速度和效率上，就去PB吧。
一些开发人员建议 Thrift，但我会使用 Google PB，因为“我相信 google”:-)..无论如何，它值得一看：
http://stuartsierra.com/2008/07/10/thrift-vs -协议缓冲区
If you are confusing between PB & native java serialization on speed and efficiency, just go for PB.
Some developers suggest Thrift, but I would use Google PB because "I believe in google" :-).. Anyway, it's worth for a look:
http://stuartsierra.com/2008/07/10/thrift-vs-protocol-buffers
这是今天的特别建议:-)（你刚刚在我的脑海中调整了一些我现在想尝试的东西）...
如果你可以通过这个来获取整个缓存解决方案，它可能会起作用：暗星计划。 它被设计为非常高性能的游戏服务器，特别是读取速度很快（对于缓存来说非常好）。 它有 Java 和 C API，所以我相信（我已经很久没有看到它了，当时我并没有想到这一点）你可以用 Java 保存对象并用 C 读回它们，反之亦然。
如果不出意外的话，今天它会给你一些值得阅读的东西:-)
Here is the off the wall suggestion of the day :-) (you just tweaked something in my head that I now want to try)...
If you can go for the whole caching solution via this it might work: Project Darkstar. It is designed as very high performance game server, specifically so that reads are fast (so good for a cache). It has Java and C APIs so I believe (thought it has been a long time since I looked at it, and I wasn't thinking of this then) that you could save objects with Java and read them back in C and vice versa.
If nothing else it'll give you something to read up on today :-)
对于有线友好的序列化，请考虑使用外部化接口。 如果巧妙地使用，您将拥有深入的知识来决定如何最佳地编组和解组特定字段。 也就是说，您需要正确管理每个对象的版本控制 - 易于取消编组，但是当您的代码支持 V1 时重新编组 V2 对象会破坏、丢失信息，或者以应用程序的方式损坏数据无法正确处理。 如果您正在寻找最佳路径，请注意没有任何库可以在不做出任何妥协的情况下解决您的问题。 一般来说，如果您选择了活跃的开源项目，库将适合大多数用例，并且会带来额外的好处，即它们会随着时间的推移而适应和增强，而无需您的输入。 它们可能会增加性能问题，引入错误，甚至修复尚未影响您的错误！
For wire-friendly serialisation, consider using the Externalizable interface. Used cleverly, you'll have intimate knowlege to decide how to optimally marshall and unmarshall specific fields. That said, you'll need to manage the versioning of each object correctly - easy to un-marshall, but re-marshalling a V2 object when your code supports V1 will either break, lose information, or worse corrupt data in a way your apps aren't able to correctly process. If you're looking for an optimal path, beware no library will solve your problem without some compromises. Generally libraries will fit most use-cases and will come with the added benefit that they'll adapt and enhance over time without your input, if you've opted for an active open source project. And they might add performance problems, introduce bugs, and even fix bugs that haven't affected you yet!