在 C# (protobuf-net) 中序列化,在 C++ 中反序列化; (protobuf) : 类中超过 5 个字段

发布于 2024-11-17 08:20:20 字数 1789 浏览 3 评论 0原文

我在反序列化 C++ 中的对象时遇到问题,该对象已在 C# 中序列化,然后使用 ZMQ 通过网络发送。我相当确定 ZMQ 部分工作正常,因为 C++ 服务器应用程序 (Linux) 成功接收来自 C# (Windows) 的序列化消息并将它们发送回 Windows,在 Windows 中它可以成功反序列化消息,所以我不认为我在这方面,我遇到了任何类型的数据包被截断或丢弃的情况。

但是,当我在 Linux 服务器上收到消息时,C++ 反序列化方法无法正确反序列化,它会将一些二进制数据扔到第 6 个字段中(我可以在 MyObject.DebugString() 中看到这一点),但在任何其他字段。然而,奇怪的是,我的一个包含 5 个字段的类运行得很好。 C++ 正确反序列化它并且所有数据都正常工作。下面是我的代码的一些花絮。任何帮助将不胜感激。

C#:
    MemoryStream stream = new MemoryStream();
    ProtoBuf.Serializer.Serialize<TestType>(stream, (TestType)data);
    _publisher.Send(stream.ToArray());

C++:
    message_t data;
    int64_t recv_more;
    size_t recv_more_sz = sizeof(recv_more);
    TestType t;
    bool isProcessing = true;
    while(isProcessing)
    {
      pSubscriber->recv(&data, 0);
      t.ParseFromArray((void*)(data.data()),sizeof(t));
      cout<<"Debug: "<<t.DebugString()<<endl;  

      pSubscriber->getsockopt(ZMQ_RCVMORE, &recv_more, &recv_more_sz);
      isProcessing = recv_more;
    }

输出看起来像这样:

Debug: f: "4\000\000\000\000\000\"

我在复制和粘贴时遇到问题,但输出仍然像这样,大约有 3 或 4 行。

这是我的 TestType 类(原始文件):

package Base_Types;

enum Enumr {
  Dog = 0;
  Cat = 1;
  Fish = 2;
}

message TestType {
  required double a = 1;
  required Enumr b = 2;
  required string c = 3;
  required string d = 4;
  required double e = 5;
  required bytes f = 6;
  required string g = 7;
  required string h = 8;
  required string i = 9;
  required string j = 10;
}

字段“f”被列为字节,因为当它是一个字符串时,它会向我发出有关 UTF-8 编码的警告,但是,当该类仅使用 5 个字段(枚举是其中之一),它没有给我那个错误。这几乎就像不是反序列化,而是将整个类的二进制文件放入字段“f”(字段 6)中。

解决方案:最终出现了一个问题:内存在发送到线程套接字之前没有被复制。当发布者发回时,它正在打包数据并更改路由器接收的内容。 C++端需要有一个memcpy()来发送内部使用的数据。感谢您的所有帮助。

I'm having trouble deserializing an object in C++ that I had serialized in C# and then sent over the network with ZMQ. I'm fairly certain the ZMQ part is working correctly because the C++ server application (Linux) successfully receives the serialized messages from C# (Windows) and sends them back to Windows where it can successfully deserialize the message, so I don't think I'm experiencing any sort of truncated or dropped packets in that regard.

However, when I receive the message on the Linux server, the C++ deserialize method does not correctly deserialize, it throws some a bunch of binary data into the 6th field (I can see this in MyObject.DebugString()), but no data in any other fields. The strange part here, however, is that a class I had with 5 fields works perfectly fine. C++ deserializes it correctly and all of the data is working properly. Below are a few tidbits of my code. Any help would be greatly appreciated.

C#:
    MemoryStream stream = new MemoryStream();
    ProtoBuf.Serializer.Serialize<TestType>(stream, (TestType)data);
    _publisher.Send(stream.ToArray());

C++:
    message_t data;
    int64_t recv_more;
    size_t recv_more_sz = sizeof(recv_more);
    TestType t;
    bool isProcessing = true;
    while(isProcessing)
    {
      pSubscriber->recv(&data, 0);
      t.ParseFromArray((void*)(data.data()),sizeof(t));
      cout<<"Debug: "<<t.DebugString()<<endl;  

      pSubscriber->getsockopt(ZMQ_RCVMORE, &recv_more, &recv_more_sz);
      isProcessing = recv_more;
    }

The output looks like this:

Debug: f: "4\000\000\000\000\000\"

I'm having trouble copy and pasting, but the output continues like that for probably 3 or 4 lines worth of that.

This is my TestType class (proto file):

package Base_Types;

enum Enumr {
  Dog = 0;
  Cat = 1;
  Fish = 2;
}

message TestType {
  required double a = 1;
  required Enumr b = 2;
  required string c = 3;
  required string d = 4;
  required double e = 5;
  required bytes f = 6;
  required string g = 7;
  required string h = 8;
  required string i = 9;
  required string j = 10;
}

Field "f" is listed as bytes because when it was a string before it was giving me a warning about UTF-8 encoding, however, when this class worked with only 5 fields (the enum was one of them), it did not give me that error. It's almost like instead of deserializing, it's throwing the binary for the entire class into field "f" (field 6).

Solution: There ended up being an issue where the memory wasn't being copied before it sent to a thread socket. When the publisher sent back out, it was packaging the data and changing what the router received. There needs to be a memcpy() on the C++ side in order to send out the data to be used internally. Thanks for all of the help.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

春风十里 2024-11-24 08:20:20

我已经通过 v2 中的阅读器解析了它,它似乎很有意义:

1=5
2=0
3=
4=yo
5=6
6=2 bytes, 68-69
7=how
8=are
9=you
10=sir

请注意,我完全是从十六进制数据(不使用 .proto)中完成的,但它应该是接近你的原始数据。但最值得注意的是,它似乎完好无损。

所以:首先要做的事;检查您在 C++ 端获得的二进制文件是否与您发送的二进制文件完全相同;如果您在此过程中进行任何翻译(例如,二进制 => 字符串 - 应通过 base-64 完成),这一点就更加重要。

第二件事;如果这不起作用,则 C++ 实现中可能存在问题。这似乎不太可能,因为这是谷歌的宠物之一,但没有什么是不可能的。如果二进制文件完好无损,但它的行为仍然很奇怪,我可以尝试与 C++ 人员交谈,看看我们中是否有人已经疯了。

I've parsed it through the reader in v2, and it seems to make perfect sense:

1=5
2=0
3=
4=yo
5=6
6=2 bytes, 68-69
7=how
8=are
9=you
10=sir

Note that I've done that purely from the hex data (not using the .proto), but it should be close to your original data. But most notably, it seems intact.

So: first thing to do; check that the binary you get at the C++ side is exactly the same as the binary you sent; this is doubly important if you are doing any translations along the way (binary => string, for example - which should be done via base-64).

second thing; if that doesn't work, it is possible that there is a problem in the C++ implementation. It seems unlikely since that is one of google's pets, but nothing is impossible. If the binary comes across intact, but it still behaves oddly, I can try speaking to the C++ folks, to see if one of us has gone cuckoo.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文