使用 Kryo 将多个对象序列化到单个文件中

发布于 2024-10-20 10:06:38 字数 120 浏览 1 评论 0 原文

据我所知,Kryo 序列化/反序列化是针对每个对象进行的。是否可以将多个对象序列化到一个文件中?另一个类似的问题中建议的解决方法之一是使用对象数组。考虑到需要序列化的数据量很大,我觉得它不会像应有的那样高效。这是正确的假设吗?

As far I know, Kryo serialization / deserialization happens per object. Is it possible to serialize multiple objects into a single file?. One of workaround suggested in another similar SO question was to use an array of objects. Considering a huge amount of data that needs to be serialized, I feel it would not be as efficient as it should be. Is it right assumption?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

好倦 2024-10-27 10:06:38

Kryo API 是否采用 OutputStream?如果是这样,只需向其提供相同的 OutputStream 即可序列化多个文件。读取时对InputStream进行同样的操作。良好的序列化格式将具有长度编码或终止符号,并且不会依赖 EOF 进行任何操作。

只要所有这些对象都已在内存中,数组方法也可以以最小的开销工作。您正在谈论的是为每个对象添加几个字节来创建一个数组来保存它们。如果它们不全部在内存中,则必须首先将它们全部加载到内存中,以便在它们周围创建一个数组。如果数据集足够大,这肯定会成为一个问题。

Does Kryo API take an OutputStream? If so, just feed it the same OutputStream to serialize multiple files. Do the same with InputStream when reading. A good serialization format will have length encodings or termination symbols and would not rely on EOF for anything.

The array approach would also work with minimal overhead as long as all of these objects are already in memory. You are talking about adding just a few bytes per object to create an array to hold them. If they aren't all in memory, you would have to load them all into memory first to create an array around them. That could definitely become a problem given large enough data set.

寄风 2024-10-27 10:06:38

由于 Kryo 支持流式传输,因此没有什么可以阻止您“在顶层”向 kryo 写入/读取多个对象。例如,以下程序将两个不相关的对象写入文件,然后再次反序列化它们

public class TestClass{


    public static void main(String[] args) throws FileNotFoundException{
        serialize();
        deSerialize();
    }

    public static void serialize() throws FileNotFoundException{
        Collection<String>collection=new ArrayList<>();
        int otherData=12;


        collection.add("This is a serialized collection of strings");

        Kryo kryo = new Kryo();
        Output output = new Output(new FileOutputStream("testfile"));
        kryo.writeClassAndObject(output, collection);
        kryo.writeClassAndObject(output, otherData); //we could add as many of these as we like
        output.close();
    }

    public static void deSerialize() throws FileNotFoundException{
        Collection<String>collection;
        int otherData;

        Kryo kryo = new Kryo();
        Input input = new Input(new FileInputStream("testfile"));
        collection=(Collection<String>)kryo.readClassAndObject(input);
        otherData=(Integer)kryo.readClassAndObject(input);

        input.close();

        for(String string: collection){
            System.out.println(string);
        }

        System.out.println("There are other things too! like; " + otherData);

    }


}

As Kryo supports streaming there is nothing to stop you writing/reading more than one object to kryo "at the top level". For example the following program writes two unrelated objects to a file and then deserializes them again

public class TestClass{


    public static void main(String[] args) throws FileNotFoundException{
        serialize();
        deSerialize();
    }

    public static void serialize() throws FileNotFoundException{
        Collection<String>collection=new ArrayList<>();
        int otherData=12;


        collection.add("This is a serialized collection of strings");

        Kryo kryo = new Kryo();
        Output output = new Output(new FileOutputStream("testfile"));
        kryo.writeClassAndObject(output, collection);
        kryo.writeClassAndObject(output, otherData); //we could add as many of these as we like
        output.close();
    }

    public static void deSerialize() throws FileNotFoundException{
        Collection<String>collection;
        int otherData;

        Kryo kryo = new Kryo();
        Input input = new Input(new FileInputStream("testfile"));
        collection=(Collection<String>)kryo.readClassAndObject(input);
        otherData=(Integer)kryo.readClassAndObject(input);

        input.close();

        for(String string: collection){
            System.out.println(string);
        }

        System.out.println("There are other things too! like; " + otherData);

    }


}
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文