bincode :: deserialize_flom with BufferedReader

发布于 2025-02-09 13:29:37 字数 1802 浏览 3 评论 0原文

我正在使用一个玩具示例,但实际的是一个非常大的二进制文件的比例,以相同的方式编写。

我试图从我用bincode编写的一个非常大的二进制文件中读取,

整个想法都是轻松使用bincode,以创建大文件并在稍后迭代

#[macro_use]
extern crate serde;
extern crate bincode;

// use std::fs::File;
use bincode::serialize_into;
use std::io::BufWriter;

#[derive(Serialize, Deserialize, PartialEq, Debug)]
pub struct MyStruct {
    counter: Vec<u32>,
    offset: usize,
}

impl MyStruct {
    // omitted for conciseness
    pub fn new(counter: Vec<u32>, offset: usize) -> Self {
        Self { counter, offset }
    }
}
fn main() {
    let mut vec = Vec::new();
    vec.push(100);
    vec.push(500);
    vec.push(100);
    vec.push(500);
    vec.push(100);
    vec.push(500);
    let m = MyStruct::new(vec, 0);

    // fill entries in the counter vector

    let mut f = BufWriter::new(File::create("foo2.bar").unwrap());
    serialize_into(&mut f, &m).unwrap();
    let mut vec = Vec::new();
    vec.push(100);
    vec.push(600);
    vec.push(100);
    vec.push(500);
    vec.push(100);
    vec.push(600);
    vec.push(500);
    vec.push(101241241);
    vec.push(600);
    let m2 = MyStruct::new(vec, 35);
    serialize_into(&mut f, &m2).unwrap();
    drop(f);
    // let mut buffer;
    // buffer = BufReader::open_raw_file("foo2.bar").unwrap();
    let input = File::open("foo2.bar").unwrap();
    // let buffered = BufReader::new(input);
    let mut vecs: Vec<MyStruct> = Vec::new();
    loop {
        match bincode::deserialize_from(&input) {
            Ok(file) => vecs.push(file),
            Err(error) => break,
        };
    }

}

,但是由于在原始文件上阅读,它是非常慢 如何使用BuffReader在Bincode :: Deserialize_from中加快代码?

let input = File::open("foo2.bar").unwrap();
let buffered = BufReader::new(input);

I am using a toy example, but the actual one is one scale of very large binary file that was written in the same manner.

Im trying to read from a very large binary file that i've written myself with bincode as well

The whole idea is to easily use Bincode in order to create big files and iterate on them later

#[macro_use]
extern crate serde;
extern crate bincode;

// use std::fs::File;
use bincode::serialize_into;
use std::io::BufWriter;

#[derive(Serialize, Deserialize, PartialEq, Debug)]
pub struct MyStruct {
    counter: Vec<u32>,
    offset: usize,
}

impl MyStruct {
    // omitted for conciseness
    pub fn new(counter: Vec<u32>, offset: usize) -> Self {
        Self { counter, offset }
    }
}
fn main() {
    let mut vec = Vec::new();
    vec.push(100);
    vec.push(500);
    vec.push(100);
    vec.push(500);
    vec.push(100);
    vec.push(500);
    let m = MyStruct::new(vec, 0);

    // fill entries in the counter vector

    let mut f = BufWriter::new(File::create("foo2.bar").unwrap());
    serialize_into(&mut f, &m).unwrap();
    let mut vec = Vec::new();
    vec.push(100);
    vec.push(600);
    vec.push(100);
    vec.push(500);
    vec.push(100);
    vec.push(600);
    vec.push(500);
    vec.push(101241241);
    vec.push(600);
    let m2 = MyStruct::new(vec, 35);
    serialize_into(&mut f, &m2).unwrap();
    drop(f);
    // let mut buffer;
    // buffer = BufReader::open_raw_file("foo2.bar").unwrap();
    let input = File::open("foo2.bar").unwrap();
    // let buffered = BufReader::new(input);
    let mut vecs: Vec<MyStruct> = Vec::new();
    loop {
        match bincode::deserialize_from(&input) {
            Ok(file) => vecs.push(file),
            Err(error) => break,
        };
    }

}

However, due to read on raw file, it is very slow
How can I use BuffReader to speed up my code inside bincode::deserialize_from?

let input = File::open("foo2.bar").unwrap();
let buffered = BufReader::new(input);

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

叹梦 2025-02-16 13:29:37

用法bufreader

let input = File::open("foo2.bar").unwrap();
let mut buffered = BufReader::new(input);
let mut vecs: Vec<MyStruct> = Vec::new();
loop {
    match bincode::deserialize_from(&mut buffered) {
        Ok(file) => vecs.push(file),
        Err(error) => break,
    };
}

Usage of BufReader:

let input = File::open("foo2.bar").unwrap();
let mut buffered = BufReader::new(input);
let mut vecs: Vec<MyStruct> = Vec::new();
loop {
    match bincode::deserialize_from(&mut buffered) {
        Ok(file) => vecs.push(file),
        Err(error) => break,
    };
}
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文