如何使用 zlib 压缩缓冲区？

Question

zlib 网站上有一个使用示例： http://www.zlib.net/zlib_how.html

但是在示例中他们正在压缩文件。我想压缩存储在内存缓冲区中的二进制数据。我也不想将压缩缓冲区保存到磁盘。

基本上这是我的缓冲区：

fIplImageHeader->imageData = (char*)imageIn->getFrame();

How can I compress it with zlib?

我希望有一些关于如何做到这一点的代码示例。

Answer 1

这是一个使用 zlib 打包缓冲区并将压缩内容保存在向量中的示例。

void compress_memory(void *in_data, size_t in_data_size, std::vector<uint8_t> &out_data)
{
 std::vector<uint8_t> buffer;

 const size_t BUFSIZE = 128 * 1024;
 uint8_t temp_buffer[BUFSIZE];

 z_stream strm;
 strm.zalloc = 0;
 strm.zfree = 0;
 strm.next_in = reinterpret_cast<uint8_t *>(in_data);
 strm.avail_in = in_data_size;
 strm.next_out = temp_buffer;
 strm.avail_out = BUFSIZE;

 deflateInit(&strm, Z_BEST_COMPRESSION);

 while (strm.avail_in != 0)
 {
  int res = deflate(&strm, Z_NO_FLUSH);
  assert(res == Z_OK);
  if (strm.avail_out == 0)
  {
   buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE);
   strm.next_out = temp_buffer;
   strm.avail_out = BUFSIZE;
  }
 }

 int deflate_res = Z_OK;
 while (deflate_res == Z_OK)
 {
  if (strm.avail_out == 0)
  {
   buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE);
   strm.next_out = temp_buffer;
   strm.avail_out = BUFSIZE;
  }
  deflate_res = deflate(&strm, Z_FINISH);
 }

 assert(deflate_res == Z_STREAM_END);
 buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE - strm.avail_out);
 deflateEnd(&strm);

 out_data.swap(buffer);
}

Answer 2

您可以通过将 fread() 和 fwrite() 调用替换为指向数据的直接指针来轻松修改该示例。对于 zlib 压缩（称为 deflate，因为您“取出数据中的所有空气”），您分配 z_stream 结构，调用 deflateInit() 然后：

1. 用要压缩的下一个数据块填充 next_in
2. set avail_in 为 next_in 中可用的字节数
3. set next_out 到压缩数据应写入的位置，通常应该是缓冲区内的指针，随着您的操作而前进，
4. 将 avail_out 设置为 next_out< 中可用的字节数/code>
5. 调用 deflate
6. 重复步骤 3-5，直到 avail_out 为非零（即输出缓冲区中的空间比 zlib 需要的空间更多 - 没有更多数据可写入）
7. 重复当您有数据要压缩时，执行步骤 1-6

最终您调用 deflateEnd() 就完成了。

你基本上是在向它提供大量的输入和输出，直到你没有输入并且它也没有输出。

Answer 3

使用 C++ 功能更方便的经典方法

这是一个完整的示例，演示了使用 C++ std::vector 对象进行压缩和解压缩：

#include <cstdio>
#include <iosfwd>
#include <iostream>
#include <vector>
#include <zconf.h>
#include <zlib.h>
#include <iomanip>
#include <cassert>

void add_buffer_to_vector(std::vector<char> &vector, const char *buffer, uLongf length) {
    for (int character_index = 0; character_index < length; character_index++) {
        char current_character = buffer[character_index];
        vector.push_back(current_character);
    }
}

int compress_vector(std::vector<char> source, std::vector<char> &destination) {
    unsigned long source_length = source.size();
    uLongf destination_length = compressBound(source_length);

    char *destination_data = (char *) malloc(destination_length);
    if (destination_data == nullptr) {
        return Z_MEM_ERROR;
    }

    Bytef *source_data = (Bytef *) source.data();
    int return_value = compress2((Bytef *) destination_data, &destination_length, source_data, source_length,
                                 Z_BEST_COMPRESSION);
    add_buffer_to_vector(destination, destination_data, destination_length);
    free(destination_data);
    return return_value;
}

int decompress_vector(std::vector<char> source, std::vector<char> &destination) {
    unsigned long source_length = source.size();
    uLongf destination_length = compressBound(source_length);

    char *destination_data = (char *) malloc(destination_length);
    if (destination_data == nullptr) {
        return Z_MEM_ERROR;
    }

    Bytef *source_data = (Bytef *) source.data();
    int return_value = uncompress((Bytef *) destination_data, &destination_length, source_data, source.size());
    add_buffer_to_vector(destination, destination_data, destination_length);
    free(destination_data);
    return return_value;
}

void add_string_to_vector(std::vector<char> &uncompressed_data,
                          const char *my_string) {
    int character_index = 0;
    while (true) {
        char current_character = my_string[character_index];
        uncompressed_data.push_back(current_character);

        if (current_character == '\00') {
            break;
        }

        character_index++;
    }
}

// https://stackoverflow.com/a/27173017/3764804
void print_bytes(std::ostream &stream, const unsigned char *data, size_t data_length, bool format = true) {
    stream << std::setfill('0');
    for (size_t data_index = 0; data_index < data_length; ++data_index) {
        stream << std::hex << std::setw(2) << (int) data[data_index];
        if (format) {
            stream << (((data_index + 1) % 16 == 0) ? "\n" : " ");
        }
    }
    stream << std::endl;
}

void test_compression() {
    std::vector<char> uncompressed(0);
    auto *my_string = (char *) "Hello, world!";
    add_string_to_vector(uncompressed, my_string);

    std::vector<char> compressed(0);
    int compression_result = compress_vector(uncompressed, compressed);
    assert(compression_result == F_OK);

    std::vector<char> decompressed(0);
    int decompression_result = decompress_vector(compressed, decompressed);
    assert(decompression_result == F_OK);

    printf("Uncompressed: %s\n", uncompressed.data());
    printf("Compressed: ");
    std::ostream &standard_output = std::cout;
    print_bytes(standard_output, (const unsigned char *) compressed.data(), compressed.size(), false);
    printf("Decompressed: %s\n", decompressed.data());
}

在您的 main.cpp 只需调用：

int main(int argc, char *argv[]) {
    test_compression();
    return EXIT_SUCCESS;
}

产生的输出：

Uncompressed: Hello, world!
Compressed: 78daf348cdc9c9d75128cf2fca495164000024e8048a
Decompressed: Hello, world!

Boost 方式

#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/zlib.hpp>

std::string compress(const std::string &data) {
    boost::iostreams::filtering_streambuf<boost::iostreams::output> output_stream;
    output_stream.push(boost::iostreams::zlib_compressor());
    std::stringstream string_stream;
    output_stream.push(string_stream);
    boost::iostreams::copy(boost::iostreams::basic_array_source<char>(data.c_str(),
                                                                      data.size()), output_stream);
    return string_stream.str();
}

std::string decompress(const std::string &cipher_text) {
    std::stringstream string_stream;
    string_stream << cipher_text;
    boost::iostreams::filtering_streambuf<boost::iostreams::input> input_stream;
    input_stream.push(boost::iostreams::zlib_decompressor());

    input_stream.push(string_stream);
    std::stringstream unpacked_text;
    boost::iostreams::copy(input_stream, unpacked_text);
    return unpacked_text.str();
}

TEST_CASE("zlib") {
    std::string plain_text = "Hello, world!";
    const auto cipher_text = compress(plain_text);
    const auto decompressed_plain_text = decompress(cipher_text);
    REQUIRE(plain_text == decompressed_plain_text);
}

Answer 4

这不是您有关 zlib API 问题的直接答案，但您可能对与 zlib 配对的 boost::iostreams 库感兴趣。

这允许使用 zlib 驱动的打包算法，使用基本的“流”操作符号，然后可以通过打开一些内存流并执行 << 来轻松压缩数据。对其进行数据操作。

对于boost::iostreams，这会自动为通过流的每个数据调用相应的打包过滤器。

Answer 5

zlib.h 具有您需要的所有功能：compress（或compress2）和uncompress。请参阅 zlib 的源代码以获得答案。

ZEXTERN int ZEXPORT compress OF((Bytef *dest,   uLongf *destLen, const Bytef *source, uLong sourceLen));
/*
         Compresses the source buffer into the destination buffer.  sourceLen is
     the byte length of the source buffer.  Upon entry, destLen is the total size
     of the destination buffer, which must be at least the value returned by
     compressBound(sourceLen).  Upon exit, destLen is the actual size of the
     compressed buffer.

         compress returns Z_OK if success, Z_MEM_ERROR if there was not
     enough memory, Z_BUF_ERROR if there was not enough room in the output
     buffer.
*/

ZEXTERN int ZEXPORT uncompress OF((Bytef *dest,   uLongf *destLen, const Bytef *source, uLong sourceLen));
/*
         Decompresses the source buffer into the destination buffer.  sourceLen is
     the byte length of the source buffer.  Upon entry, destLen is the total size
     of the destination buffer, which must be large enough to hold the entire
     uncompressed data.  (The size of the uncompressed data must have been saved
     previously by the compressor and transmitted to the decompressor by some
     mechanism outside the scope of this compression library.) Upon exit, destLen
     is the actual size of the uncompressed buffer.

         uncompress returns Z_OK if success, Z_MEM_ERROR if there was not
     enough memory, Z_BUF_ERROR if there was not enough room in the output
     buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete.  In
     the case where there is not enough room, uncompress() will fill the output
     buffer with the uncompressed data up to that point.
*/