Boost Intersocess支持共享过程之间包含指针的对象吗?

发布于 2025-01-30 08:57:45 字数 451 浏览 2 评论 0 原文

我希望在Linux上共享包含内部指针的对象。

类似于

BOOST.INTERPROCESS是否支持这一点?

除非每个过程都是以某种方式能够将内存映射到同一位置

Boost的答案是什么?

I wish to share objects containing internal pointers between unrelated processes on Linux.

Similar to this question?

Does boost.interprocess support this?

Any data structure containing raw pointers cannot be used for IPC unless each process is somehow able to map the memory to the same location.

What is boost's answer to this?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

╰沐子 2025-02-06 08:57:45

如果共享地址空间,则可以安全地将这种分配器用于完整的结构

您要混合的“共享地址空间”和“地址映射在同一地址”的完整结构。因为如果没有地址空间共享,也不会出现存储原始指针的问题。

重要的是使用 boost :: intercess :: offset_ptr&lt< t> 之类的精美指针。解释分配器使用 offset_ptr< t>

这就是为什么您需要使用支持花式指针的容器实现。


你很近。做您素描的规范方法:

>

#include <boost/container/string.hpp>
#include <boost/container/vector.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <string>
namespace bip = boost::interprocess;

namespace Shared {
#ifdef COLIRU
    using Segment = bip::managed_mapped_file;
#else
    using Segment = bip::managed_shared_memory;
#endif
    using Mgr     = Segment::segment_manager;

    template <typename T> using Alloc  = bip::allocator<T, Mgr>;
    template <typename T> using Vector = std::vector<T, Alloc<T>>;

    using String = boost::container::basic_string< //
        char, std::char_traits<char>, Alloc<char>>;

    struct Bar {
        using allocator_type = Alloc<char>;
        String first_name, last_name;

        template <typename Alloc> Bar(std::string_view first_name, std::string_view last_name, Alloc alloc) : 
            first_name(first_name, alloc), last_name(last_name, alloc) {}
    };

    struct Snafu {
        std::array<int, 5> no_problem;
    };

    struct Foo {
        Vector<Bar>   bars;
        Vector<Snafu> snafus;

        template <typename Alloc> //
        Foo(Alloc alloc) : bars(alloc)
                         , snafus(alloc) {}
    };
}

#include <iostream>
static inline std::ostream& operator<<(std::ostream& os, Shared::Foo const& foo) {
    os << "Foo\n================\nBars:\n";
    for (auto& [f, l] : foo.bars)
        os << " - " << f << ", " << l << "\n";

    os << "nafus:";
    for (auto& s : foo.snafus) {
        os << "\n - ";
        for (auto el : s.no_problem)
            os << " " << el;
    }
    return os << "\n";
}


int main() { Shared::Segment msm(bip::open_or_create, "my_shared_mem", 10ull << 10);

    Shared::Foo& foo = *msm.find_or_construct<Shared::Foo>("the_foo") //
                        (msm.get_segment_manager()); // constructor arguments

    foo.bars.emplace_back("John", "Doe", msm.get_segment_manager());
    foo.bars.emplace_back("Jane", "Deer", msm.get_segment_manager());
    foo.bars.emplace_back("Igor", "Stravinsky", msm.get_segment_manager());
    foo.bars.emplace_back("Rimsky", "Korsakov", msm.get_segment_manager());

    foo.snafus.push_back({1, 2, 3});
    foo.snafus.push_back({2, 3, 4});
    foo.snafus.push_back({3, 4, 5, 6, 7});

    std::cout << foo << "\n";
}

输出第一次运行:

Foo
================
Bars:
 - John, Doe
 - Jane, Deer
 - Igor, Stravinsky
 - Rimsky, Korsakov
nafus:
 -  1 2 3 0 0
 -  2 3 4 0 0
 -  3 4 5 6 7

输出第二运行:

Foo
================
Bars:
 - John, Doe
 - Jane, Deer
 - Igor, Stravinsky
 - Rimsky, Korsakov
 - John, Doe
 - Jane, Deer
 - Igor, Stravinsky
 - Rimsky, Korsakov
nafus:
 -  1 2 3 0 0
 -  2 3 4 0 0
 -  3 4 5 6 7
 -  1 2 3 0 0
 -  2 3 4 0 0
 -  3 4 5 6 7

一如既往的奖金

,我不能对 scoped_allocator_adaptor ,这是切片半导体以来最好的东西:

template <typename T>
using Alloc = boost::container::scoped_allocator_adaptor< //
    bip::allocator<T, Mgr>>;

现在,wherewherewherewhere awhere use_allocator 协议被用了,您会得到魔术分配的传播:

>

#include <boost/container/scoped_allocator.hpp>
#include <boost/container/string.hpp>
#include <boost/container/vector.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <string>
namespace bip = boost::interprocess;

namespace Shared {
#ifdef COLIRU
    using Segment = bip::managed_mapped_file;
#else
    using Segment = bip::managed_shared_memory;
#endif
    using Mgr     = Segment::segment_manager;

    template <typename T>
    using Alloc = boost::container::scoped_allocator_adaptor< //
        bip::allocator<T, Mgr>>;

    template <typename T> using Vector = std::vector<T, Alloc<T>>;

    using String = boost::container::basic_string< //
        char, std::char_traits<char>, Alloc<char>>;

    struct Bar {
        using allocator_type = Alloc<char>;
        String first_name, last_name;

        template <typename Alloc>
        Bar(std::allocator_arg_t, Alloc alloc) : first_name(alloc)
                                               , last_name(alloc) {}

        template <typename Alloc>
        Bar(Alloc&& alloc) : first_name(alloc)
                           , last_name(alloc) {}

        Bar(Bar const&) = default;
        template <typename Alloc>
        Bar(Bar const& rhs, Alloc alloc) : first_name(rhs.first_name, alloc), last_name(rhs.last_name, alloc) {}

        Bar& operator=(Bar const&) = default;

        template <typename Alloc> Bar(std::string_view first_name, std::string_view last_name, Alloc alloc) : 
            first_name(first_name, alloc), last_name(last_name, alloc) {}
    };

    struct Snafu {
        std::array<int, 5> no_problem;
    };

    struct Foo {
        Vector<Bar>   bars;
        Vector<Snafu> snafus;

        template <typename Alloc> //
        Foo(Alloc alloc) : bars(alloc)
                         , snafus(alloc) {}
    };
}

#include <iostream>
static inline std::ostream& operator<<(std::ostream& os, Shared::Foo const& foo) {
    os << "Foo\n================\nBars:\n";
    for (auto& [f, l] : foo.bars)
        os << " - " << f << ", " << l << "\n";

    os << "Snafus:";
    for (auto& s : foo.snafus) {
        os << "\n - ";
        for (auto el : s.no_problem)
            os << " " << el;
    }
    return os << "\n";
}


int main() { Shared::Segment msm(bip::open_or_create, "my_shared_mem", 10ull << 10);
    Shared::Foo& foo = *msm.find_or_construct<Shared::Foo>("the_foo") //
                        (msm.get_segment_manager()); // constructor arguments

    foo.bars.emplace_back("John", "Doe");
    foo.bars.emplace_back("Jane", "Deer");
    foo.bars.emplace_back("Igor", "Stravinsky");
    foo.bars.emplace_back("Rimsky", "Korsakov");

    foo.snafus.push_back({1, 2, 3});
    foo.snafus.push_back({2, 3, 4});
    foo.snafus.push_back({3, 4, 5, 6, 7});

    std::cout << foo << "\n";
}

输出相同

If the address space is shared it is safe to use such an allocator for the complete structure

You're mixing "shared address space" and "address mapped at the same address". Because if there was no sharing of address space, there would also not be an issue storing raw pointers.

What is important is to use fancy pointers like boost::interprocess::offset_ptr<T>. The interprocess allocator uses offset_ptr<T>.

This is why you need to use container implementations that support fancy pointers.


You're very close. The canonical way to do what you sketch would look like this:

Live On Coliru

#include <boost/container/string.hpp>
#include <boost/container/vector.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <string>
namespace bip = boost::interprocess;

namespace Shared {
#ifdef COLIRU
    using Segment = bip::managed_mapped_file;
#else
    using Segment = bip::managed_shared_memory;
#endif
    using Mgr     = Segment::segment_manager;

    template <typename T> using Alloc  = bip::allocator<T, Mgr>;
    template <typename T> using Vector = std::vector<T, Alloc<T>>;

    using String = boost::container::basic_string< //
        char, std::char_traits<char>, Alloc<char>>;

    struct Bar {
        using allocator_type = Alloc<char>;
        String first_name, last_name;

        template <typename Alloc> Bar(std::string_view first_name, std::string_view last_name, Alloc alloc) : 
            first_name(first_name, alloc), last_name(last_name, alloc) {}
    };

    struct Snafu {
        std::array<int, 5> no_problem;
    };

    struct Foo {
        Vector<Bar>   bars;
        Vector<Snafu> snafus;

        template <typename Alloc> //
        Foo(Alloc alloc) : bars(alloc)
                         , snafus(alloc) {}
    };
}

#include <iostream>
static inline std::ostream& operator<<(std::ostream& os, Shared::Foo const& foo) {
    os << "Foo\n================\nBars:\n";
    for (auto& [f, l] : foo.bars)
        os << " - " << f << ", " << l << "\n";

    os << "nafus:";
    for (auto& s : foo.snafus) {
        os << "\n - ";
        for (auto el : s.no_problem)
            os << " " << el;
    }
    return os << "\n";
}


int main() { Shared::Segment msm(bip::open_or_create, "my_shared_mem", 10ull << 10);

    Shared::Foo& foo = *msm.find_or_construct<Shared::Foo>("the_foo") //
                        (msm.get_segment_manager()); // constructor arguments

    foo.bars.emplace_back("John", "Doe", msm.get_segment_manager());
    foo.bars.emplace_back("Jane", "Deer", msm.get_segment_manager());
    foo.bars.emplace_back("Igor", "Stravinsky", msm.get_segment_manager());
    foo.bars.emplace_back("Rimsky", "Korsakov", msm.get_segment_manager());

    foo.snafus.push_back({1, 2, 3});
    foo.snafus.push_back({2, 3, 4});
    foo.snafus.push_back({3, 4, 5, 6, 7});

    std::cout << foo << "\n";
}

Output first run:

Foo
================
Bars:
 - John, Doe
 - Jane, Deer
 - Igor, Stravinsky
 - Rimsky, Korsakov
nafus:
 -  1 2 3 0 0
 -  2 3 4 0 0
 -  3 4 5 6 7

Output second run:

Foo
================
Bars:
 - John, Doe
 - Jane, Deer
 - Igor, Stravinsky
 - Rimsky, Korsakov
 - John, Doe
 - Jane, Deer
 - Igor, Stravinsky
 - Rimsky, Korsakov
nafus:
 -  1 2 3 0 0
 -  2 3 4 0 0
 -  3 4 5 6 7
 -  1 2 3 0 0
 -  2 3 4 0 0
 -  3 4 5 6 7

BONUS

As always, I cannot be remiss about scoped_allocator_adaptor, the best thing since sliced semiconductors:

template <typename T>
using Alloc = boost::container::scoped_allocator_adaptor< //
    bip::allocator<T, Mgr>>;

Now, anywhere uses_allocator protocol is used, you get magic allocator propagation:

Live On Coliru

#include <boost/container/scoped_allocator.hpp>
#include <boost/container/string.hpp>
#include <boost/container/vector.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <string>
namespace bip = boost::interprocess;

namespace Shared {
#ifdef COLIRU
    using Segment = bip::managed_mapped_file;
#else
    using Segment = bip::managed_shared_memory;
#endif
    using Mgr     = Segment::segment_manager;

    template <typename T>
    using Alloc = boost::container::scoped_allocator_adaptor< //
        bip::allocator<T, Mgr>>;

    template <typename T> using Vector = std::vector<T, Alloc<T>>;

    using String = boost::container::basic_string< //
        char, std::char_traits<char>, Alloc<char>>;

    struct Bar {
        using allocator_type = Alloc<char>;
        String first_name, last_name;

        template <typename Alloc>
        Bar(std::allocator_arg_t, Alloc alloc) : first_name(alloc)
                                               , last_name(alloc) {}

        template <typename Alloc>
        Bar(Alloc&& alloc) : first_name(alloc)
                           , last_name(alloc) {}

        Bar(Bar const&) = default;
        template <typename Alloc>
        Bar(Bar const& rhs, Alloc alloc) : first_name(rhs.first_name, alloc), last_name(rhs.last_name, alloc) {}

        Bar& operator=(Bar const&) = default;

        template <typename Alloc> Bar(std::string_view first_name, std::string_view last_name, Alloc alloc) : 
            first_name(first_name, alloc), last_name(last_name, alloc) {}
    };

    struct Snafu {
        std::array<int, 5> no_problem;
    };

    struct Foo {
        Vector<Bar>   bars;
        Vector<Snafu> snafus;

        template <typename Alloc> //
        Foo(Alloc alloc) : bars(alloc)
                         , snafus(alloc) {}
    };
}

#include <iostream>
static inline std::ostream& operator<<(std::ostream& os, Shared::Foo const& foo) {
    os << "Foo\n================\nBars:\n";
    for (auto& [f, l] : foo.bars)
        os << " - " << f << ", " << l << "\n";

    os << "Snafus:";
    for (auto& s : foo.snafus) {
        os << "\n - ";
        for (auto el : s.no_problem)
            os << " " << el;
    }
    return os << "\n";
}


int main() { Shared::Segment msm(bip::open_or_create, "my_shared_mem", 10ull << 10);
    Shared::Foo& foo = *msm.find_or_construct<Shared::Foo>("the_foo") //
                        (msm.get_segment_manager()); // constructor arguments

    foo.bars.emplace_back("John", "Doe");
    foo.bars.emplace_back("Jane", "Deer");
    foo.bars.emplace_back("Igor", "Stravinsky");
    foo.bars.emplace_back("Rimsky", "Korsakov");

    foo.snafus.push_back({1, 2, 3});
    foo.snafus.push_back({2, 3, 4});
    foo.snafus.push_back({3, 4, 5, 6, 7});

    std::cout << foo << "\n";
}

With identical output

也只是曾经 2025-02-06 08:57:45

根据文档的答案是 no

在映射区域构造对象时的限制

抵消指针而不是原始指针
禁止参考
虚拟性禁止
小心静态类成员

当两个过程创建一个同一映射对象的映射区域时,
两个过程可以传达写作和读取该记忆。一个
进程可以在该内存中构造C ++对象,以便第二个
过程可以使用它。 但是,多个共享的映射区域
过程,无法容纳任何C ++对象,因为并非每个类都准备好
要成为一个过程共享的对象,特别是如果映射的区域为
在每个过程中映射在不同的地址中。

在每个过程中都

Boost确实提供了替代解决方案。
有几块魔术可以使这项工作:

  • 偏移指针
  • 告诉模板类型(例如容器)使用偏移指针,而不是通过分配器通过层叠来级联的指示器
  • ,将此指令级联到模板类型上自动

提升,可自动提供使用智能偏移指针的容器(<

https://stackoverflow.com/a/a/7159729/1569204 有一个清晰的解释(不是特定的):

template<class T> class offset_ptr {
    size_t offset; public:
    T* operator->() { return reinterpret_cast<T*>(reinterpret_cast<char*>(this)+offset); } };

这很聪明。我不确定为什么这尚未将其纳入ISO标准(也许它已经是无知的)。

请参阅 https://en.cppreference.com/w/nemed_req/nemed_req/allemed_req/allagreq/allalocator. #Fancy_Pointers

当成员类型指针不是原始指针类型时,通常是
称为“花哨的指针”。这样的指针被介绍给
支持分段的内存体系结构,今天用于访问
在地址空间中分配的对象,与同质不同
原始指针访问的虚拟地址空间。一个例子
花式指针是映射独立的指针
BOOST :: interpocess :: offset_ptr,这使得分配成为可能
基于节点的数据结构,例如std ::设置在共享存储器中和
内存映射的文件在每个过程中的不同地址中映射。
花哨的指针可以独立于分配者使用
通过类模板std :: pointer_traits向他们提供(因为
C ++ 11)。函数std :: to_address可用于获得RAW
从花式指针的指针。 (由于C ++ 20)

使用精美的指针和自定义大小/不同类型
有条件地支持标准性LIBARY。实施可能
需要该会员类型指针,const_pointer,size_type和
差异_type是value_type*,const value_type*,std :: size_t和
std :: ptrdiff_t。

这里有一个很好的解释:

The boost version of the containers uses the pointer type由分配器指定。
这就是为什么 sallocator_traits 包括一个指针类型。
从理论上讲,STL实现(C ++ 11)应使用它,而不是假设原始指针。提升容器确保完成此操作。因此,增强分配器不仅返回Offset_ptr指针,而且还告知容器模板也应该使用它们。

另一个魔术是获取模板使用结构包含它的分配器的标准方法。
参见什么是”在C ++中使用分配器“和“范围分配器”构建

至少C ++ 11以来,这种魔术一直存在于C ++中。

The answer according to the documentation is no.

Limitations When Constructing Objects In Mapped Regions

Offset pointers instead of raw pointers
References forbidden
Virtuality forbidden
Be careful with static class members

When two processes create a mapped region of the same mappable object,
two processes can communicate writing and reading that memory. A
process could construct a C++ object in that memory so that the second
process can use it. However, a mapped region shared by multiple
processes, can't hold any C++ object, because not every class is ready
to be a process-shared object, specially, if the mapped region is
mapped in different address in each process.

If you could get it to work with boost.interprocess it probably wouldn't be portable.

Boost does provide an alternative solution.
There are several pieces of magic which make this work:

  • offset pointers
  • telling a template type (e.g. container) to use offset pointers instead of raw pointers via the allocator
  • cascading this instruction to templated types automatically

Boost provides containers which use smart offset pointers (boost::interprocess::offset_ptr) which can be used in place of regular STL containers.

https://stackoverflow.com/a/7159729/1569204 has a concise explanation of how this type of pointer can be implemented (not boost specific):

template<class T> class offset_ptr {
    size_t offset; public:
    T* operator->() { return reinterpret_cast<T*>(reinterpret_cast<char*>(this)+offset); } };

This is very clever. I'm not sure why this has not yet made it into the ISO standard (perhaps it has and I am just ignorant).

See https://en.cppreference.com/w/cpp/named_req/Allocator#Fancy_pointers

When the member type pointer is not a raw pointer type, it is commonly
referred to as a "fancy pointer". Such pointers were introduced to
support segmented memory architectures and are used today to access
objects allocated in address spaces that differ from the homogeneous
virtual address space that is accessed by raw pointers. An example of
a fancy pointer is the mapping address-independent pointer
boost::interprocess::offset_ptr, which makes it possible to allocate
node-based data structures such as std::set in shared memory and
memory mapped files mapped in different addresses in every process.
Fancy pointers can be used independently of the allocator that
provided them, through the class template std::pointer_traits (since
C++11). The function std::to_address can be used to obtain a raw
pointer from a fancy pointer. (since C++20)

Use of fancy pointers and customized size/different type in the
standard libary are conditionally supported. Implementations may
require that member type pointer, const_pointer, size_type, and
difference_type are value_type*, const value_type*, std::size_t, and
std::ptrdiff_t, respectively.

There is a good explanation here:

The boost version of the containers uses the pointer type specified by the allocator.
This is why allocator_traits includes a pointer type.
In theory an STL implementation (post C++11) should use it instead of assuming a raw pointer. Boosts containers ensure that this is done. So the boost allocator not only returns offset_ptr pointers but informs the container template that it should use them as well.

The other bit of magic is a standard way of getting a template to use the allocator of the structure containing it.
See What is "uses allocator" and "scoped allocator" construction in c++

This bit of magic has been in C++ since at least C++11.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文