posix_memalign/memalign 是做什么的

发布于 2024-11-18 09:45:20 字数 176 浏览 4 评论 0原文

我试图了解 memalign()posix_memalign() 的功能。阅读可用的文档没有帮助。

有人可以帮助我了解它的工作原理和用途吗?或者,也许提供一个使用示例?

我试图了解linux内存是如何工作的,我需要编写自己的简单内存池(低碎片堆)。

I'm trying to understand what functions memalign() and posix_memalign() do. Reading the available documentation didn't help.

Can someone help me understand how it works and what is it used for? Or, perhaps provide a usage example?

I'm trying to understand how linux memory works, I need to write my own simple memory pool (low-fragmentation heap).

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(6

后eg是否自 2024-11-25 09:45:20

malloc 为您提供了一块可以进行任何对齐的内存(唯一的要求是它必须针对实现支持的最大基元类型进行对齐),而 posix_memalign 则为您提供了您保证具有所请求的对齐方式的一块内存。

因此,例如 posix_memalign(&p, 32, 128) 的结果将是一个 128 字节的内存块,其起始地址保证是 32 的倍数。

这对于各种低位地址很有用。需要遵循特定对齐方式的内存的级别操作(例如使用 SSE 指令或 DMA)。

Whereas malloc gives you a chunk of memory that could have any alignment (the only requirement is that it must be aligned for the largest primitive type that the implementation supports), posix_memalign gives you a chunk of memory that is guaranteed to have the requested alignment.

So the result of e.g. posix_memalign(&p, 32, 128) will be a 128-byte chunk of memory whose start address is guaranteed to be a multiple of 32.

This is useful for various low-level operations (such as using SSE instructions, or DMA), that require memory that obeys a particular alignment.

锦欢 2024-11-25 09:45:20

除了奥利的回答之外,我还想向您指出一个更重要的问题。

在最新的 x86 架构上,缓存行(可以从内存获取到缓存的最小数据量)为 64 字节。假设您的结构大小为 56 字节,您有一个很大的数组。当你查找一个元素时,CPU将需要发出2个内存请求(即使它位于高速缓存线的中间,它也可能发出2个请求)。这对性能不利,因为您必须等待内存,并且使用更多缓存,这最终会导致更高的缓存未命中率。在这种情况下,仅使用 posix_memalign 是不够的,您应该将结构填充或压缩到 64 字节边界。

拥有 40 字节的结构只是运气不好:)

In addition to Oli's answer I would like to point you to an even more important issue.

On recent x86 architectures a cache-line, which is the smallest amount of data that can fetched from memory to cache, is 64 bytes. Suppose your structure size is 56 bytes, you have a large array of them. When you lookup one element, the CPU will need to issue 2 memory requests (it might issue 2 requests even if it is in the middle of the cacheline). That is bad for performance, as you have to wait for memory, and you use more cache, which ultimately gives a higher cache-miss ratio. In this case it is not enough to just use posix_memalign, but you should pad or compact your structure to be on 64byte boundaries.

Having 40 byte struct is just bad luck :)

千年*琉璃梦 2024-11-25 09:45:20

malloc 始终返回设置为任何基元类型所需的最大对齐方式的内存。这允许malloc的内存存储您可能需要的任何类型。我对 posix_memalign 的描述的理解是,它返回一个内存位置,该位置的地址将是您指定的对齐方式的倍数。

我不确定这在编写自定义内存池时有多大用处,但我尝试提供一个示例来说明如何实现这一点。与我的示例不同的是,使用 malloc_aligned 分配的任何内容都必须使用 free_aligned 释放;但是,通过posix_memalign,您可以使用free

#include <stdlib.h>
#include <stdio.h>

void *malloc_aligned(size_t alignment, size_t bytes)
{
    // we need to allocate enough storage for the requested bytes, some 
    // book-keeping (to store the location returned by malloc) and some extra
    // padding to allow us to find an aligned byte.  im not entirely sure if 
    // 2 * alignment is enough here, its just a guess.
    const size_t total_size = bytes + (2 * alignment) + sizeof(size_t);

    // use malloc to allocate the memory.
    char *data = malloc(sizeof(char) * total_size);

    if (data)
    {
        // store the original start of the malloc'd data.
        const void * const data_start = data;

        // dedicate enough space to the book-keeping.
        data += sizeof(size_t);

        // find a memory location with correct alignment.  the alignment minus 
        // the remainder of this mod operation is how many bytes forward we need 
        // to move to find an aligned byte.
        const size_t offset = alignment - (((size_t)data) % alignment);

        // set data to the aligned memory.
        data += offset;

        // write the book-keeping.
        size_t *book_keeping = (size_t*)(data - sizeof(size_t));
        *book_keeping = (size_t)data_start;
    }

    return data;
}

void free_aligned(void *raw_data)
{
    if (raw_data)
    {
        char *data = raw_data;

        // we have to assume this memory was allocated with malloc_aligned.  
        // this means the sizeof(size_t) bytes before data are the book-keeping 
        // which points to the location we need to pass to free.
        data -= sizeof(size_t);

        // set data to the location stored in book-keeping.
        data = (char*)(*((size_t*)data));

        // free the memory.
        free(data);
    }
}

int main()
{
    char *ptr = malloc_aligned(7, 100);

    printf("is 5 byte aligned = %s\n", (((size_t)ptr) % 5) ? "no" : "yes");
    printf("is 7 byte aligned = %s\n", (((size_t)ptr) % 7) ? "no" : "yes");

    free_aligned(ptr);

    return 0;
}

malloc always returns memory that is set to the maximum alignment required by any of the primitive types. This allows malloc'd memory to store any type you may need. My understanding of the description of posix_memalign, is that it returns a memory location who's address will be a multiple of whatever you specify as the alignment.

Im not sure how useful this would be when writing a custom memory pool, but I have had a go at providing an example of how this could be implemented. The difference is with my example, anything allocated with malloc_aligned has to be freed with free_aligned; however, with posix_memalign you can use free.

#include <stdlib.h>
#include <stdio.h>

void *malloc_aligned(size_t alignment, size_t bytes)
{
    // we need to allocate enough storage for the requested bytes, some 
    // book-keeping (to store the location returned by malloc) and some extra
    // padding to allow us to find an aligned byte.  im not entirely sure if 
    // 2 * alignment is enough here, its just a guess.
    const size_t total_size = bytes + (2 * alignment) + sizeof(size_t);

    // use malloc to allocate the memory.
    char *data = malloc(sizeof(char) * total_size);

    if (data)
    {
        // store the original start of the malloc'd data.
        const void * const data_start = data;

        // dedicate enough space to the book-keeping.
        data += sizeof(size_t);

        // find a memory location with correct alignment.  the alignment minus 
        // the remainder of this mod operation is how many bytes forward we need 
        // to move to find an aligned byte.
        const size_t offset = alignment - (((size_t)data) % alignment);

        // set data to the aligned memory.
        data += offset;

        // write the book-keeping.
        size_t *book_keeping = (size_t*)(data - sizeof(size_t));
        *book_keeping = (size_t)data_start;
    }

    return data;
}

void free_aligned(void *raw_data)
{
    if (raw_data)
    {
        char *data = raw_data;

        // we have to assume this memory was allocated with malloc_aligned.  
        // this means the sizeof(size_t) bytes before data are the book-keeping 
        // which points to the location we need to pass to free.
        data -= sizeof(size_t);

        // set data to the location stored in book-keeping.
        data = (char*)(*((size_t*)data));

        // free the memory.
        free(data);
    }
}

int main()
{
    char *ptr = malloc_aligned(7, 100);

    printf("is 5 byte aligned = %s\n", (((size_t)ptr) % 5) ? "no" : "yes");
    printf("is 7 byte aligned = %s\n", (((size_t)ptr) % 7) ? "no" : "yes");

    free_aligned(ptr);

    return 0;
}
猫烠⑼条掵仅有一顆心 2024-11-25 09:45:20

它如何工作取决于实现。该函数的目的是为您提供一个 n 字节对齐的内存块(块的起始地址是 n 的倍数)。

How does it work is implementation dependent. The purpose of the function is to give you an n-bytes aligned memory block (the start address of the block is a multiply of n).

甜尕妞 2024-11-25 09:45:20

由于 memalign 已过时(参考:手册页),因此这里仅描述 malloc() 和 posix_memalign() 之间的区别。 malloc() 是 8 字节对齐的(例如,对于 RHEL 32 位),但对于 posix_memalign(),对齐方式是用户可定义的。要了解它的用法,也许一个很好的例子是使用 mprotect() 设置内存属性。要使用 mprotect(),内存指针必须页对齐。因此,如果您以 pagesize 作为对齐方式调用 posix_memalign(),则返回的指针可以轻松提交给 mprotect() 以设置读写可执行属性。 (例如,将数据复制到内存指针后,可以将其设置为只读属性,以防止其被修改)。 “malloc()”返回的指针不能在这里使用。

As memalign is obsolete (ref: man page), only the difference between malloc() and posix_memalign() will be described here. malloc() is 8-byte aligned (eg, for RHEL 32-bit), but for posix_memalign(), alignment is user-definable. To know the use of this, perhaps one good example is setting memory attribute using mprotect(). To use mprotect(), the memory pointer must be PAGE aligned. And so if you call posix_memalign() with pagesize as the alignment, then the returned pointer can easily submit to mprotect() to set the read-write-executable attributes. (for example, after you copy the data into the memory pointer, you can set it to read-only attribute to protect it from being modified). "malloc()"'s returned pointer cannot be use here.

薄荷港 2024-11-25 09:45:20

当您在 GNU C 中使用 posix_memalign 时,您应该注意第二个参数不仅应该是 2 的幂,而且还应该是 sizeof (void*) 的倍数。请注意,此要求与 memalign 函数中的要求不同,后者仅需要 2 的幂。

int
__posix_memalign (void **memptr, size_t alignment, size_t size)
{
  void *mem;

  /* Test whether the SIZE argument is valid.  It must be a power of
     two multiple of sizeof (void *).  */
  if (alignment % sizeof (void *) != 0
      || !powerof2 (alignment / sizeof (void *))
      || alignment == 0)
    return EINVAL;


  void *address = RETURN_ADDRESS (0);
  mem = _mid_memalign (alignment, size, address);

  if (mem != NULL)
    {
      *memptr = mem;
      return 0;
    }

  return ENOMEM;
}
weak_alias (__posix_memalign, posix_memalign) 

当我们查看 posix_memalign 实现中的第一个 if 条件时,它检查对齐是否是 sizeof(void*) 的倍数。

When you use posix_memalign in GNU C, you should be careful that second parameter should not only be a power of two, but also be a multiple of sizeof (void*). Note that this requirement is different from the one in the memalign function, which requires only power of two.

int
__posix_memalign (void **memptr, size_t alignment, size_t size)
{
  void *mem;

  /* Test whether the SIZE argument is valid.  It must be a power of
     two multiple of sizeof (void *).  */
  if (alignment % sizeof (void *) != 0
      || !powerof2 (alignment / sizeof (void *))
      || alignment == 0)
    return EINVAL;


  void *address = RETURN_ADDRESS (0);
  mem = _mid_memalign (alignment, size, address);

  if (mem != NULL)
    {
      *memptr = mem;
      return 0;
    }

  return ENOMEM;
}
weak_alias (__posix_memalign, posix_memalign) 

When we look at the first if condition in the posix_memalign implementation, it checks whether alignment is multiple of sizeof(void*).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文