重新分配失败的可能性有多大?

发布于 2024-10-16 10:13:12 字数 56 浏览 12 评论 0原文

malloc类似,当它用完可用内存时会失败吗?或者可能还有其他原因吗?

Does it fail when it runs out of free memory similar to malloc or could there be other reasons?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

睫毛上残留的泪 2024-10-23 10:13:12

任何分配函数(mallocrealloccalloc 以及 POSIX 上的 posix_memalign)都可能因任何原因而失败原因如下,可能还有其他原因:

  • 您已经用完了整个虚拟地址空间,或者至少是其中的可用部分。在 32 位机器上,只有 4GB 的地址,并且可能保留 1GB 左右的地址供操作系统内核使用。即使您的机器有 16GB 物理内存,单个进程也不能使用超过其地址的内存。
  • 您尚未用完虚拟地址空间,但您已将其严重碎片化,以至于没有所请求大小的连续地址范围可用。如果您成功分配 6 512MB 块,释放所有其他块,然后尝试分配 1GB 块,则可能会发生这种情况(在 32 位计算机上)。当然,还有很多内存较小的其他示例。
  • 您的计算机已耗尽物理内存,这可能是由于您自己的程序已使用了所有物理内存,也可能是由于该计算机上运行的其他程序已使用了所有物理内存。某些系统(默认配置下的 Linux)会过度使用,这意味着在这种情况下 malloc 不会失败,但操作系统稍后会在发现时杀死一个或多个程序确实没有足够的物理内存可供使用。但在健壮的系统(包括禁用过量使用的 Linux)上,如果没有剩余物理内存,malloc 将失败。

请注意,严格来说,分配函数可以在任何时间因任何原因失败。最大限度地减少失败是一个实施质量问题。即使减小对象的大小,realloc 也可能会失败;这可能发生在严格按大小隔离分配的实现上。当然,在这种情况下,您可以简单地继续使用旧的(较大的)对象。

Any of the allocation functions (malloc, realloc, calloc, and on POSIX, posix_memalign) could fail for any of the following reasons, and possibly others:

  • You've used up your entire virtual address space, or at least the usable portion of it. On a 32-bit machine, there are only 4GB worth of addresses, and possibly 1GB or so is reserved for use by the OS kernel. Even if your machine has 16GB of physical memory, a single process cannot use more than it has addresses for.
  • You haven't used up your virtual address space, but you've fragmented it so badly that no contiguous range of addresses of the requested size are available. This could happen (on a 32-bit machine) if you successfully allocate 6 512MB blocks, free every other one, then try to allocate a 1GB block. Of course there are plenty of other examples with smaller memory sizes.
  • Your machine has run out of physical memory, either due to your own program having used it all, or other programs running on the machine having used it all. Some systems (Linux in the default configuration) will overcommit, meaning malloc won't fail in this situation, but instead the OS will later kill one or more programs when it figures out there's not really enough physical memory to go around. But on robust systems (including Linux with overcommit disabled), malloc will fail if there's no physical memory left.

Note that strictly speaking, the allocation functions are allowed to fail at any time for any reason. Minimizing failure is a quality-of-implementation issue. It's also possible that realloc could fail, even when reducing the size of an object; this could happen on implementations that strictly segregate allocations by size. Of course, in this case you could simply continue to use the old (larger) object.

夜深人未静 2024-10-23 10:13:12

您应该将 realloc 视为这样工作:

void *realloc(void *oldptr, size_t newsize)
{
   size_t oldsize = __extract_size_of_malloc_block(oldptr);
   void *newptr = malloc(newsize);

   if (!newptr)
     return 0;

   if (oldsize > newsize)
     oldsize = newsize;

   memcpy(newptr, oldptr, oldsize);
   free(oldptr);
   return newptr;
}

实现可能能够比这更有效地处理特定情况,但完全按照所示方式工作的实现是 100% 正确的。这意味着 realloc(ptr, newsize) 可能会在 malloc(newsize) 失败的任何时候失败;特别是,即使您缩小分配,它也可能会失败。

现在,在现代桌面系统上,有充分的理由不尝试从 malloc 故障中恢复,而是将 malloc 包装在函数中(通常称为 xmalloc) code>),如果 malloc 失败,则立即终止程序;当然,同样的论点也适用于realloc。情况是:

  1. 桌面系统通常在“过度使用”模式下运行,假设程序实际上不会使用所有地址空间,内核会乐意分配比 RAM+swap 所能支持的更多的地址空间。如果程序确实尝试使用所有这些,它将被强制终止。在此类系统上,只有在耗尽地址空间时,malloc 才会失败,这在 32 位系统上不太可能,在 64 位系统上几乎不可能。
  2. 即使您没有处于过量使用模式,桌面系统也有可能拥有如此多的 RAM 和交换空间,早在您导致 malloc 失败之前,用户就会厌倦他们的颠簸磁盘并强制终止您的程序。
  3. 没有实际的方法可以测试从分配失败中恢复;即使您有一个垫片库可以准确控制对 malloc 的哪些调用失败(此类垫片在最好的情况下是困难的,在最坏的情况下是不可能创建的,具体取决于操作系统),您也必须测试2N 种故障模式,其中 N 是程序中调用 malloc 的次数。

论点 1 和 2 不适用于嵌入式或移动系统(目前!),但论点 3 在那里仍然有效。

参数 3 仅适用于必须在每个调用站点检查和传播分配失败的程序。如果您很幸运能够按照预期使用 C++(即有例外),您可以依靠编译器为您创建错误恢复路径,因此测试负担会大大减轻。如今,在任何值得使用的高级语言中,您都有异常和垃圾收集器,这意味着即使您愿意,也不必担心分配失败。

You should think of realloc as working this way:

void *realloc(void *oldptr, size_t newsize)
{
   size_t oldsize = __extract_size_of_malloc_block(oldptr);
   void *newptr = malloc(newsize);

   if (!newptr)
     return 0;

   if (oldsize > newsize)
     oldsize = newsize;

   memcpy(newptr, oldptr, oldsize);
   free(oldptr);
   return newptr;
}

An implementation may be able to do specific cases more efficiently than that, but an implementation that works exactly as shown is 100% correct. That means realloc(ptr, newsize) can fail anytime malloc(newsize) would have failed; in particular it can fail even if you are shrinking the allocation.

Now, on modern desktop systems there is a strong case for not trying to recover from malloc failures, but instead wrapping malloc in a function (usually called xmalloc) that terminates the program immediately if malloc fails; naturally the same argument applies to realloc. The case is:

  1. Desktop systems often run in "overcommit" mode where the kernel will happily hand out more address space than can be backed by RAM+swap, assuming that the program isn't actually going to use all of it. If the program does try to use all of it, it will get forcibly terminated. On such systems, malloc will only fail if you exhaust the address space, which is unlikely on 32-bit systems and nigh-impossible on 64-bit systems.
  2. Even if you're not in overcommit mode, the odds are that a desktop system has so much RAM and swap available that, long before you cause malloc to fail, the user will get fed up with their thrashing disk and forcibly terminate your program.
  3. There is no practical way to test recovery from an allocation failure; even if you had a shim library that could control exactly which calls to malloc failed (such shims are at best difficult, at worst impossible, to create, depending on the OS) you would have to test order of 2N failure patterns, where N is the number of calls to malloc in your program.

Arguments 1 and 2 do not apply to embedded or mobile systems (yet!) but argument 3 is still valid there.

Argument 3 only applies to programs where allocation failures must be checked and propagated at every call site. If you are so lucky as to be using C++ as it is intended to be used (i.e. with exceptions) you can rely on the compiler to create the error recovery paths for you, so the testing burden is much reduced. And in any higher level language worth using nowadays you have both exceptions and a garbage collector, which means you couldn't worry about allocation failures even if you wanted to.

新雨望断虹 2024-10-23 10:13:12

我想说它主要是特定于实现的。有些实施很可能会失败。有些可能会在重新分配之前使程序的其他部分失败。始终保持防御状态并检查是否失败。

请记住释放您尝试重新分配的指针。

ptr=realloc(ptr,10);

总是可能存在内存泄漏。

总是这样做:

void *tmp=ptr;
if(ptr=realloc(ptr,10)==NULL){
  free(tmp);
  //handle error...
}

I'd say it mostly implementation specific. Some implementations may be very likely to fail. Some may have other parts of the program fail before realloc will. Always be defensive and check if it does fail.

And remember to free the old pointer that you tried to realloc.

ptr=realloc(ptr,10);

is ALWAYS a possible memory leak.

Always do it rather like this:

void *tmp=ptr;
if(ptr=realloc(ptr,10)==NULL){
  free(tmp);
  //handle error...
}
各自安好 2024-10-23 10:13:12

你有两个问题。

在大多数现代系统上,mallocrealloc 失败的可能性可以忽略不计。仅当虚拟内存不足时才会发生这种情况。您的系统将在访问内存时失败,而不是在保留内存时失败。

Wrt 失败reallocmalloc 几乎相同。 realloc 可能另外失败的唯一原因是您给它一个错误的参数,即尚未使用 mallocrealloc 分配的内存或者之前已经免费了。

编辑:鉴于R.的评论。是的,您可以配置您的系统,使其在分配时失败。但首先,据我所知,这不是默认设置。它需要以这种方式配置权限,作为应用程序程序员,这不是您可以指望的。其次,即使您有一个以这种方式配置的系统,只有当您的可用交换空间被耗尽时才会出错。通常,您的机器在此之前很久就会无法使用:它将在您的硬盘上进行机械计算(又称交换)。

You have two questions.

The chances that malloc or realloc fail are negligible on most modern system. This only occurs when you run out of virtual memory. Your system will fail on accessing the memory and not on reserving it.

W.r.t failure realloc and malloc are almost equal. The only reason that realloc may fail additionally is that you give it a bad argument, that is memory that had not been allocated with malloc or realloc or that had previously been freed.

Edit: In view of R.'s comment. Yes, you may configure your system such that it will fail when you allocate. But first of all, AFAIK, this is not the default. It needs privileges to be configured in that way and as an application programmer this is nothing that you can count on. Second, even if you'd have a system that is configured in that way, this will only error out when your available swap space has been eaten up. Usually your machine will be unusable long before that: it will be doing mechanical computations on your harddisk (AKA swapping).

裂开嘴轻声笑有多痛 2024-10-23 10:13:12

来自zwol答案< /a>:

现在,在现代桌面系统上,有充分的理由不尝试从 malloc 故障中恢复,而是将 malloc 包装在函数中(通常称为 xmalloc),如果 malloc 失败,则立即终止程序;
当然,同样的论点也适用于realloc

您可以看到 Git 2.29(2020 年第 4 季度)应用了该原则:xrealloc() 可以发送已释放的非 NULL 指针,这一点已得到修复。

请参阅 提交 6479ea4(2020 年 9 月 2 日),作者:杰夫·金 (peff)
(由 Junio C Hamano -- gitster -- 合并于 提交 56b891e,2020 年 9 月 3 日)

xrealloc:不要重用由零长度 realloc()

签字人:杰夫·金

此补丁修复了 xrealloc(ptr, 0) 可能在某些平台(至少包括 glibc)上双重释放并损坏堆的错误。

C99 标准对 malloc 的描述(第 7.20.3 节):

如果请求的空间大小为零,则行为为
实现定义的:要么返回空指针,要么返回
行为就好像大小是某个非零值,除了
返回的指针不得用于访问对象。  

因此我们可能会返回NULL,或者我们可能会得到一个实际的指针(但我们不允许查看其内容)。

为了简化代码,我们的 xmalloc() 通过将 NULL 返回转换为单字节分配来处理它。
这样呼叫者就可以获得一致的行为。这是早在 4e7a2eccc2 ("?alloc : 当要求零字节时不返回 NULL", 2005-12-29, Git v1.1.0 -- 合并)。

我们还给予了 xcalloc()xrealloc() 相同的处理。根据 C99 的说法,这很好;上面的文字位于适用于所有三个的段落中。

但是在这种情况下我们传递给 realloc() 的内存会发生什么情况呢?即,如果我们这样做:

ret = realloc(ptr, 0);  

并且“ptr”不是NULL,但我们得到了NULL:“ptr”仍然有效吗?
C99 没有具体涵盖这种情况,但说(第 7.20.3.4 节):

realloc函数释放ptr指向的旧对象并
返回一个指向新对象的指针,该对象的大小由 size 指定。  

因此“ptr”现在已被释放,我们只需查看“ret”。
由于“ret”是NULL,这意味着我们根本没有分配的对象。但这还不是故事的全部。它还说:

如果无法为新对象分配内存,则旧对象被分配
没有被释放并且它的值没有改变。
[...]
realloc 函数返回一个指向新对象的指针(可能是
与指向旧对象的指针具有相同的值),或空指针
如果无法分配新对象。  

因此,如果我们看到 NULL 返回一个非零大小,我们可以预期原始对象仍然有效。
但对于非零大小,它是不明确的。 NULL 返回可能意味着失败(在这种情况下对象有效),或者可能意味着我们成功分配了任何内容,并使用 NULL 来表示这一点。

realloc() 的 glibc 联机帮助页明确指出:

[...]如果 size 等于 0,并且 ptr 不为 NULL,则调用为
相当于 free(ptr)。  

同样,StackOverflow 对“malloc(0) 返回什么?”的回答:
声称 C89 提供了类似的指导(但我没有副本来验证)。

评论对“malloc(0) 的意义何在?”的回答 声称 Microsoft 的 CRT 具有相同的行为。

但是我们当前的“用 1 个字节重试”代码再次传递了原始指针。
因此,在glibc上,我们有效地free()指针,然后尝试再次realloc()它,这是未定义的行为。

这里最简单的解决方法是将“ret”(我们知道它是NULL)传递给后续的realloc().
但这意味着不释放原始指针的系统会泄漏它。目前尚不清楚是否存在任何这样的系统,并且对标准的解释似乎不太可能(我希望在这种情况下不释放的系统只是返回原始指针)。
但为了安全起见,很容易犯错误,根本不向 realloc() 传递零大小。

From zwol's answer:

Now, on modern desktop systems there is a strong case for not trying to recover from malloc failures, but instead wrapping malloc in a function (usually called xmalloc) that terminates the program immediately if malloc fails;
Naturally the same argument applies to realloc.

You can see that principle applied with Git 2.29 (Q4 2020): It was possible for xrealloc() to send a non-NULL pointer that has been freed, which has been fixed.

See commit 6479ea4 (02 Sep 2020) by Jeff King (peff).
(Merged by Junio C Hamano -- gitster -- in commit 56b891e, 03 Sep 2020)

xrealloc: do not reuse pointer freed by zero-length realloc()

Signed-off-by: Jeff King

This patch fixes a bug where xrealloc(ptr, 0) can double-free and corrupt the heap on some platforms (including at least glibc).

The C99 standard says of malloc (section 7.20.3):

If the size of the space requested is zero, the behavior is
implementation-defined: either a null pointer is returned, or the
behavior is as if the size were some nonzero value, except that the
returned pointer shall not be used to access an object.  

So we might get NULL back, or we might get an actual pointer (but we're not allowed to look at its contents).

To simplify our code, our xmalloc() handles a NULL return by converting it into a single-byte allocation.
That way callers get consistent behavior. This was done way back in 4e7a2eccc2 ("?alloc: do not return NULL when asked for zero bytes", 2005-12-29, Git v1.1.0 -- merge).

We also gave xcalloc() and xrealloc() the same treatment. And according to C99, that is fine; the text above is in a paragraph that applies to all three.

But what happens to the memory we passed to realloc() in such a case? I.e., if we do:

ret = realloc(ptr, 0);  

and "ptr" is non-NULL, but we get NULL back: is "ptr" still valid?
C99 doesn't cover this case specifically, but says (section 7.20.3.4):

The realloc function deallocates the old object pointed to by ptr and
returns a pointer to a new object that has the size specified by size.  

So "ptr" is now deallocated, and we must only look at "ret".
And since "ret" is NULL, that means we have no allocated object at all. But that's not quite the whole story. It also says:

If memory for the new object cannot be allocated, the old object is
not deallocated and its value is unchanged.
[...]
The realloc function returns a pointer to the new object (which may
have the same value as a pointer to the old object), or a null pointer
if the new object could not be allocated.  

So if we see a NULL return with a non-zero size, we can expect that the original object is still valid.
But with a non-zero size, it's ambiguous. The NULL return might mean a failure (in which case the object is valid), or it might mean that we successfully allocated nothing, and used NULL to represent that.

The glibc manpage for realloc() explicitly says:

[...]if size is equal to zero, and ptr is not NULL, then the call is
equivalent to free(ptr).  

Likewise, this StackOverflow answer to "What does malloc(0) return?":
claims that C89 gave similar guidance (but I don't have a copy to verify it).

A comment on this answer to "What's the point of malloc(0)?" claims that Microsoft's CRT behaves the same.

But our current "retry with 1 byte" code passes the original pointer again.
So on glibc, we effectively free() the pointer and then try to realloc() it again, which is undefined behavior.

The simplest fix here is to just pass "ret" (which we know to be NULL) to the follow-up realloc().
But that means that a system which doesn't free the original pointer would leak it. It's not clear if any such systems exist, and that interpretation of the standard seems unlikely (I'd expect a system that doesn't deallocate to simply return the original pointer in this case).
But it's easy enough to err on the safe side, and just never pass a zero size to realloc() at all.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文