ByteBuffer 不释放内存

发布于 2024-10-18 07:38:38 字数 1439 浏览 9 评论 0原文

在 Android 上,直接 ByteBuffer 似乎永远不会释放其内存,即使在调用 System.gc() 时也是如此。

示例:执行操作

Log.v("?", Long.toString(Debug.getNativeHeapAllocatedSize()));
ByteBuffer buffer = allocateDirect(LARGE_NUMBER);
buffer=null;
System.gc();
Log.v("?", Long.toString(Debug.getNativeHeapAllocatedSize()));

会在日志中给出两个数字,第二个数字至少比第一个数字大 LARGE_NUMBER。

我该如何摆脱这个泄漏?


添加:

按照 Gregory 在 C++ 端处理分配/释放的建议,然后我定义了

JNIEXPORT jobject JNICALL Java_com_foo_bar_allocNative(JNIEnv* env, jlong size)
    {
    void* buffer = malloc(size);
    jobject directBuffer = env->NewDirectByteBuffer(buffer, size);
    jobject globalRef = env->NewGlobalRef(directBuffer);
    return globalRef;
    }

JNIEXPORT void JNICALL Java_com_foo_bar_freeNative(JNIEnv* env, jobject globalRef)
    {
    void *buffer = env->GetDirectBufferAddress(globalRef);
    free(buffer);
    env->DeleteGlobalRef(globalRef);
    }

然后在 JAVA 端获取我的 ByteBuffer

ByteBuffer myBuf = allocNative(LARGE_NUMBER);

并释放它

freeNative(myBuf);

不幸的是,虽然它确实分配得很好,但它a) 仍然保留根据 Debug.getNativeHeapAllocatedSize() 分配的内存,b) 导致错误

W/dalvikvm(26733): JNI: DeleteGlobalRef(0x462b05a0) failed to find entry (valid=1)

我现在完全困惑了,我以为我至少理解了 C++ 方面的事情......为什么是free() 不返回内存?我对 DeleteGlobalRef() 做错了什么?

On Android, a direct ByteBuffer does not ever seem to release its memory, not even when calling System.gc().

Example: doing

Log.v("?", Long.toString(Debug.getNativeHeapAllocatedSize()));
ByteBuffer buffer = allocateDirect(LARGE_NUMBER);
buffer=null;
System.gc();
Log.v("?", Long.toString(Debug.getNativeHeapAllocatedSize()));

gives two numbers in the log, the second one being at least LARGE_NUMBER larger than the first.

How do I get rid of this leak?


Added:

Following the suggestion by Gregory to handle alloc/free on the C++ side, I then defined

JNIEXPORT jobject JNICALL Java_com_foo_bar_allocNative(JNIEnv* env, jlong size)
    {
    void* buffer = malloc(size);
    jobject directBuffer = env->NewDirectByteBuffer(buffer, size);
    jobject globalRef = env->NewGlobalRef(directBuffer);
    return globalRef;
    }

JNIEXPORT void JNICALL Java_com_foo_bar_freeNative(JNIEnv* env, jobject globalRef)
    {
    void *buffer = env->GetDirectBufferAddress(globalRef);
    free(buffer);
    env->DeleteGlobalRef(globalRef);
    }

I then get my ByteBuffer on the JAVA side with

ByteBuffer myBuf = allocNative(LARGE_NUMBER);

and free it with

freeNative(myBuf);

Unfortunately, while it does allocate fine, it a) still keeps the memory allocated according to Debug.getNativeHeapAllocatedSize() and b) leads to an error

W/dalvikvm(26733): JNI: DeleteGlobalRef(0x462b05a0) failed to find entry (valid=1)

I am now thoroughly confused, I thought I at least understood the C++ side of things... Why is free() not returning the memory? And what am I doing wrong with the DeleteGlobalRef()?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

Spring初心 2024-10-25 07:38:38

没有泄漏。

ByteBuffer.allocateDirect() 从本机堆/空闲存储(例如 malloc())分配内存,该内存又包装在 ByteBuffer 中实例。

当 ByteBuffer 实例被垃圾回收时,本机内存将被回收(否则您将泄漏本机内存)。

您调用 System.gc() 是希望立即回收本机内存。但是,调用 System.gc() 只是一个请求,这解释了为什么您的第二条日志语句没有告诉您内存已被释放:因为它还没有释放!

在您的情况下,Java 堆中显然有足够的可用内存,并且垃圾收集器决定不执行任何操作:因此,尚未收集无法访问的 ByteBuffer 实例,其终结器未运行并且本机内存没有被释放。

另外,请记住 JVM 中的这个 bug (不确定它如何应用但对于 Dalvik),直接缓冲区的大量分配会导致不可恢复的 OutOfMemoryError


您评论了从 JNI 进行控制的事情。这实际上是可能的,您可以实现以下内容:

  1. 发布一个native ByteBuffer allocateNative(long size)入口点:

    • 调用void* buffer = malloc(size)来分配本机内存
    • 通过调用 (*env)->NewDirectByteBuffer(env, buffer, size); 将新分配的数组包装到 ByteBuffer 实例中;
    • 使用 (*env)->NewGlobalRef(env, directBuffer);ByteBuffer 本地引用转换为全局引用李>
  2. 入口点

    发布一个 native void disposeNative(ByteBuffer buffer) 入口点:

    • *(env)->GetDirectBufferAddress(env, directBuffer); 返回的直接缓冲区地址调用 free()
    • 使用 (*env)->DeleteGlobalRef(env, directBuffer); 删除全局引用

一旦调用 disposeNative< /code> 在缓冲区上,您不应该再使用该引用,因此它可能很容易出错。重新考虑您是否真的需要对分配模式进行如此明确的控制。


忘记我所说的全局引用。实际上,全局引用是一种在本机代码中存储引用的方法(例如在全局变量中),以便进一步调用 JNI 方法可以使用该引用。例如,您可以:

  • 从 Java 调用本机方法 foo(),该方法从本地引用创建全局引用(通过从本机端创建对象获得)并将其存储在本机全局变量中(作为 jobject
  • 再次从 Java 回来,调用本机方法 bar() 获取jobjectfoo() 存储并进一步处理它
  • 最后,仍然来自 Java,最后一次调用本机 baz() 删除全局引用

抱歉造成混淆。

There is no leak.

ByteBuffer.allocateDirect() allocates memory from the native heap / free store (think malloc()) which is in turn wrapped in to a ByteBuffer instance.

When the ByteBuffer instance gets garbage collected, the native memory is reclaimed (otherwise you would leak native memory).

You're calling System.gc() in hope the native memory is reclaimed immediately. However, calling System.gc() is only a request which explains why your second log statement doesn't tell you memory has been released: it's because it hasn't yet!

In your situation, there is apparently enough free memory in the Java heap and the garbage collector decides to do nothing: as a consequence, unreachable ByteBuffer instances are not collected yet, their finalizer is not run and native memory is not released.

Also, keep in mind this bug in the JVM (not sure how it applies to Dalvik though) where heavy allocation of direct buffers leads to unrecoverable OutOfMemoryError.


You commented about doing controlling things from JNI. This is actually possible, you could implement the following:

  1. publish a native ByteBuffer allocateNative(long size) entry point that:

    • calls void* buffer = malloc(size) to allocate native memory
    • wraps the newly allocated array into a ByteBuffer instance with a call to (*env)->NewDirectByteBuffer(env, buffer, size);
    • converts the ByteBuffer local reference to a global one with (*env)->NewGlobalRef(env, directBuffer);
  2. publish a native void disposeNative(ByteBuffer buffer) entry point that:

    • calls free() on the direct buffer address returned by *(env)->GetDirectBufferAddress(env, directBuffer);
    • deletes the global ref with (*env)->DeleteGlobalRef(env, directBuffer);

Once you call disposeNative on the buffer, you're not supposed to use the reference anymore, so it could be very error prone. Reconsider whether you really need such explicit control over the allocation pattern.


Forget what I said about global references. Actually global references are a way to store a reference in native code (like in a global variable) so that a further call to JNI methods can use that reference. So you would have for instance:

  • from Java, call native method foo() which creates a global reference out of a local reference (obtained by creating an object from native side) and stores it in a native global variable (as a jobject)
  • once back, from Java again, call native method bar() which gets the jobject stored by foo() and further processes it
  • finally, still from Java, a last call to native baz() deletes the global reference

Sorry for the confusion.

§对你不离不弃 2024-10-25 07:38:38

我一直在使用 TurqMage 的解决方案,直到我在 Android 4.0.3 模拟器(Ice Cream Sandwich)上对其进行测试。由于某种原因,对DeleteGlobalRef 的调用失败并出现jni 警告:JNI 警告:非全局0x41301ea8 (type=1) 上的DeleteGlobalRef,随后出现分段错误。

我取出了创建 NewGlobalRef 和 DeleteGlobalRef 的调用(见下文),它似乎在 Android 4.0.3 模拟器上运行良好。事实证明,我只在 java 端使用创建的字节缓冲区,这无论如何应该保存对它的java引用,所以我认为首先不需要调用NewGlobalRef()。

JNIEXPORT jobject JNICALL Java_com_foo_allocNativeBuffer(JNIEnv* env, jobject thiz, jlong size)
{
    void* buffer = malloc(size);
    jobject directBuffer = env->NewDirectByteBuffer(buffer, size);
    return directBuffer;
}

JNIEXPORT void JNICALL Java_comfoo_freeNativeBuffer(JNIEnv* env, jobject thiz, jobject bufferRef)
{
    void *buffer = env->GetDirectBufferAddress(bufferRef);

    free(buffer);
}

I was using TurqMage's solution until I tested it on a Android 4.0.3 emulator (Ice Cream Sandwich). For some reason, the call to DeleteGlobalRef fails with a jni warning: JNI WARNING: DeleteGlobalRef on non-global 0x41301ea8 (type=1), followed by a segmentation fault.

I took out the calls to create a NewGlobalRef and DeleteGlobalRef (see below) and it seems to work fine on the Android 4.0.3 emulator.. As it turns out, I'm only using the created byte buffer on the java side, which should hold a java reference to it anyways, so I think the call to NewGlobalRef() was not needed in the first place..

JNIEXPORT jobject JNICALL Java_com_foo_allocNativeBuffer(JNIEnv* env, jobject thiz, jlong size)
{
    void* buffer = malloc(size);
    jobject directBuffer = env->NewDirectByteBuffer(buffer, size);
    return directBuffer;
}

JNIEXPORT void JNICALL Java_comfoo_freeNativeBuffer(JNIEnv* env, jobject thiz, jobject bufferRef)
{
    void *buffer = env->GetDirectBufferAddress(bufferRef);

    free(buffer);
}
风透绣罗衣 2024-10-25 07:38:38

不确定你最后的评论是旧的还是卡斯帕的。我做了以下操作...

JNIEXPORT jobject JNICALL Java_com_foo_allocNativeBuffer(JNIEnv* env, jobject thiz, jlong size)
{
    void* buffer = malloc(size);
    jobject directBuffer = env->NewDirectByteBuffer(buffer, size);
    jobject globalRef = env->NewGlobalRef(directBuffer);

    return globalRef;
}

JNIEXPORT void JNICALL Java_comfoo_freeNativeBuffer(JNIEnv* env, jobject thiz, jobject globalRef)
{
    void *buffer = env->GetDirectBufferAddress(globalRef);

    env->DeleteGlobalRef(globalRef);
    free(buffer);
}

然后在 Java 中...

mImageData = (ByteBuffer)allocNativeBuffer( mResX * mResY * mBPP );

并且

freeNativeBuffer(mImageData);
mImageData = null;

一切似乎对我来说都工作正常。非常感谢格雷戈里的这个想法。 JVM 中引用的 Bug 的链接已失效。

Not sure if your last comments are old or what Kasper. I did the following...

JNIEXPORT jobject JNICALL Java_com_foo_allocNativeBuffer(JNIEnv* env, jobject thiz, jlong size)
{
    void* buffer = malloc(size);
    jobject directBuffer = env->NewDirectByteBuffer(buffer, size);
    jobject globalRef = env->NewGlobalRef(directBuffer);

    return globalRef;
}

JNIEXPORT void JNICALL Java_comfoo_freeNativeBuffer(JNIEnv* env, jobject thiz, jobject globalRef)
{
    void *buffer = env->GetDirectBufferAddress(globalRef);

    env->DeleteGlobalRef(globalRef);
    free(buffer);
}

Then in Java...

mImageData = (ByteBuffer)allocNativeBuffer( mResX * mResY * mBPP );

and

freeNativeBuffer(mImageData);
mImageData = null;

and everything seems to be working fine for me. Thanks a lot Gregory for this idea. The link to the referenced Bug in the JVM has gone bad.

梦晓ヶ微光ヅ倾城 2024-10-25 07:38:38

使用反射调用 java.nio.DirectByteBuffer.free()。我提醒您,Android DVM 受到 Apache Harmony 的启发,它支持上述方法。

直接 NIO 缓冲区分配在本机堆上,而不是分配在垃圾回收管理的 Java 堆上。由开发人员决定释放其本机内存。这与 OpenJDK 和 Oracle Java 有点不同,因为当直接 NIO 缓冲区创建失败时,它们会尝试调用垃圾收集器,但不能保证它会有所帮助。

注意:如果您使用 asFloatBuffer()、asIntBuffer() 等,则必须进行更多修改,因为只有直接字节缓冲区可以被“释放”。

Use the reflection to call java.nio.DirectByteBuffer.free(). I remind you that Android DVM is inspired by Apache Harmony, which supports the method above.

The direct NIO buffers are allocated on the native heap, not on the Java heap managed by the garbage collection. It's up to the developer to release their native memory. It's a bit different with OpenJDK and Oracle Java because they try to call the garbage collector when the creation of a direct NIO buffer fails but there is no guarantee that it helps.

N.B: You'll have to tinker a bit more if you use asFloatBuffer(), asIntBuffer(), ... because only the direct byte buffer can be "freed".

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文