为什么 MappedByteBuffer 的 array() 方法不起作用?

发布于 2024-12-22 13:53:54 字数 1512 浏览 2 评论 0原文

我对 Java 非常陌生,并尝试使用 Mathematica 的 Java 接口通过内存映射来访问文件(希望提高性能)。

我拥有的 Mathematica 代码(我相信)相当于以下 Java 代码(基于 this ):

import java.io.FileInputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MainClass {
  private static final int LENGTH = 8*100;

  public static void main(String[] args) throws Exception {
    MappedByteBuffer buffer = new FileInputStream("test.bin").getChannel().map(FileChannel.MapMode.READ_ONLY, 0, LENGTH);
    buffer.load();
    buffer.isLoaded(); // returns false, why?
  }
}

我想在缓冲区上使用 array() 方法,因此我尝试首先使用 load() 将缓冲区内容加载到内存中。然而,即使在 load() 之后,isLoaded() 也会返回 false,并且 buffer.array() 会抛出一个异常:java.lang.UnsupportedOperationException 在 java.nio.ByteBuffer.array(ByteBuffer.java:940) 处。

为什么缓冲区不加载以及如何调用 array() 方法?

我的最终目标是使用 asDoubleBuffer().array() 获取一个 double 数组。 getDouble() 方法确实可以正常工作,但我希望一次性完成此操作以获得良好的性能。我做错了什么?


当我从 Mathematica 中执行此操作时,我将发布我也使用的实际 Mathematica 代码(相当于 Java 中的上述代码):

Needs["JLink`"]
LoadJavaClass["java.nio.channels.FileChannel$MapMode"]
buffer = JavaNew["java.io.FileInputStream", "test.bin"]@getChannel[]@map[FileChannel$MapMode`READUONLY, 0, 8*100]

buffer@load[]
buffer@isLoaded[] (* returns False *)

I am very new to Java, and trying to use Mathematica's Java interface to access a file using memory mapping (in hope of a performance improvement).

The Mathematica code I have is (I believe) equivalent to the following Java code (based on this):

import java.io.FileInputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MainClass {
  private static final int LENGTH = 8*100;

  public static void main(String[] args) throws Exception {
    MappedByteBuffer buffer = new FileInputStream("test.bin").getChannel().map(FileChannel.MapMode.READ_ONLY, 0, LENGTH);
    buffer.load();
    buffer.isLoaded(); // returns false, why?
  }
}

I would like to use the array() method on buffer, so I am trying to load the buffers contents into memory first using load(). However, even after load(), isLoaded() returns false, and buffer.array() throws an exception: java.lang.UnsupportedOperationException
at java.nio.ByteBuffer.array(ByteBuffer.java:940)
.

Why doesn't the buffer load and how can I call the array() method?

My ultimate aim here is to get an array of doubles using asDoubleBuffer().array(). The method getDouble() does work correctly, but I was hoping to get this done in one go for good performance. What am I doing wrong?


As I am doing this from Mathematica, I'll post the actual Mathematica code I used too (equivalent to the above in Java):

Needs["JLink`"]
LoadJavaClass["java.nio.channels.FileChannel$MapMode"]
buffer = JavaNew["java.io.FileInputStream", "test.bin"]@getChannel[]@map[FileChannel$MapMode`READUONLY, 0, 8*100]

buffer@load[]
buffer@isLoaded[] (* returns False *)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

风尘浪孓 2024-12-29 13:53:54

根据Javadoc
“映射字节缓冲区的内容可以随时更改,例如,如果映射文件的相应区域的内容被该程序或其他程序更改。无论是否发生此类更改以及何时发生,取决于操作系统,因此未指定。

映射字节缓冲区的全部或部分可能随时变得不可访问,例如,如果映射文件被截断,则尝试访问映射的不可访问区域。字节缓冲区不会改变缓冲区的内容,并会导致在访问时或稍后某个时间抛出未指定的异常,因此强烈建议采取适当的预防措施,以避免此程序或同时运行的映射文件的操作。程序,除了读取或写入文件的内容。”

在我看来,这似乎是许多条件和不良行为。你特别需要这门课吗?

如果您只是需要以最快的方式读取文件内容,请尝试一下:

FileChannel fChannel = new FileInputStream(f).getChannel();
    byte[] barray = new byte[(int) f.length()];
    ByteBuffer bb = ByteBuffer.wrap(barray);
    bb.order(ByteOrder.LITTLE_ENDIAN);
    fChannel.read(bb);

它的工作速度几乎等于磁盘系统测试速度。

对于 double,您可以使用 DoubleBuffer(如果 f.length()/4 大小,则使用 double[] 数组)或仅调用 ByteBuffer 的 getDouble(int) 方法。

According to Javadoc
"The content of a mapped byte buffer can change at any time, for example if the content of the corresponding region of the mapped file is changed by this program or another. Whether or not such changes occur, and when they occur, is operating-system dependent and therefore unspecified.

All or part of a mapped byte buffer may become inaccessible at any time, for example if the mapped file is truncated. An attempt to access an inaccessible region of a mapped byte buffer will not change the buffer's content and will cause an unspecified exception to be thrown either at the time of the access or at some later time. It is therefore strongly recommended that appropriate precautions be taken to avoid the manipulation of a mapped file by this program, or by a concurrently running program, except to read or write the file's content."

To me it seems to many conditions and undesirable misbehavior. Do you need particularly this class?

If you just need to read file contents in fastest way, give a try:

FileChannel fChannel = new FileInputStream(f).getChannel();
    byte[] barray = new byte[(int) f.length()];
    ByteBuffer bb = ByteBuffer.wrap(barray);
    bb.order(ByteOrder.LITTLE_ENDIAN);
    fChannel.read(bb);

It works at speed almost equal to disk system test speed.

For double you can use DoubleBuffer (with double[] array if f.length()/4 size) or just call getDouble(int) method of ByteBuffer.

芯好空 2024-12-29 13:53:54

在Java中:

final byte[] hb;                  // Non-null only for heap buffers

所以它甚至不是为MappedByteBuffer实现的,而是为HeapByteBuffer实现的。

在 Android 中:

**
     * Child class implements this method to realize {@code array()}.
     *
     * @see #array()
     */
    abstract byte[] protectedArray();

同样不在 MappedByteBuffer 中,但例如 ByteArrayBuffer 确实实现了后备数组。

 @Override byte[] protectedArray() {
    if (isReadOnly) {
      throw new ReadOnlyBufferException();
    }
    return backingArray;
  }

内存映射的要点是不在堆外。后备数组将位于堆上。
如果您可以从 RandomAccessFile 打开 FileChannel,然后在通道上调用 map,您还可以使用 MappedByteBuffer 上的批量 get() 方法来读入 byte[]。这会从堆外复制,避免 IO,再次复制到堆中。

buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
byte[] b = new byte[buffer.limit()];
buffer.get(b);

in Java:

final byte[] hb;                  // Non-null only for heap buffers

so it is not even implemented for MappedByteBuffer but is for HeapByteBuffer.

in Android:

**
     * Child class implements this method to realize {@code array()}.
     *
     * @see #array()
     */
    abstract byte[] protectedArray();

and again not in MappedByteBuffer, but for example ByteArrayBuffer does implement the backing array.

 @Override byte[] protectedArray() {
    if (isReadOnly) {
      throw new ReadOnlyBufferException();
    }
    return backingArray;
  }

The point of memory map is to be off heap. A backing array would be on heap.
If you can get the FileChannel open from RandomAccessFile and then call map on the channel, you can also use the bulk get() method on the MappedByteBuffer to read into a byte[]. This copies from off heap, avoiding IO, into heap again.

buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
byte[] b = new byte[buffer.limit()];
buffer.get(b);
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文