使用内存映射文件实现虚拟内存
是否可以像这样包装内存映射文件？
TVirtualMemoryManager = class
public
function AllocMem (Size : Integer) : Pointer;
procedure FreeMem (Ptr : Pointer);
end;
由于内存映射文件 API 函数都采用偏移量，因此我不知道如何管理内存映射文件中的空闲区域。我唯一的想法是实现某种基本的内存管理（维护不同块大小的空闲列表），但我不知道这会有多高效。
编辑：我真正想要的（正如大卫向我明确表示的那样）是：
IVirtualMemory = interface
function ReadMem (Addr : Int64) : TBytes;
function AllocateMem (Data : TBytes) : Int64;
procedure FreeMem (Addr : Int64);
end;
我需要在虚拟内存中存储连续的字节块（每个都相对较小）并且能够使用 64 位地址将它们读回内存。大多数时候访问是只读的。如果需要写入，我只会使用 FreeMem ，然后使用 AllocMem 因为大小无论如何都会不同。
我想要一个带有此接口的内存映射文件的包装器。在内部，它有一个内存映射文件的句柄，并在每个 ReadMem 请求上使用 MapViewOfFile。 Addr 64 位整数只是内存映射文件中的偏移量。悬而未决的问题是如何分配这些地址 - 我目前保留了我维护的空闲块列表。
Is it possible to wrap up memory mapped files something like this?
TVirtualMemoryManager = class
public
function AllocMem (Size : Integer) : Pointer;
procedure FreeMem (Ptr : Pointer);
end;
Since the memory mapped file API functions all take offsets I don't know how to manage the free areas in the memory mapped files. My only idea is to implement some kind of basic memory management (mainting free lists for different block sizes) but I don' t know how efficient this will be.
EDIT: What I really want (as David made clear to me) is this:
IVirtualMemory = interface
function ReadMem (Addr : Int64) : TBytes;
function AllocateMem (Data : TBytes) : Int64;
procedure FreeMem (Addr : Int64);
end;
I need to store continous blocks of bytes (each relatively small) in virtual memory and be able to read them back into memory using a 64-bit adress. Most of the time access is read-only. If a write is necessary I would just use FreeMem followed by AllocMem since the size will be different anyway.
I want a wrapper for a memory mapped file with this interface. Internally it has a handle to a memory mapped files and uses MapViewOfFile on each ReadMem request. The Addr 64-bit integers are just offsets into the memory mapped file. The open question is how to assign those adresses - I currently keep a list of free blocks that I maintain.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
值得一提的是，您的
GetMem / FreeMem要求将无法突破 3/4 GB 的障碍。由于在调用 FreeMem 之前所有分配的内存都将映射到内存中，因此您将缺乏内存空间，就像常规的 Delphi 内存管理器一样。您能做的最好的事情就是依靠FastMM4，并更改您的程序以减少其内存使用。恕我直言，您必须更改/更新您的规格。例如，您的“更新”问题听起来就像常规存储问题。
您希望能够为您的应用程序分配超过 3/4 GB 的数据。您可以在我们的 SynBigTable 开源单元中实现此类功能。这是一个快速、轻量的纯 Delphi 中的 NoSQL 解决方案。
它能够创建任意大小的文件（仅限 64 位），然后根据请求将每个记录的内容映射到内存中。如果可能的话，它将使用文件的内存映射。您可以使用
TSynBigTable方法非常直接地实现您的接口：ReadMem=Get、AllocMem=Add、FreeMem=Delete。 ID 将是类似指针的值，并且将使用RawByteString而不是TBytes。您可以使用整数 ID 或字符串 ID 访问任何数据块，甚至可以使用复杂的字段布局（在记录内，或作为内存中元数据 - 包括索引和快速搜索）。
或者依赖常规的嵌入式 SQL 数据库。例如， SQLite3 非常好擅长处理BLOB字段，并且能够存储大量数据。通过针对大多数使用的记录的简单内存缓存机制，它可能是一个强大的解决方案。
Your proposal that "Internally it has a handle to a memory mapped files and uses MapViewOfFile on each ReadMem request" will be just a waste of CPU resource, IMHO.
It is worth saying that your
GetMem / FreeMemrequirement won't be able to break the 3/4 GB barrier. Since all allocated memory will be mapped into memory until a call toFreeMem, you'll be short of memory space, just as with the regular Delphi memory manager. The best you can do is to rely ofFastMM4, and change your program to reduce its memory use.IMHO you'll have to change/update your specification. For instance, your "updated" question sounds just like a regular storage problem.
What you want is to be able to allocate more than 3/4 GB of data for your application. You have a working implementation of such a feature in our SynBigTable open source unit. This is a fast and light NoSQL solution in pure Delphi.
It is able to create a file of any size (only 64 bit limited), then will map the content of each record into memory, on request. It will use a memory mapping of the file, if possible. You can implement your interface very directly with
TSynBigTablemethods:ReadMem=Get, AllocMem=Add, FreeMem=Delete. The IDs will be yourpointer-like values, andRawByteStringwill be used instead ofTBytes.You can access any block of data using an integer ID, or a string ID, or even use a sophisticated field layout (inside the record, or as in-memory metadata - including indexes and fast search).
Or rely on a regular embedded SQL database. For instance, SQLite3 is very good at handling BLOB fields, and is able to store huge amount of data. With a simple in-memory caching mechanism for most used records, it could be a powerful solution.