“现场 C”存在于内存映射文件中的对象”?

发布于 2024-12-01 02:30:57 字数 631 浏览 2 评论 0原文

因此,我在 Gamasutra 中阅读了 John Carmack 的采访,其中他谈到了他所说的“存在于内存映射文件中的活动 C++ 对象”。以下是一些引述:

JC:是的。事实上,我从中得到了多种好处...在上一个 iOS Rage 项目中,我们采用了一些新技术,该技术使用一些巧妙的东西来创建存在于内存映射文件中的实时 C++ 对象,并由闪存文件系统支持在这里,这就是我想要构建我们所有未来在 PC 上的工作的方式。

...

我对自己的命令是,我希望 PC 平台上的游戏加载时间为两秒,这样我们就可以更快地迭代。现在,即使使用固态驱动器,您也会受到加载时所做的所有事情的支配,因此需要这种不同的规则才能说“所有内容都将被删除并在相对地址中使用”,所以你只需说,“映射文件,我所有的资源都在那里,15 毫秒内就完成了。”

此处找到)

(完整采访可以在 知道卡马克在说什么以及你将如何建立这样的东西吗?我在网上搜索了一下,但似乎找不到任何相关内容。

So I read this interview with John Carmack in Gamasutra, in which he talks about what he calls "live C++ objects that live in memory mapped files". Here are some quotes:

JC: Yeah. And I actually get multiple benefits out of it in that... The last iOS Rage project, we shipped with some new technology that's using some clever stuff to make live C++ objects that live in memory mapped files, backed by the flash file system on here, which is how I want to structure all our future work on PCs.

...

My marching orders to myself here are, I want game loads of two seconds on our PC platform, so we can iterate that much faster. And right now, even with solid state drives, you're dominated by all the things that you do at loading times, so it takes this different discipline to be able to say "Everything is going to be decimated and used in relative addresses," so you just say, "Map the file, all my resources are right there, and it's done in 15 milliseconds."

(Full interview can be found here)

Does anybody have any idea what Carmack is talking about and how you would set up something like this? I've searched the web for a bit but I can't seem to find anything on this.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

荭秂 2024-12-08 02:30:57

这个想法是,通过内存映射访问该文件,您可以随时将全部或部分程序状态序列化到文件中。这将要求您没有通常的指针,因为指针仅在您的进程持续时才有效。相反,您必须存储从映射开始的偏移量,以便当您重新启动程序并重新映射文​​件时,您可以继续使用它。此方案的优点是您没有单独的序列化,这意味着您没有额外的代码,并且不需要立即保存所有状态 - 相反,您的(全部或大部分)程序状态是始终由文件支持。

The idea is that you have all or part of your program state serialized into a file at all times by accessing that file via memory mapping. This will require you not having usual pointers because pointers are only valid while your process lasts. Instead you have to store offsets from the mapping start so that when you restart the program and remap the file you can continue working with it. The advantage of this scheme is that you don't have separate serialization which means you don't have extra code for that and you don't need to save all the state at once - instead your (all or most of) program state is backed by the file at all times.

放赐 2024-12-08 02:30:57

您可以直接或通过自定义分配器使用新的放置位置。

查看 EASTL 的实现(子集)STL,专门适合与自定义分配方案良好配合(例如在嵌入式系统或游戏控制台上运行的游戏所需的)。

EASTL 的免费子集位于:

You'd use placement new, either directly or via custom allocators.

Look at EASTL for an implementation of (subset) STL that is specifically geared to working well with custom allocation schemes (such as required for games running on embedded systems or game consoles).

A free subset of EASTL is here:

傾旎 2024-12-08 02:30:57

我们多年来一直使用一种称为“相对指针”的东西,它是某种智能指针。它本质上是非标准的,但在大多数平台上都能很好地工作。它的结构如下:

template<class T>
class rptr
{
    size_t offset;
public:
    T* operator->() { return reinterpret_cast<T*>(reinterpret_cast<char*>(this)+offset); }
};

这要求所有对象都存储在同一共享内存中(也可以是文件映射)。它通常还要求我们只在那里存储我们自己的兼容类型,并且必须编写自己的分配器来管理该内存。

为了始终拥有一致的数据,我们通过 COW mmap 技巧使用快照(它在 Linux 上的用户空间中工作,不知道其他操作系统)。

随着向 64 位的重大转变,我们有时也只使用固定映射,因为相对指针会产生一些运行时开销。通常有 48 位地址空间,我们为我们的应用程序选择了一个保留的内存区域,我们总是将这样的文件映射到其中。

We use for years something we call "relative pointers" which is some kind of smart pointer. It is inherently nonstandard, but works nice on most platforms. It is structured like:

template<class T>
class rptr
{
    size_t offset;
public:
    T* operator->() { return reinterpret_cast<T*>(reinterpret_cast<char*>(this)+offset); }
};

This requires that all objects are stored into the same shared memory (which can be a filemap too). It also usually requires us to only store our own compatible types in there, as well as havnig to write own allocators to manage that memory.

To always have consistent data, we use snapshots via COW mmap tricks (which work in userspace on linux, no idea about other OSs).

With the big move to 64bit we also sometimes just use fixed mappings, as the relative pointers incur some runtime overhead. With usually 48bits of address space, we chose a reserved memry area for our applications that we always map such a file to.

深府石板幽径 2024-12-08 02:30:57

这让我想起了我提出的一个文件系统,它可以在极短的时间内加载 CD 的级别文件(它将加载时间从 10 秒提高到接近瞬时),并且它也适用于非 CD 媒体。它由一个类的三个版本组成,用于包装文件 IO 函数,所有版本都具有相同的接口:

class IFile
{
public:
  IFile (class FileSystem &owner);
  virtual Seek (...);
  virtual Read (...);
  virtual GetFilePosition ();
};

和一个附加类:

class FileSystem
{
public:
  BeginStreaming (filename);
  EndStreaming ();
  IFile *CreateFile ();
};

并且您可以编写如下加载代码:

void LoadLevel (levelname)
{
  FileSystem fs;
  fs.BeginStreaming (levelname);
  IFile *file = fs.CreateFile (level_map_name);
  ReadLevelMap (fs, file);
  delete file;
  fs.EndStreaming ();
}

void ReadLevelMap (FileSystem &fs, IFile *file)
{
  read some data from fs
  get names of other files to load (like textures, object definitions, etc...)
  for each texture file
  {
    IFile *texture_file = fs.CreateFile (some other file name)
    CreateTexture (texture_file);
    delete texture_file;
  }
}

然后,您将具有三种操作模式:调试模式,流文件构建模式和发布模式。

在每种模式下,FileSystem 对象都会创建不同的 IFile 对象。

在调试模式下,IFile 对象仅包装标准 IO 函数。

在流文件构建中,IFile对象还包装了标准IO,但具有将读取的每个字节写入流文件(所有者FileSystem打开流文件)的附加功能,以及写入任何文件指针位置查询的返回值(因此,如果需要了解文件大小,则该信息将写入流文件)。这会将各种文件连接成一个大文件,但仅包含实际读取的数据。

释放模式将创建一个不打开文件或在文件内查找的 IFile,它只是从流文件中读取(由所有者 FileSystem 对象打开)。

这意味着在发布模式下,所有数据都是在一系列连续的读取中读取的(操作系统会很好地缓冲它),而不是大量的查找和读取。这对于寻道时间非常慢的 CD 来说是理想的选择。不用说,这是为基于 CD 的控制台系统开发的。

副作用是数据被剥离了通常会被跳过的不必要的元数据。

它确实有缺点 - 一个级别的所有数据都在一个文件中。这些可能会变得非常大,并且数据无法在文件之间共享,例如,如果您有一组在两个或多个级别上通用的纹理,则数据将在每个流文件中重复。另外,每次加载数据时的加载过程必须相同,不能有条件地跳过或向某个级别添加元素。

This reminds me of a file system I came up with that loaded level files of CD in an amazingly short time (it improved the load time from 10s of seconds to near instantaneous) and it works on non-CD media as well. It consisted of three versions of a class to wrap the file IO functions, all with the same interface:

class IFile
{
public:
  IFile (class FileSystem &owner);
  virtual Seek (...);
  virtual Read (...);
  virtual GetFilePosition ();
};

and an additional class:

class FileSystem
{
public:
  BeginStreaming (filename);
  EndStreaming ();
  IFile *CreateFile ();
};

and you'd write the loading code like:

void LoadLevel (levelname)
{
  FileSystem fs;
  fs.BeginStreaming (levelname);
  IFile *file = fs.CreateFile (level_map_name);
  ReadLevelMap (fs, file);
  delete file;
  fs.EndStreaming ();
}

void ReadLevelMap (FileSystem &fs, IFile *file)
{
  read some data from fs
  get names of other files to load (like textures, object definitions, etc...)
  for each texture file
  {
    IFile *texture_file = fs.CreateFile (some other file name)
    CreateTexture (texture_file);
    delete texture_file;
  }
}

Then, you'd have three modes of operation: debug mode, stream file build mode and release mode.

In each mode, the FileSystem object would create different IFile objects.

In debug mode, the IFile object just wrapped the standard IO functions.

In stream file building, the IFile object also wrapped the standard IO but had the additional functions of writing to the stream file (the owner FileSystem opened the stream file) every byte that was read, and writing the return value of any file pointer position queries (so if anything needed to know a file size, that information is written to the stream file). This would sort of concatenate the various files into one big file, but only the data that was actually read.

The release mode would create an IFile that did not open files or seek within files, it just read from the streaming file (as opened by the owner FileSystem object).

This means that in release mode, all data is read in one sequential series of reads (the OS would buffer it nicely) rather than lots of seeks and reads. This is ideal for CDs where seek times are really slow. Needless to say, this was developed for a CD based console system.

A side effect is that the data is stripped of unnecessary meta data that would normally be skipped.

It does have drawbacks - all the data for a level is in one file. These can get quite large and the data can't be shared between files, if you had a set of textures, say, that were common across two or more levels, the data would be duplicated in each stream file. Also, the load process must be the same every time the data is loaded, you can't conditionally skip or add elements to a level.

她如夕阳 2024-12-08 02:30:57

正如卡马克指出的那样,许多游戏(和其他应用程序)加载代码都是结构化的,就像很多小的读取和分配一样。

您可以不执行此操作,而是将关卡文件执行一次fread(或等效操作)到内存中,然后修复指针。

As Carmack indicates many games (and other applications) loading code is structured lika a lot of small reads and allocations.

Instead of doing this you do a single fread (or equivalent) of say a level file into memory and just fixup the pointers afterwards.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文