设计内存有限的记忆系统的简单方法是什么?

发布于 2024-10-17 22:17:02 字数 524 浏览 4 评论 0原文

我正在编写一个手动计算记忆系统(呃,在 Matlab 中)。简单的部分很简单:

  • 一种在执行计算后将数据放入记忆系统的方法。
  • 一种从记忆中查询和获取数据的方法。
  • 一种查询系统所有“密钥”的方法。

这些部分没有太多疑问。问题是我的计算机的内存量有限,因此有时“放置”操作必须将某些对象转储出内存。我担心“缓存未命中”,所以我想要一些相对简单的系统来删除不经常使用和/或重新计算成本不高的记忆对象。我该如何设计该系统?我可以想象它具有的部分:

  • 一种告诉“放置”操作计算成本(相对而言)有多大的方法。
  • 一种可选地提示“放置”操作可能需要计算的频率(未来)的方法。
  • “get”操作需要记录查询给定对象的频率。
  • 一种告诉整个系统要使用的最大内存量的方法(好吧,这很简单)。

它的真正核心在于“放置”操作,当您达到内存限制时,它必须根据对象的内存占用、成本和有用性来剔除一些对象。我该怎么做?

抱歉,如果这太模糊或偏离主题。

I am writing a manual computation memoization system (ugh, in Matlab). The straightforward parts are easy:

  • A way to put data into the memoization system after performing a computation.
  • A way to query and get data out of the memoization.
  • A way to query the system for all the 'keys'.

These parts are not so much in doubt. The problem is that my computer has a finite amount of memory, so sometime the 'put' operation will have to dump some objects out of memory. I am worried about 'cache misses', so I would like some relatively simple system for dropping the memoized objects which are not used frequently and/or are not costly to recompute. How do I design that system? The parts I can imagine it having:

  • A way to tell the 'put' operation how costly (relatively speaking) the computation was.
  • A way to optionally hint to the 'put' operation how often the computation might be needed (going forward).
  • The 'get' operation would want to note how often a given object is queried.
  • A way to tell the whole system the maximum amount of memory to use (ok, that's simple).

The real guts of it would be in the 'put' operation when you hit the memory limit and it has to cull some objects, based on their memory footprint, their costliness, and their usefulness. How do I do that?

Sorry if this is too vague, or off-topic.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

怀中猫帐中妖 2024-10-24 22:17:02

我会通过创建一个使用单元格的 DYNAMICPROPS 的子类来实现数组用于内部存储数据。这样,您可以动态地向对象添加更多数据。

基本设计思想如下:

数据存储在元胞数组中。每个属性都有自己的行,第一列是属性名称(为了方便起见),第二列是计算数据的函数句柄,第三列是数据,第四列是生成数据所花费的时间,第五列是一个长度为 100 的数组,存储与属性最近 100 次访问时间相对应的时间戳,第六列包含变量大小。

有一个通用的 get 方法,它将与属性对应的行号作为输入(见下文)。 get 方法首先检查第 3 列是否为空。如果否,则返回该值并存储时间戳。如果是,它将使用 TIC/TOC< 内第 1 列的句柄执行计算/a> 语句来衡量计算的成本(存储在 col4 中,时间戳存储在 col5 中)。然后它检查是否有足够的空间来存储结果。如果是,它会存储数据,否则它会检查大小,以及数据被访问的次数与重新生成所需时间的乘积,以决定剔除哪些数据。

此外,还有一个“add”属性,它向元胞数组添加一行,创建一个与函数句柄同名的动态属性(使用 addprops),并设置 get-method到myGetMethod(myPropertyIndex)。如果您需要将参数传递给函数,您可以使用 set 方法创建一个附加属性 myDynamicPropertyName_parameters,每当参数值发生更改时,该属性都会删除之前计算的数据。

最后,您可以添加一些依赖属性,它们可以告诉您有多少个属性(元胞数组中的行数)、它们的调用方式(元胞数组的第一个列)等。

I'd do it by creating a subclass to DYNAMICPROPS that uses a cell array to store the data internally. This way, you can dynamically add more data to the object.

Here's the basic design idea:

The data is stored in a cell array. Each property gets its own row, with the first column being the property name (for convenience), the second column a function handle to calculate the data, the third column the data, the fourth column the time it took to generate the data, the fifth column an array of, say, length 100 storing the timestamps corresponding to when the property was accessed the last 100 times, and the sixth column contains the variable size.

There is a generic get method that takes as input the row number corresponding to the property (see below). The get method first checks whether column 3 is empty. If no, it returns the value and stores the timestamp. If yes, it performs the computation using the handle from column 1 inside a TIC/TOC statement to measure how expensive the computation is (which is stored in col4, and the timestamp is stored in col5). Then it checks whether there is enough space for storing the result. If yes, it stores the data, otherwise it checks size, as well as the product of how many times data were accessed with how long it would take to regenerate, to decide what to cull.

In addition, there is an 'add' property, that adds a row to the cell array, creates a dynamic property (using addprops) of the same name as the function handle, and sets the get-method to myGetMethod(myPropertyIndex). If you need to pass parameters to the function, you can create an additional property myDynamicPropertyName_parameters with a set method that will remove previously calculated data whenever the parameters change value.

Finally, you can add a few dependent properties, that can tell how many properties there are (# of rows in the cell array), how they're called (first col of the cell array), etc.

最单纯的乌龟 2024-10-24 22:17:02

考虑使用 Java,因为 MATLAB 在它之上运行,并且可以访问它。如果您有可编组值(整数、双精度数、字符串、矩阵,但没有结构体或元胞数组),则这将起作用。

LRU 容器可用于 Java:

Consider using Java, since MATLAB runs on top of it, and can access it. This would work if you have marshallable values (ints, doubles, strings, matrices, but not structs or cell arrays).

LRU containers are available for Java:

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文