从 CFC 返回查询的内存影响

发布于 2024-08-23 11:25:55 字数 394 浏览 8 评论 0原文

我在 ColdFusion 中编写了一个数据库加载脚本,但遇到了该脚本慢慢耗尽内存的问题。我已使用将每个表加载拆分为自己的线程。当内存低于 50% 时,我会调用垃圾收集器(确保 gc() 调用之间有 30 秒的时间,以防止垃圾收集器占用内存)。

我创建了一个 CFC 来保存脚本所需的所有查询。该脚本调用相应的 CFC 函数,该函数然后返回查询,其中一些查询大小超过 2 MB。当我在“活动线程的内存”页面的详细信息视图中查看“服务器监视器”时,看起来我的 CFC 正在内存中保留查询的副本,即使我对查询变量进行了 varscope 处理并且该变量最后超出了范围的函数。此外,我的线程内存中有查询的副本。所以我在内存中留下了看起来像查询的两个副本。这真的是发生的事情吗?如果是,如何从内存中消除该查询的一份副本?

I've written a database load script in ColdFusion and I'm having a problem that the script slowly runs out of memory. I've split each table load into its own thread with <cfthread> and I'm calling the garbage collector when memory dips below 50% (making sure to have 30 seconds between gc() calls to prevent the garbage collector from hogging memory).

I created a CFC to hold all the queries needed by the script. The script calls the appropriate CFC function which then returns the query, some of which are over 2 MB in size. When I look in the Server Monitor in the details view of the Memory page for Active Threads, it looks like my CFC is keeping a copy of the query in memory even though I varscoped the query variable and the variable went out of scope at the end of the function. In addition, I have a copy of the query in memory in my thread. So I'm left with what looks like two copies of the query in memory. Is this really what's happening? If it is, how can I eliminate one copy of the query from memory?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

简单 2024-08-30 11:25:55

这里有很多潜在的问题,但我将尝试强调一些需要您考虑的最重要的事情:

  1. 为什么要使用线程?你需要线程吗?在某个时刻,你可能会为了自己的利益而进行过多的修改。
  2. 手动强制垃圾收集不一定是个好主意。调整 JVM 以自动执行垃圾收集,但也不要过度。垃圾收集往往很昂贵,并且如果应用程序运行过于频繁,可能会影响应用程序的性能。
  3. 您如何实例化您的 CFC?如果您在每个查询请求上实例化 CFC,随着时间的推移,您将遇到 RAM 问题,即缓慢的内存泄漏,因为 CFC 加载到 RAM 的速度太快,垃圾收集无法跟上。你最好的选择是使其成为单例。 (即,将其设置到应用程序范围中)。
  4. 请注意,变量的 var 作用域不会(据我所知)在变量停止使用后立即自动释放内存。内存仍然被保留,尽管它可能以某种方式标记为短暂生成的一部分,以便它(可能?)更快地被清理。但这并不能保证任何事情。
  5. 如果您正在查看活动线程,则也有可能直到请求结束(不一定是函数调用结束)才清除查询。似乎不耐烦会促使您期望查询在函数调用完成后立即终止。
  6. ColdFusion 传递查询通过引用,而不是通过值。应该不可能在内存中获得查询的 2 个副本,除非您以某种方式使用重复()或类似的函数来显式复制查询。

该查询可能会从您的 cfreturn 语句返回指向该查询的指针。在所有进程完成引用该查询之前,该查询不会被清除。因此,如果它将查询传递给其他进程,您将不会清除该查询的内存。例如,如果将该查询设置为会话变量,则无论您尝试强制垃圾回收的频率如何,该指针都不会移动到任何地方,直到该会话变量消失。

只需考虑几件事。

There's a lot of potential issues here, but I'll try to underline some of the most important things for you to consider:

  1. Why the threads? Do you need the threads? There's a certain point at which you're probably tinkering too much for your own good.
  2. Manually forcing garbage collection isn't necessarily a good idea. Tune the JVM to perform its garbage collection automatically, but don't overdo it, either. Garbage Collection tends to be expensive, and can impact the performance of your app if it is running too frequently.
  3. How are you instantiating your CFC? If you are instantiating the CFC on every request for the query, you're going to experience RAM issues over time, a slow memory leak as CFCs are loaded up into RAM too quickly for garbage collection to keep up. Your best bet is to make this a singleton. (ie., set it into the application scope).
  4. Be aware that var-scoping a variable doesn't (as far as I understand it) automatically free up the memory as soon as the variable stops being used. The memory is still reserved, though it's likely flagged somehow as being part of a short-lived generation so that it will (probably?) be cleaned up faster. But this doesn't guarantee anything.
  5. If you're looking at active threads, it's also possible that the query isn't going to be cleared until the end of the request--not necessarily the end of the function call. It seems impatience that would motivate you to expect a query to immediately die as soon as the function call is completed.
  6. ColdFusion queries are passed by reference, not by value. It should be impossible to get 2 copies of the query in memory, unless you're somehow using duplicate() or a similar function to explicitly copy the query.

The query is likely returning a pointer to the query from your cfreturn statement. That query will not be cleaned up until all processes are done referencing it. So if it passes the query to some other process, you're not going to get that query cleaned out of memory. If you set that query to a session variable, for instance, that pointer isn't going anywhere until that session variable is gone, no matter how frequently you try to force garbage collection.

Just a few things to consider.

南渊 2024-08-30 11:25:55

我在处理大型数据插入时遇到了类似的问题,其中每一行都需要涉及多个 CFC 的大量处理。看来创建的 JDBC ResultSet、Statement 和 Connection 引用是正确的。一直保留到请求结束。这意味着将查询变量清空不会影响内存使用。我解决这个问题的方法是对 CFC 函数进行网关调用以处理 100 行,然后该函数对接下来的 100 行进行另一个网关调用,依此类推,直到处理完所有行。因为每个单独的网关调用实际上都会退出,所以它会释放所有句柄并恢复内存。

I had a similar problem with processing a large data insert, where each row requires extensive processing involving multiple CFCs. It appears that th JDBC ResultSet, Statement and Connection references created by <cfquery> are held until the end of the request. This means that nulling your query variable has no affect on memory usage. The way I got around this was to make a gateway call to a CFC function to processes 100 rows, then that function makes another gateway call for the next 100 rows etc until all rows are processed. Because each individual gateway call actually exits, it releases all it's handles and that memory gets recovered.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文