我如何知道我的缓存对象使用了多少内存?

发布于 2024-11-11 14:03:21 字数 283 浏览 3 评论 0原文

我们正在尝试缓存数据库选择的结果(在哈希映射中),因此我们不必多次执行它们。每当我们更改数据库时,为了获取应用程序中的更改,我们添加了刷新列表功能。

现在我们有大量的列表需要获取,因此从数据库加载选择列表需要花费太多时间。

所以我对这个问题有一些疑问:

  1. 如何找出列表使用了多少内存? (我已经使用了使用垃圾收集器来收集内存并获取差异的方法,但是有很多列表,因此花费了太多时间)

  2. 如何优化刷新列表?

感谢您的帮助。

We are trying to cache the results of database selects (in hash map), so we wouldn’t have to execute them multiple times. and whenever we are changing data base, so for getting the changes in app we have added refresh list functionality.

Now we have a large no of list to fetch, so it taking too much time to load pick list from the data base.

So I have some question regarding this issue:

  1. How I can find out how much memory the list is using? (I have used the method where we are using garbage collector for collecting the memory and taking the difference but there are many list and so it is taking too much time)

  2. How I can optimize the refresh list?

Thanks for the help.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

情话墙 2024-11-18 14:03:22

首先,虽然不想在性能问题方面一概而论,但您所看到的问题不太可能纯粹归因于内存使用,尽管如果列表很大,则在刷新它们并且数量很大时,这可能会发挥作用对象有资格收集。

要解决与垃圾收集相关的问题,有一些经验法则,但归根结底总是要打破分析器并调整垃圾收集器 - 还有更多内容 此处

但在此之前,数据库的任何加载都将涉及结果集的迭代,因此您可以进行的最大优化将是减少结果集的大小。有几种方法可以做到这一点:

  1. 如果您使用地图,请尝试使用不需要加载的键,并在错过时进行加载。
  2. 加载后,仅刷新自上次加载数据以来已更改的行,尽管这显然不能解决启动问题。

话虽如此,我首先不建议您编写自己的缓存代码。我这样说的原因是:

  1. 所有现代 RDBMS 缓存,因此只要您的查询具有高性能,获取实际结果集就不应该成为瓶颈。
  2. Hibernate 不仅提供 ORM,还提供强大且易于理解的缓存解决方案。
  3. 如果您确实需要缓存大量数据集,请使用 Coherence 或类似的 - 缓存可以在单独的 JVM 中启动,并且您的应用程序不需要承受负载。

First, while not wanting to generalize when it comes to performance problems, the issue you're seeing are unlikely to be purely down to memory use, though if the lists are large this could come into play when they're refreshed and a large number of objects become eligible for collection.

To solve issues relating to garbage collection there's a few rules of thumb, but it always comes down to breaking out a profiler an tuning the garbage collector - there's more on that here.

But before that any loading of a database is going to involve iteration over a result set, so the biggest optimization you can make will be to reduce the size of the result sets. There's a couple of ways to do that:

  1. if you using a map, try to use keys that don't require loading and do the load when you get a miss.
  2. once loaded, only refresh the rows that have changed since you last loaded the data, though this obivously doesn't solve the start-up problem.

Now all that said, I would not recommend you write your own caching code in the first place. The reasons I say this are:

  1. all modern RDBMS cache, so providing your queries are performant getting the actual result set should not be a bottleneck.
  2. Hibernate provides not only ORM but a robust and well understood caching solution.
  3. if you really need to cache massive datasets, use Coherence or similar - the cache can be started in a seperate JVM and your application doesn't need to take the load hit.
入画浅相思 2024-11-18 14:03:22

这里有两个问题:发现正在使用多少内存以及管理缓存。我不确定这两者是否真的密切相关,尽管可能确实如此。

了解对象使用了多少内存并不是非常困难:您可以参考的一篇优秀文章是“Sizeof for Java" 来自 JavaWorld。它避免了整个垃圾收集惨败,其中有很多漏洞(它很慢,它不计算对象,而是计算堆 - 这意味着其他对象会影响您可能不想要的结果,等等

)初始化缓存的时间是另一个问题。我在一家以数据网格为产品的公司工作,因此我有偏见;请注意。

一种选择是根本不使用缓存,而是使用数据网格。我在 GigaSpaces Technologies 工作,我觉得我们的是最好的;我们可以在启动时从数据库加载数据,并将您的数据作为分布式事务数据存储保存在内存中(因此您最大的成本是网络访问。)我们有社区版以及全功能平台,具体取决于您的需要和预算。 (社区版是免费的。)我们支持各种协议,包括 JDBC、JPA、JMS、Memcached、Map API(类似于 JCache)和本机 API。

其他类似的选项包括 Coherence(它本身就是一个数据网格)和 Terracotta DSO(可以在 JVM 堆上分布对象图)。

您还可以查看缓存项目本身:其中两个包括 Ehcache 和 OSCache。 (再次强调:偏见。我是 OpenSymphony 的创始人之一,所以我对 OSCache 情有独钟。)在你的情况下,会发生的不是缓存的预加载 - 请注意,我不知道你的应用程序,所以我猜测并且可能是错误的 - 但是按需缓存。当你获取数据时,你会首先检查缓存中是否有数据,只有当数据不在缓存中时才从数据库中获取,并在读取时加载缓存。

当然,您也可以查看 memcached,尽管我显然更喜欢我雇主在这里提供的产品。

You have two problems here: discovering how much memory is in use, and managing a cache. I'm not sure that the two are really closely related, although they may be.

Discovering how much memory an object uses isn't extremely difficult: one excellent article you can use for reference is "Sizeof for Java" from JavaWorld. It escapes the whole garbage collection fiasco, which has a ton of holes in it (it's slow, it doesn't count the object but the heap - meaning that other objects factor into your results that you may not want, etc.)

Managing the time to initialize the cache is another problem. I work for a company that has a data grid as a product, and thus I'm biased; be aware.

One option is not using a cache at all, but using a data grid. I work for GigaSpaces Technologies, and I feel ours is the best; we can load data from a database on startup, and hold your data there as a distributed, transactional data store in memory (so your greatest cost is network access.) We have a community edition as well as full-featured platforms, depending on your need and budget. (The community edition is free.) We support various protocols, including JDBC, JPA, JMS, Memcached, a Map API (similar to JCache), and a native API.

Other similar options include Coherence, which is itself a data grid, and Terracotta DSO, which can distribute an object graph on a JVM heap.

You can also look at the cache projects themselves: Two include Ehcache and OSCache. (Again: bias. I was one of the people who started OpenSymphony, so I've a soft spot for OSCache.) In your case, what would happen is not a preload of cache - note that I don't know your application, so I'm guessing and might be wrong - but a cache on demand. When you acquire data, you'd check the cache for data first and fetch from the DB only if the data is not in cache, and load the cache on read.

Of course, you can also look at memcached, although I obviously prefer my employer's offering here.

谎言 2024-11-18 14:03:22

请注意,

System.gc()

否则调用or

Runtime.getRuntime().gc()

除非您确实需要这样做, 是一个坏主意。您应该让 VM 来决定何时释放对象,除非在分析后您发现这是使应用程序在客户端 VM 上运行得更快的唯一方法。

Be aware that invoking

System.gc()

or

Runtime.getRuntime().gc()

is a bad idea unless you really need to do that. You should leave the VM the task of deciding when to free objects, unless after profiling you found that it's the only way to make the application go faster on your client's VM.

秋凉 2024-11-18 14:03:22

我倾向于使用 YourKit 来处理这类事情。它需要花钱,但在我看来,每一分钱都是值得的(除了作为客户之外没有任何联系)。

I tend to use YourKit for this sort of thing. It costs money but IMO is worth every penny (no connection other than as a customer).

巷雨优美回忆 2024-11-18 14:03:21

如何找到列表使用了多少内存

如何优化刷新列表。

确保您为数据使用正确的集合类型。
请查看此处

另请查看番石榴集合


最后一件事,ignis 建议您不要使用 System.gc() ,这是非常正确的,这可能就是您遇到性能问题的原因。 就是原因。

how i can find how much memory the list is using

how i can optimize the refresh list.

Make sure you're using the correct collection type for your data.
Have a look here.

Also have a look at the Guava collections.


One last thing, ignis is very right by advising you not to use System.gc() this might be the very reason you're having performance problems. This is why.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文