ActiveMQ 主题内存使用情况

发布于 2024-12-10 00:58:52 字数 447 浏览 4 评论 0原文

我需要发布大量[仅标题]消息,每条消息都属于一个唯一的类别。考虑:

A.B.C.D.E.F

理想情况下,抛开所有其他考虑因素,我想在自己的主题上发布每条消息,但这可能会导致数千个主题。好处是非常简单的基于通配符的订阅者模型。我只是不确定单个主题的基准内存使用情况以及数千个主题的聚合使用情况是多少。

对于调优,主题将是非持久且非持久的。如果可能的话,我还可以将消息数量限制为一次 1,这样较旧的消息将被删除。消息将每n秒(通常> 15秒)分波发布。这将减少内存使用量。

我的替代方案是根据前 2 个片段(如:

AB)

发布到主题,并将其余片段放入键控标题中。然后,订阅者必须使用主题通配符和选择器的组合来订阅他们想要的提要。

有人对此有任何见解吗?

谢谢 !

//尼古拉斯

I need to publish a large number of [header only] messages, each of which belongs to a unique category. Consider:

A.B.C.D.E.F

Ideally, all other considerations aside, I would like to publish each message on its own topic but this could result in thousands of topics. The benefit is a very simple wildcard based subscriber model. I am just not sure what the baseline memory usage is of a single topic and the aggregate usage of thousands of them.

For tuning, the topics will be non-persisted and non-durable. If possible, I could also limit the number of messages to 1 at a time, where an older message would be dropped. Messages will be published in waves every n seconds (usually > 15s). This would the reduce the memory usage.

My alternative is to publish to topics based on the first 2 segments like:

A.B

and put the rest of the segments in keyed headers. Subscribers would then have to use a combination of topic wildcards and selectors to subscribe to their desired feeds.

Anyone have any insight into this ?

Thanks !

//Nicholas

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

ぃ双果 2024-12-17 00:58:52

我设置了一个测试来测量主题的实际内存消耗。简而言之:

  1. 我使用基本数量的主题启动一个循环,这些主题是通过 JMS 会话即时创建的。
  2. 在内部循环中,我调用 System.gc(),然后测量使用的堆空间。重复 10 次,每次 10 秒。在每个内循环结束时暂停。
  3. 我计算在内循环中采样的平均堆使用情况。
  4. 我将主题的基本数量增加 100。

结果如下所示:

在此处输入图像描述

总之,每个主题消耗大约 29.5 KB 的堆空间。请注意,我没有花费太多精力来调整主题的目标策略,这可能会在一定程度上减少这种情况。

总的来说,我不认为这很糟糕,但我认为它不会扩展到我想要的数字,所以我使用 Camel 采取不同的方法(如 boday)。

基本上,当客户端使用主题通配符订阅时,我会缓存该模式。当Camel消费者收到[non-jms]消息时,它会检查缓存,如果该消息与缓存中的任何通配符条目匹配,它将发布到相应的主题(将动态创建)。一旦主题停止接收发布(因为缓存已删除匹配的通配符),该主题将超时并被“GC”。

效果相当好。

//尼古拉斯

I setup a test to measure the actual memory consumption of topics. In a nutshell:

  1. I start a loop with a base number of topics, created on the fly through a JMS session.
  2. In an inner loop, I call System.gc() and then measure the heap space used. This is done 10 times with a 10 s. pause at the end of each inner loop.
  3. I calculate the average heap usage sampled in the inner loop.
  4. I increment the base number of topics by 100.

The results looked like this:

enter image description here

In summary, each topic consumed about 29.5 KB of heap space. Note that I did not expend much effort in tuning the topics' destination policies which might have reduced this somewhat.

Overall, I don't think this is bad, but I don't think it will scale to the numbers I want it to, so I am taking a different approach using Camel (as suggested by boday).

Basically, when a client subscribes using a topic wildcard, I am caching the pattern. When the Camel consumer receives a [non-jms] message, it checks the cache and if the message matches any of the wildcard entries in the cache, it will publish to the corresponding topic (which will be dynamically created). Once a topic stops receiving publications (because the cache had a matching wildcard removed), the topic will timeout and be "GC'ed".

Works fairly well.

//Nicholas

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文