Terracotta 性能和技巧

发布于 2024-07-18 17:00:20 字数 404 浏览 6 评论 0原文

大约一个月前我发现了 Terracotta,现在正在学习如何使用它。 这是一项非常酷的技术。

基本上我想做的是:

我的根(记录系统)是一个 ConcurrentHashMap。

主要的 Instrumented 类是一个“JavaBean”,包含 30 个左右的字段,我希望它们存在于 HashMap 中。

Hashmap 中将存在大约 20000 个这样的 JavaBean。

每个 bean(至少)有 5 个字段,每 5 秒更新一次。

(我使用 Terracotta 的原因是因为这些 JavaBean 需要能够跨 JVM 和节点访问。)

任何比我更有 TC 经验的人都有什么建议吗? 性能是关键。

还有其他类似应用程序的例子吗?

I am just learning how to use Terracotta after discovering it about a month ago. It is a very cool technology.

Basically what I am trying to do:

My root (System of Record) is a ConcurrentHashMap.

The main Instrumented Class is a "JavaBean" with 30 or so fields that I want to exist in the HashMap.

There will be about 20000 of these JavaBeans that exist in the Hashmap.

Each bean has (at least) 5 fields that will be updated every 5 seconds.

(The reason I am using Terracotta for this is because these JavaBeans need to be accessible across JVMs and nodes.)

Anyone with more experience than me with TC have any tips? Performance is key.

Any examples other similar applications?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

惯饮孤独 2024-07-25 17:00:20

您可能会发现,在一个锁定范围内批量进行多个更改会执行得更好。 每个同步块/方法形成一个写事务(假设您使用写锁),该事务必须发送到服务器(并且可能返回到其他节点)。 通过更改一堆字段(可能是在一个锁下的一堆对象上),可以减少创建事务的开销。 至少有东西可以玩。

分区也是提高性能的关键方法。 更改只需要发送到实际使用对象的节点。 因此,如果您可以对哪些节点通常接触特定对象进行分区,就可以减少必须在集群中发送的更改数量,从而提高性能。

unnutz 关于使用 CHM 或 CSM 的建议很好。 CHM 允许更大的并发性(因为每个内部段都可以被锁定并同时使用) - 确保也尝试更大的段数。 CSM 实际上对每个条目有一个锁,因此在 N 大小的表中实际上有 N 个分区。 这可以大大减少锁争用(以管理更多内部锁对象为代价)。 CSM 即将发生的变化将使锁定管理成本大大降低。

一般来说,我们发现一个好的策略是:

  1. 构建性能测试(应该是多线程和多节点,并且类似于您的应用程序(或您的实际应用程序!)
  2. 调整对象 - 在开发控制台中查看集群对象图以查找根本不需要聚类的对象 - 有时这种情况会意外发生(删除或剪切带有瞬态字段的聚类)有时您可能会对一个 Date 进行聚类,其中 long 会做一些小更改,但这只是每个映射条目一个对象。 。
  3. 锁 - 使用开发控制台中的锁分析器来查找热锁或太窄或太宽的锁,集群统计记录器也可以帮助查看事务大小
  4. 调整 - 调整 JVM 垃圾收集,然后通过打开更改年轻代 GC
  5. 服务器的频率来调整 Terracotta 分布式 GC - 这里需要进行大量非常详细的调整,但通常不值得,直到上面的内容调整完毕为止

。也在 Terracotta 论坛上提问 - 所有工程、现场工程、产品管理人员都会观看这些内容并在那里回答。

You might find that batching several changes under one lock scope will perform better. Each synchronized block/method forms a write transaction (assuming you use a write lock) that must be sent to the server (and possibly back out to other nodes). By changing a bunch of fields, possibly on a bunch of objects under one lock, you reduce the overhead of creating a transaction. Something to play with at least.

Partitioning is also a key way to improve performance. Changes only need to be sent to nodes that are actually using an object. So if you can partition which nodes usually touch specific objects that reduces the number of changes that have to be sent around the cluster, which improves performance.

unnutz's suggestions about using CHM or CSM are good ones. CHM allows greater concurrency (as each internal segment can be locked and used concurrently) - make sure to experiment with larger segment counts too. CSM has effectively one lock per entry so has effectively N partitions in an N-sized table. That can greatly reduce lock contention (at the cost of managing more internal lock objects). Changes coming soon for CSM will make the lock mgmt cost much lower.

Generally we find a good strategy is:

  1. Build a performance test (should be multi-threaded and multi-node and similar to your app (or your actual app!)
  2. Tune objects - look at your clustered object graph in the dev-console to find objects that don't need to be clustered at all - sometimes this happens accidentally (remove or cut the cluster with a transient field). Sometimes you might be clustering a Date where a long would do. Small change but that's one object per map entry and that might make a difference.
  3. Tune locks - use the lock profiler in the dev-console to find hot locks or locks that are too narrow or too wide. The clustered stats recorder can help look at transaction size as well.
  4. Tune GC and DGC - tune JVM garbage collection, then tune Terracotta distributed GC by turning on changing the frequency of young gen gc.
  5. Tune TC server - lots of very detailed tunings to do here, but usually not worth it till the stuff above is tuned.

Feel free to ask on the Terracotta forums as well - all of engineering, field engineering, product mgmt watch those and answer there.

羞稚 2024-07-25 17:00:20

首先,我建议您也在他们的论坛上提出这个问题。

其次,实际上,Terracotta 上集群的应用程序的性能将取决于发生的写入事务的数量。 因此,您可以考虑使用 ConcurrentStringMap (如果您的键是字符串)或 ConcurrentHashMap。 请注意,从性能角度来看,CSM 比 CHM 好得多。

毕竟,POJO 是延迟加载的。 这意味着每个属性都是按需加载的。

希望有帮助。

干杯

Firstly, I would suggest you to raise this question on their forums too.

Secondly, actually, performance of your application clustered over the Terracotta willl depend on number of write transactions that happen. So you could consider using ConcurrentStringMap (if your keys are Strings) or ConcurrentHashMap. Note that CSM is much more better than CHM from point of performance.

After all, POJOs are loaded lazily. That means each property is loaded on-demand.

Hope that helps.

Cheers

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文