并发应用程序不如单线程快

发布于 2024-12-03 21:42:04 字数 2990 浏览 0 评论 0原文

我已经实施了管道方法。我要遍历一棵树,我需要某些事先不可用的值...所以我必须并行(或之前)遍历树,并且对于我想要保存值的每个节点再次遍历(例如,descendantCount) )。

因此,我通过树进行交互,然后从构造函数调用一个方法,该方法调用通过 ExecutorService 启动的新线程。提交的 Callable 是:

    @Override
    public Void call() throws Exception {
        // Get descendants for every node and save it to a list.
        final ExecutorService executor =
            Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
        int index = 0;
        final Map<Integer, Diff> diffs = mDiffDatabase.getMap();
        final int depth = diffs.get(0).getDepth().getNewDepth();
        try {
            boolean first = true;
            for (final AbsAxis axis = new DescendantAxis(mNewRtx, true); index < diffs.size()
                && ((diffs.get(index).getDiff() == EDiff.DELETED && depth < diffs.get(index).getDepth()
                    .getOldDepth()) || axis.hasNext());) {
                if (axis.getTransaction().getNode().getKind() == ENodes.ROOT_KIND) {
                    axis.next();
                } else {
                    if (index < diffs.size() && diffs.get(index).getDiff() != EDiff.DELETED) {
                        axis.next();
                    }

                    final Future<Integer> submittedDescendants =
                        executor.submit(new Descendants(mNewRtx.getRevisionNumber(), mOldRtx
                            .getRevisionNumber(), axis.getTransaction().getNode().getNodeKey(), mDb
                            .getSession(), index, diffs));
                    final Future<Modification> submittedModifications =
                        executor.submit(new Modifications(mNewRtx.getRevisionNumber(), mOldRtx
                            .getRevisionNumber(), axis.getTransaction().getNode().getNodeKey(), mDb
                            .getSession(), index, diffs));
                    if (first) {
                        first = false;
                        mMaxDescendantCount = submittedDescendants.get();
                        // submittedModifications.get();
                    }
                    mDescendantsQueue.put(submittedDescendants);
                    mModificationQueue.put(submittedModifications);
                    index++;
                }
            }

            mNewRtx.close();
        } catch (final AbsTTException e) {
            LOGWRAPPER.error(e.getMessage(), e);
        }
        executor.shutdown();
        return null;
    }

因此,对于每个节点,它都会创建一个新的 Callable,它遍历每个节点的树并计算后代和修改(我实际上将两个树修订融合在一起)。嗯,mDescendantsQueue 和 mModificationQueue 是 BlockingQueue。起初,我只有 DescendantsQueue 并再次遍历树以获取每个节点的修改(计算在当前节点的子树中所做的修改)。然后我想为什么不并行执行两者并实现流水线方法。遗憾的是,每次我实现另一个多线程“步骤”时,性能似乎都会下降。

也许是因为 XML 树通常不是那么深,并且并发开销太重:-/

起初我按顺序执行所有操作,这是最快的: - 遍历树 - 对于每个节点遍历后代并计算后代计数和修改计数

在使用带有 BlockingQueues 的管道方法后,性能似乎有所下降,但我实际上没有进行任何时间测量,我必须恢复许多更改才能返回:(也许CPU 数量越多,性能就会越高,因为我现在只有一个 Core2Duo 可供测试,


约翰内斯

I've implemented a pipeline approach. I'm going to traverse a tree and I need certain values which aren't available beforehand... so I have to traverse the tree in parallel (or before) and once more for every node I want to save values (descendantCount for example).

As such I'm interating through the tree, then from the constructor I'm calling a method which invokes a new Thread started through an ExecutorService. The Callable which is submitted is:

    @Override
    public Void call() throws Exception {
        // Get descendants for every node and save it to a list.
        final ExecutorService executor =
            Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
        int index = 0;
        final Map<Integer, Diff> diffs = mDiffDatabase.getMap();
        final int depth = diffs.get(0).getDepth().getNewDepth();
        try {
            boolean first = true;
            for (final AbsAxis axis = new DescendantAxis(mNewRtx, true); index < diffs.size()
                && ((diffs.get(index).getDiff() == EDiff.DELETED && depth < diffs.get(index).getDepth()
                    .getOldDepth()) || axis.hasNext());) {
                if (axis.getTransaction().getNode().getKind() == ENodes.ROOT_KIND) {
                    axis.next();
                } else {
                    if (index < diffs.size() && diffs.get(index).getDiff() != EDiff.DELETED) {
                        axis.next();
                    }

                    final Future<Integer> submittedDescendants =
                        executor.submit(new Descendants(mNewRtx.getRevisionNumber(), mOldRtx
                            .getRevisionNumber(), axis.getTransaction().getNode().getNodeKey(), mDb
                            .getSession(), index, diffs));
                    final Future<Modification> submittedModifications =
                        executor.submit(new Modifications(mNewRtx.getRevisionNumber(), mOldRtx
                            .getRevisionNumber(), axis.getTransaction().getNode().getNodeKey(), mDb
                            .getSession(), index, diffs));
                    if (first) {
                        first = false;
                        mMaxDescendantCount = submittedDescendants.get();
                        // submittedModifications.get();
                    }
                    mDescendantsQueue.put(submittedDescendants);
                    mModificationQueue.put(submittedModifications);
                    index++;
                }
            }

            mNewRtx.close();
        } catch (final AbsTTException e) {
            LOGWRAPPER.error(e.getMessage(), e);
        }
        executor.shutdown();
        return null;
    }

Therefore for every node it's creating a new Callable which traverses the tree for every node and counts descendants and modifications (I'm actually fusing two tree-revisions together). Well, mDescendantsQueue and mModificationQueue are BlockingQueues. At first I've only had the descendantsQueue and traversed the tree once more to get modifications of every node (counting modifications made in the subtree of the current node). Then I thought why not do both in parallel and implement a pipelined approach. Sadly the performance seemed to have decreased everytime I've implemented another multithreaded "step".

Maybe because an XML-tree usually isn't that deep and the Concurrency-Overhead is too heavy :-/

At first I did everything sequential, which was the fastest:
- traversing the tree
- for every node traverse the descendants and compute descendantCount and modificationCount

After using a pipelined approach with BlockingQueues it seems the performance has decreased, but I haven't actually made any time measures and I would have to revert many changes to go back :( Maybe the performance increases with more CPUs, because I only have a Core2Duo for testing right now.

best regards,
Johannes

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

深居我梦 2024-12-10 21:42:04

也许这应该有所帮助:阿玛达定律,它基本上说生产力的提高取决于(成反比)必须通过同步处理的代码的百分比。因此,即使通过增加更多的计算资源来增加,最终也不会得到更好的结果。理想情况下,如果(同步部分与总部分)的比率较低,则使用(处理器数量+1)应该给出最佳输出(除非您使用网络或其他 I/O,在这种情况下您可以增加大小池的)。
因此,只需从上面的链接进行操作,看看是否有帮助

Probably this should help: Amadahl's law, what it basically says it that the increase in productivity depends (inversely proportional) to the percentage of the code which has to be processed by synchronization. Hence even by increasing by increasing more computing resources, it wont end up to the better result. Ideally if the ratio of ( the synchronized part to the total part) is low, then with (number of processors +1) should give the best output (unless you are using network or other I/O in which case you can increase the size of the pool).
So just follow it up from the above link and see if it helps

北风几吹夏 2024-12-10 21:42:04

从您的描述来看,您似乎正在递归地创建线程,每个线程处理一个节点,然后生成一个新线程?这是正确的吗?如果是这样,我对您遭受性能下降的困扰并不感到惊讶。

简单的递归下降方法实际上可能是做到这一点的最佳方法。我看不出多线程将如何为您带来任何优势。

From your description it sounds like you're recursively creating threads, each of which processes one node and then spawns a new thread? Is this correct? If so, I'm not surprised that you're suffering from performance degradation.

A simple recursive descent method might actually be the best way to do this. I can't see how multithreading will gain you any advantages here.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文