多线程——匹配实例
我想在数据库的两个修订版上同时运行两个 XPath 表达式,它们都从迭代器/可迭代器返回结果,并将结果节点与列表中的节点相匹配。
我认为最好的办法是在 executorservice 的两个线程中运行两个查询,并将两个线程的结果保存在 BlockingQueue 中,而另一个线程将对 BlockingQueue 中的结果进行排序code> 或者实际上将传入的节点或nodeKeys保存在正确的位置。
然后获得结果排序列表和另一个排序列表的交集就很简单了。
还有其他建议吗?我还可以自由地使用我喜欢的任何技术(最好是 Java)。 Guava 在类路径中,但我已经考虑过使用 Akka 中的 Actor。
编辑:另一个相关问题是,以管道方式使用 InsertionSort(在收到生成的 XPath 结果时对其进行处理)更快,还是等到整个结果生成并使用 QuickSort 或 MergeSort 更快。我认为无论结果元素的数量如何,插入排序应该是更可取的。
一般来说,我希望排序然后计算两个列表的交集比搜索 XPath 结果列表中的每个项目的 O(n^2)
更快,即使列表除以数字可用的 CPU 处理器数。
编辑: 我目前已经实现了第一部分:
final ExecutorService executor = Executors.newFixedThreadPool(2);
final AbsTemporalAxis axis =
new NextRevisionAxis.Builder(mSession).setRevision(mRevision)
.setIncludeSelf(EIncludeSelf.YES).build();
for (final IReadTransaction rtx : axis) {
final ListenableFuture<Void> future =
Futures.makeListenable(executor.submit(new XPathEvaluation(rtx, mQuery)));
future.addListener(new Runnable() {
@Override
public void run() {
try {
mSemaphore.acquire();
} catch (final InterruptedException e) {
LOGWRAPPER.error(e.getMessage(), e);
}
}
}, executor);
}
executor.shutdown();
final ExecutorService sameThreadExecutor = MoreExecutors.sameThreadExecutor();
sameThreadExecutor.submit(new XPathResult());
sameThreadExecutor.shutdown();
return null;
信号量初始化为 2,并在 XPathEvaluation 中将生成的 nodeKey 添加到 LinkedBlockingQueue 中。
然后我将对用注释表示的 XPathResults 进行排序,该注释尚未实现:
private final class XPathResult implements Callable<Void> {
@Override
public Void call() throws AbsTTException, InterruptedException {
while (true) {
final long key = mQueue.take();
if (key == -1L) {
break;
}
if (mSemaphore.availablePermits() == 0) {
mQueue.put(-1L);
}
// Do InsertionSort.
}
return null;
}
}
没有任何 JavaDoc,但我认为至少它应该可以工作,您认为呢?您有更好的解决方案吗?还是我到目前为止犯了一些错误?
亲切的问候,
约翰内斯
I want to run two XPath-Expressions concurrently on two revisions of a database which both return results from an Iterator/Iterable and match resulting nodes with nodes in a List.
I think the best thing is to run both queries in two threads from an executorservice and save results from both threads in a BlockingQueue
, whereas another Thread is going to sort the results from the BlockingQueue
or actually saves the incoming nodes or nodeKeys in the right position.
Then it's trivial to get the intersection of the resulting sorted List and another sorted List.
Any other suggestions? I'm also free to use whatever technology I like (preferably Java). Guava is in the classpath, but I already thought about using Actors from Akka.
Edit: An additional related question would be if it's faster to use InsertionSort in a pipeline manner (to process the generated XPath results right when they are received) or to wait until the whole result has been generated and use QuickSort or MergeSort. I think InsertionSort should be preferable regardless of the resulting number of elements.
In general I hope sorting and then computing the intersection of two lists is faster than O(n^2)
for the search of each item in the XPath result list, even if the list is divided by the number of CPU processors available.
Edit:
I've currently implemented the first part:
final ExecutorService executor = Executors.newFixedThreadPool(2);
final AbsTemporalAxis axis =
new NextRevisionAxis.Builder(mSession).setRevision(mRevision)
.setIncludeSelf(EIncludeSelf.YES).build();
for (final IReadTransaction rtx : axis) {
final ListenableFuture<Void> future =
Futures.makeListenable(executor.submit(new XPathEvaluation(rtx, mQuery)));
future.addListener(new Runnable() {
@Override
public void run() {
try {
mSemaphore.acquire();
} catch (final InterruptedException e) {
LOGWRAPPER.error(e.getMessage(), e);
}
}
}, executor);
}
executor.shutdown();
final ExecutorService sameThreadExecutor = MoreExecutors.sameThreadExecutor();
sameThreadExecutor.submit(new XPathResult());
sameThreadExecutor.shutdown();
return null;
The semaphore is initialized to 2 and in XPathEvaluation
the resulting nodeKeys are added to a LinkedBlockingQueue
.
Then I'm going to sort the XPathResults denoted with the comment, which isn't implemented yet:
private final class XPathResult implements Callable<Void> {
@Override
public Void call() throws AbsTTException, InterruptedException {
while (true) {
final long key = mQueue.take();
if (key == -1L) {
break;
}
if (mSemaphore.availablePermits() == 0) {
mQueue.put(-1L);
}
// Do InsertionSort.
}
return null;
}
}
Without any JavaDoc, but I think at least it should work, what do you think? Do you have any preferable solutions or do I have made some mistakes so far?
kind regards,
Johannes
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
您确定需要同时执行此操作吗?您不能连续构建两个列表,然后执行排序/相交吗? - 这将需要主题变得很多复杂性。
我认为在两个列表都完全填满之前无法进行相交,对吗?然后,不需要队列或同步,只需填充两个列表/集合,完成后处理两个完整列表。
但也许我不太明白你的意思......
Are you sure you need to do this concurrently? Can't you just build the two lists consecutively and after that perform your sorting/intersecting? - That would take a lot of complexity from the subject.
I assume that intersecting cannot be done until both lists are filled completely, am I correct? Then, no queue or synchronization would be needed, just fill two lists/sets and, once done, process both full lists.
But maybe I'm not quite getting your point...