在处理其他任务时,如何使用OpenMP创建新任务?
我已经四处寻找几天了,我找不到解决问题的解释(因此是解决方案)。
我有一个数百万行的文件,我想独立处理。每行代表我称之为函数的对象,其执行时间取决于对象本身的某些属性。执行时间无法平衡,在某些情况下或几分钟或几个小时内可能需要不到一秒钟。
我用段循环解析文件并使用任务:
ifstream fin(FileName);
#pragma omp parallel num_threads(N)
{
#pragma omp single
{
string line;
#pragma omp task untied
while (getline(fin, line)) {
#pragma omp task firstprivate(line)
function(line);
}
}
}
我观察到的是有时会悬挂一些线程,例如任务池是空的,他们等待创建新任务。我注意到我达到了悬挂一个线程的所有点,并且一旦该线程执行其任务,它们都会重新启动运行。在我的第一个版本中,我没有使用未绑定的子句,我认为问题可能是正在读取文件的线程(例如线程0)正在忙于执行长任务本身,并且无法继续阅读文件并创建新的新任务任务。但是,添加未键的子句(以便任何其他线程可以继续读取文件)无法解决问题。
我有一些问题:
如果任务数量比池大小大得多(我知道是64*number_of_threads),则该线程0是否暂停其当前任务(即读取文件并生成新任务)并帮助其他线程执行任务?我通常会读到,当它结束时,它可以帮助其他线程创建任务(例如:如何做事 - single-and-pask-task-provide-parallealisl ),但我的印象不是我的情况。
未划分的子句在做我期望的吗?也就是说,如果线程0忙于执行任务,那么另一个线程可以继续读取文件并填充任务池吗?如果这是真的,为什么我会看到没有做任何事情的线程?
是否有一个障碍可以阻止任何线程继续读取文件并创建新任务,直到池为空?我希望一旦线程看到任务池是空的,它就会填充新任务。即使线程专门用于在池中通过另一个线程从池中挑选任务后,我也会很高兴。
。
感谢您的建议!
I've been looking around for days now, and I couldn't find the explanation (and thus a solution) to my problem.
I have a file of millions of lines that I want to process independently. Each line represents an object on which I call a function, whose execution time depends on some properties of the object itself. The execution time is not balanced, it can take less than a second in some cases, or minutes or hours.
I parse the file with a while loop and use tasks:
ifstream fin(FileName);
#pragma omp parallel num_threads(N)
{
#pragma omp single
{
string line;
#pragma omp task untied
while (getline(fin, line)) {
#pragma omp task firstprivate(line)
function(line);
}
}
}
What I observe is that sometimes some threads are suspended, like if the task pool is empty and they wait for the creation of new tasks. I noticed that I reach the point where all but one thread are suspended, and as soon as this thread ends performing its task, they all restart running. In my first version I wasn't using the untied clause and I thought the problem could be that the thread that is reading the file (say thread 0) is busy executing a long task itself and cannot go on with reading the file and creating new tasks. But, adding the untied clause (so that any other thread can continue reading the file) doesn't solve the issue.
I have some questions:
If the number of tasks is much bigger than the pool size (which I understood is 64*number_of_threads), does the thread 0 suspend its current task (i.e. reading the file and generating new tasks) and help the other threads execute tasks? I usually read that it helps other threads when it ends creating tasks (like here: how-do-omp-single-and-omp-task-provide-parallelism), but I have the impression it's not my case.
Is the untied clause doing what I expect? That is, if the thread 0 is busy in executing a task, then another thread can continue reading the file and fill the pool of tasks? If this is true, why I see threads that are not doing anything?
Is there a barrier that is preventing any thread to go on reading the file and create new tasks until the pool is empty? I would like that as soon as a thread sees that the task pool is empty, it fills it with new tasks. I would be happy even if a thread is dedicated only to read a new line as soon as a task is picked from the pool by another thread.
Thanks for any suggestion!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
删除
while
之前的任务。一种典型的习惯用法是,一个线程(并行/单线程)生成所有任务,其他线程将它们从内部任务队列中取出。所以你的 while 循环应该由线程执行,而不是在任务中执行。然后,它将生成与您的行相对应的任务,供其他线程执行。Remove the task before the
while
. A typical idiom is that one thread (from parallel/single) generates all the tasks, and the others take them off the internal task queue. So your while loop should be executed by a thread, not in a task. It will then generate the tasks, corresponding to your lines, for the other threads to execute.