LWP::UserAgent 不是线程安全的吗?
我正在使用以下子例程运行 40 个左右的线程:
my $app = shift;
my $ua = LWP::UserAgent->new();
$ua->timeout(5);
my $response = $ua->get($$app{'watch_url'});
my $new_md5;
if ($response->is_success()) {
$new_md5 = md5_hex($response->content());
}
return ($$app{'short_name'}, $$app{'watch_md5'}, $new_md5);
大约 3/4 的时间会发生核心转储。 LWP 和 LWP::UserAgent 是纯 Perl 的,所以我对此感到措手不及。 LWP::UserAgent 不是线程安全的吗?
更新:
这是重现该问题的最小版本:
use strict;
use warnings;
use threads;
use LWP::UserAgent;
sub check_app {
my $ua = LWP::UserAgent->new();
$ua->timeout(5);
$ua->get('http://www.flatdoc.com/?' . rand(10));
}
my @threads;
for (my $i = 0; $i < 40; $i++) {
my $thread = threads->create(\&check_app);
push(@threads, $thread);
}
foreach (@threads) {
$_->join();
}
I'm running 40-or-so threads with the following subroutine:
my $app = shift;
my $ua = LWP::UserAgent->new();
$ua->timeout(5);
my $response = $ua->get($app{'watch_url'});
my $new_md5;
if ($response->is_success()) {
$new_md5 = md5_hex($response->content());
}
return ($app{'short_name'}, $app{'watch_md5'}, $new_md5);
Core dumps ensue about 3/4 of the time. LWP and LWP::UserAgent are pure Perl, so I'm caught off-guard by this. Is LWP::UserAgent not thread-safe?
Update:
Here's a minimal version to reproduce the issue:
use strict;
use warnings;
use threads;
use LWP::UserAgent;
sub check_app {
my $ua = LWP::UserAgent->new();
$ua->timeout(5);
$ua->get('http://www.flatdoc.com/?' . rand(10));
}
my @threads;
for (my $i = 0; $i < 40; $i++) {
my $thread = threads->create(\&check_app);
push(@threads, $thread);
}
foreach (@threads) {
$_->join();
}
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
非线程安全的纯 Perl 代码不会导致段错误(事实上,任何纯 Perl 代码都不应该导致段错误)。 Perl 中的错误会导致段错误。 Perl 中的线程历来都是有很多 bug 的,但现在已经好多了。
您的代码在 5.10.1 中运行良好,并且 HTTP::Lite 可能不会引起您遇到的任何 Perl 错误。可能您只需要使用较新版本的 Perl。你越老、越接近 Redhat,线程就越不稳定。如果您要使用线程,请使用您可以使用的最新 Perl。
作为线程的替代方案,您可以使用类似 Parallel::ForkManager、LWP::Parallel 甚至令人惊叹的 forks 模块,使用 fork 模拟线程。
Non-thread safe pure-Perl code does not cause a segfault (in fact, no pure Perl code should cause a segfault). A bug in Perl causes a segfault. And threads in Perl are historically very buggy, but they've gotten a lot better.
Your code runs fine in 5.10.1, and HTTP::Lite probably just doesn't tickle whatever perl bug you've run into. Likely you just need to use a newer version of Perl. The older and closer to Redhat you get, the less stable threads are. If you're going to use threads, use the newest Perl you can get your hands on.
As an alternative to threads, you can use something like Parallel::ForkManager, LWP::Parallel or even the amazing forks module which emulates threads using fork.