当前位置：文江博客话题详情

Perl 内存使用情况以及映射和文件句柄

发布于 2024-11-09 12:07:17 字数 95 浏览 0 评论 0原文

使用 perl 时调用 map { function($_) }; 是否会将整个文件加载到内存中？

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

街道布景 2024-11-16 12:07:17

是的——或者至少我是这么解释这个结果的。

$ perl -e "map {0} <>" big_data_file
Out of memory!

$ perl -e "map {0} 1 .. 1000000000"
Out of memory!

人们可能想知道我们是否耗尽了内存，因为 Perl 正在尝试存储 map 的输出。然而，我的理解是，Perl 进行了优化，以避免在 void 上下文中调用 map 时的工作。有关具体示例，请参阅此问题中的讨论。

也许是一个更好的例子：

$ perl -e "sub nothing {}  map nothing(), <>" big_data_file
Out of memory!

根据评论，这个问题似乎是出于在处理大数据时对紧凑语法的渴望而提出的。

open(my $handle, '<', 'big_data_file') or die $!;

# An ordinary while loop to process a data file.
while (my $line = <$handle>){
    foo($line);
}

# Here Perl assigns each line to $_.
while (<$handle>){
    foo($_);
}

# And here we do the same thing on one line.
foo($_) while <$handle>;

Yes -- or at least that's how I interpret this outcome.

$ perl -e "map {0} <>" big_data_file
Out of memory!

$ perl -e "map {0} 1 .. 1000000000"
Out of memory!

One might wonder whether we run out of memory because Perl is trying to store the output of map. However, my understanding is that Perl is optimized to avoid that work whenever map is called in a void context. For a specific example, see the discussion in this question.

Perhaps a better example:

$ perl -e "sub nothing {}  map nothing(), <>" big_data_file
Out of memory!

Based on the comments, it appears that the question is motivated by a desire for a compact syntax when processing large data.

open(my $handle, '<', 'big_data_file') or die $!;

# An ordinary while loop to process a data file.
while (my $line = <$handle>){
    foo($line);
}

# Here Perl assigns each line to $_.
while (<$handle>){
    foo($_);
}

# And here we do the same thing on one line.
foo($_) while <$handle>;

回复收藏 0 原文

我的鱼塘能养鲲 2024-11-16 12:07:17

是的，map、foreach 循环和 sub 调用的操作数在 map、foreach 循环或 sub 调用甚至开始之前计算。

一个例外：（

for my $i (EXPR_X..EXPR_Y)

有或没有 my $i）被优化为计数循环，类似

my $x = EXPR_X;
my $y = EXPR_Y;
for (my $i = $x; $i <= $y; ++$i)

Perl6 的东西将对惰性列表提供本机支持。

Yes, the operands for map, foreach loop and sub calls are evaluated before map, the foreach loop or the sub call even begins.

One exception:

for my $i (EXPR_X..EXPR_Y)

(with or without my $i) is optimised into a counting loop, something along the lines of

my $x = EXPR_X;
my $y = EXPR_Y;
for (my $i = $x; $i <= $y; ++$i)

Perl6 will have native support for lazy lists.

回复收藏 0 原文

时光暖心i 2024-11-16 12:07:17

我假设您问的问题是这样的：map 函数是否在开始处理之前读取文件，或者是否逐行使用。

让我们快速比较一下处理列表：

while (<FILEHANDLE>) { ... }

这个例子显然是逐行使用的。每次迭代，都会获取 $_ 的新值。

for my $line (<FILEHANDLE>) { ... }

在这种情况下，LIST 在循环开始之前展开。在http://perldoc.perl.org/functions/map.html中有一个引用 map 类似于 foreach 循环，并且我确实相信 LIST 在传递给函数之前会被扩展。

The question you are asking I assume is this: Does the map function slurp the file before it begins processing, or does it use line by line.

Lets do a quick comparison about handling lists:

while (<FILEHANDLE>) { ... }

This case clearly uses line by line. Each iteration, a new value for $_ is fetched.

for my $line (<FILEHANDLE>) { ... }

In this case, the LIST is expanded before the loop starts. In http://perldoc.perl.org/functions/map.html there is a reference to map being similar to a foreach loop, and I do believe that LISTs are expanded before being passed to a function.

回复收藏 0 原文

~没有更多了~