bash 脚本的奇怪行为

发布于 2024-11-08 12:38:47 字数 1471 浏览 0 评论 0原文

这是一个片段：

var=`ls | shuf | head -2 | xargs cat | sed -e 's/\(.\)/\1\n/g' | shuf | tr -d '\n'`

这将从当前目录中选择两个随机文件，组合它们的内容，对它们进行洗牌，并将结果分配给 var。这在大多数情况下都可以正常工作，但大约千分之一的情况下，只有 ls 的输出绑定到 var （这不仅仅是输出，请参阅编辑 II）。可能是什么解释？

一些更可能相关的事实：

该目录至少包含两个文件目录
中只有文本文件文件
名不包含空格
文件的长度从 5 到大约 1000 个字符
片段是更大脚本的一部分它以并行 bash 版本运行两个实例
：GNU bash，版本 4.1.5(1)-release (i686-pc-linux-gnu)
uname: Linux 2.6.35-28-generic -pae #50-Ubuntu

编辑： 我单独运行了该代码片段几千次，没有任何错误。然后我尝试将它与整个脚本的各个其他部分一起运行。下面是一个会产生错误的配置：

cd dir_with_text_files
var=`ls | shuf | head -2 | xargs cat | sed -e 's/\(.\)/\1\n/g' | shuf | tr -d '\n'`
cd ..

cd 之间有数百行脚本，但这是重现错误的最小配置。请注意，异常输出绑定到 var 当前目录的输出，而不是 dir_with_text_files。

编辑二：我一直在更详细地查看输出。 ls 输出不会单独出现，它与两个打乱的文件一起出现（在它们的内容之间，或者在它们之后或之前，完好无损）。但情况会变得更好；让我搭台来讨论特定的目录。

[~/projects/upload] ls -1
checked // dir
lines   // dir, the files to shuffle are here
pages   // also dir
proxycheck
singlepost
uploader
indexrefresh
t
tester

到目前为止，我已经看到 ls 的输出从 upload 运行，但现在我看到了 ls */* 的输出（也运行来自上传）。它的形式为“someMangledText ls moreMangledText ls */* FinalBatchOfText”。毫无疑问生成的序列 ls 是否有可能以某种方式被执行？

原文

Here's a snippet:

var=`ls | shuf | head -2 | xargs cat | sed -e 's/\(.\)/\1\n/g' | shuf | tr -d '\n'`

This will select two random files from the current directory, combine their contents, shuffle them, and assign the result to var. This works fine most of the time, but about once in a thousand cases, instead just the output of ls is bound to var (It's not just the output, see EDIT II). What could be the explanation?

Some more potentially relevant facts:

the directory contains at least two files
there are only text files in the directory
file names don't contain spaces
the files are anywhere from 5 to about 1000 characters in length
the snippet is a part of a larger script that it ran two instances in parallel
bash version: GNU bash, version 4.1.5(1)-release (i686-pc-linux-gnu)
uname: Linux 2.6.35-28-generic-pae #50-Ubuntu

EDIT: I ran the snippet by itself a couple of thousand times with no errors. Then I tried running it with various other parts of the whole script. Here's a configuration that produces errors:

cd dir_with_text_files
var=`ls | shuf | head -2 | xargs cat | sed -e 's/\(.\)/\1\n/g' | shuf | tr -d '\n'`
cd ..

There are several hundred lines of the script between the cds, but this is the minimal configuration to reproduce the error. Note that the anomalous output binds to var the output of the current directory, not dir_with_text_files.

EDIT II: I've been looking at the outputs in more detail. The ls output doesn't appear alone, it's along with with two shuffled files (between their contents, or after or before them, intact). But it gets better; let me set up the stage to talk about particular directories.

[~/projects/upload] ls -1
checked // dir
lines   // dir, the files to shuffle are here
pages   // also dir
proxycheck
singlepost
uploader
indexrefresh
t
tester

So far, I've seen the output of ls ran from upload, but now I saw the output of ls */* (also ran from upload). It was in the form of "someMangledText ls moreMangledText ls */* finalBatchOfText". Is it possible that the sequence ls that undoubtedly was generated was somehow executed?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

白况 2024-11-15 12:38:47

这里也没有问题。
我还将上面的内容重写为：

sed 's:\(.\):\1\n:g' < <(shuf -e * | head -2 | xargs cat) | shuf | tr -d '\n'

不要使用 ls 列出目录的内容，使用 *。
此外，进行一些调试。使用 shebang 后跟：

set -e
set -o pipefail

并像这样运行脚本：

/bin/bash -x /path/to/script

并检查输出。
您不必调试整个脚本，只需用 -x 包围似乎有问题的部分，

set -x
...code that may have problems...
set +x

以便输出集中于代码的该部分。
另外，请使用pipefail选项。

一些定义：
-e ：如果简单命令以非零状态退出，则立即退出，除非失败的命令是紧随 while 或 Until 关键字（测试的一部分）的命令列表的一部分在 if 语句中， && 的一部分或||列表，或者如果使用 ! 反转命令的返回状态。如果设置了 ERR 陷阱，则会在 shell 退出之前执行
-x ：在扩展后和执行前打印简单命令的跟踪，包括命令、case 命令、选择命令以及命令及其参数或关联单词列表的算术。 PS4 变量的值被扩展，并且结果值在命令及其扩展参数之前打印
pipefail ：如果设置，管道的返回值是最后一个（最右边）以非零状态退出的命令的值，如果管道中的所有命令都成功退出，则返回零

No problems here either.
I would also rewrite the above to this:

sed 's:\(.\):\1\n:g' < <(shuf -e * | head -2 | xargs cat) | shuf | tr -d '\n'

Do not use ls to list a directory's content, use *.
Moreover, do some debugging. Use a shebang followed by:

set -e
set -o pipefail

and run the script like this:

/bin/bash -x /path/to/script

and do inspect the output.
Instead of debugging the whole script, you can surround just the part that seems to be problematic with -x

set -x
...code that may have problems...
set +x

so that the output focuses on that part of the code.
Also, use the pipefail option.

Some definitions:
-e : Exit immediately if a simple command exits with a non-zero status, unless the command that fails is part of the command list immediately following a while or until keyword, part of the test in an if statement, part of a && or || list, or if the command's return status is being inverted using !. A trap on ERR, if set, is executed before the shell exits
-x : Print a trace of simple commands, for commands, case commands, select commands, and arithmetic for commands and their arguments or associated word lists after they are expanded and before they are executed. The value of the PS4 variable is expanded and the resultant value is printed before the command and its expanded arguments
pipefail : If set, the return value of a pipeline is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands in the pipeline exit successfully

回复收藏 0 原文

笑看君怀她人 2024-11-15 12:38:47

出于调试目的，您还可以使用 env -i 清除环境并过滤掉不可打印的字符：

#!/usr/bin/env -i /bin/bash --

set -ef
set -o pipefail

unset IFS PATH LC_ALL
IFS=
在运行脚本之前，您还可以禁用 GNU readline 库和 !风格历史扩展：
bash --noediting
set +H

 \t\n'
PATH="$(PATH=/bin:/usr/bin getconf PATH)"
LC_ALL=C
export IFS PATH LC_ALL

#var="$((find . -type f -maxdepth 1 -print0 | shuf -z -n 2 | xargs -0 cat) | sed -e 's/\(.\)/\1\n/g' | shuf | tr -d '\n')"

var="$((find . -type f -maxdepth 1 -print0 | shuf -z -n 2 | xargs -0 cat) | tr -cd '[[:print:]]' | grep -o '.' | shuf | tr -d '\n')"

在运行脚本之前，您还可以禁用 GNU readline 库和 !风格历史扩展：

For debugging purposes you may also clear the environment using env -i and filter out non-printable characters:

#!/usr/bin/env -i /bin/bash --

set -ef
set -o pipefail

unset IFS PATH LC_ALL
IFS=
Before running the script you may also disable the GNU readline library and ! style history expansion:
bash --noediting
set +H

 \t\n'
PATH="$(PATH=/bin:/usr/bin getconf PATH)"
LC_ALL=C
export IFS PATH LC_ALL

#var="$((find . -type f -maxdepth 1 -print0 | shuf -z -n 2 | xargs -0 cat) | sed -e 's/\(.\)/\1\n/g' | shuf | tr -d '\n')"

var="$((find . -type f -maxdepth 1 -print0 | shuf -z -n 2 | xargs -0 cat) | tr -cd '[[:print:]]' | grep -o '.' | shuf | tr -d '\n')"

Before running the script you may also disable the GNU readline library and ! style history expansion:

回复收藏 0 原文