如何在连接之前插入新行?

发布于 2024-11-26 23:49:54 字数 279 浏览 2 评论 0原文

我有大约 80000 个文件正在尝试连接。这一个:

cat files_*.raw >> All

非常快,而以下:

for f in `ls files_*.raw`; do cat $f >> All; done;

非常慢。由于这个原因,我试图坚持使用第一个选项,除了我需要能够在每个文件连接到 All 之后插入一个新行。有什么快速的方法可以做到这一点吗?

I have about 80000 files which I am trying to concatenate. This one:

cat files_*.raw >> All

is extremely fast whereas the following:

for f in `ls files_*.raw`; do cat $f >> All; done;

is extremely slow. Because of this reason, I am trying to stick with the first option except that I need to be able to insert a new line after each file is concatenated to All. Is there any fast way of doing this?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

心如荒岛 2024-12-03 23:49:54

ls files_*.raw | xargs -L1 sed -e '$s/$/\n/' >>ALL

会在连接文件时在每个文件的末尾插入一个额外的换行符。

如果您不关心串联顺序,则可以使用并行版本:

find ./ -name "*.raw" -print | xargs -n1 -P4 sed -e '$s/$/\n/' >>All

What about

ls files_*.raw | xargs -L1 sed -e '$s/$/\n/' >>ALL

That will insert an extra newline at the end of each file as you concat them.

And a parallel version if you don't care about the order of concatenation:

find ./ -name "*.raw" -print | xargs -n1 -P4 sed -e '$s/$/\n/' >>All
撩心不撩汉 2024-12-03 23:49:54

第二个命令可能会很慢,因为您要打开“全部”文件进行追加 80000 次,而第一个命令中则为 1 次。尝试第二个命令的简单变体:

for f in `ls files_*.raw`; do cat $f ; echo '' ; done >> All

The second command might be slow because you are opening the 'All' file for append 80000 times vs. 1 time in the first command. Try a simple variant of the second command:

for f in `ls files_*.raw`; do cat $f ; echo '' ; done >> All
以歌曲疗慰 2024-12-03 23:49:54

我不知道为什么会很慢,但我认为你没有太多选择:

for f in `ls files_*.raw`; do cat $f >> All; echo '' >> All; done

I don't know why it would be slow, but I don't think you have much choice:

for f in `ls files_*.raw`; do cat $f >> All; echo '' >> All; done
忘东忘西忘不掉你 2024-12-03 23:49:54

每次 awk 打开另一个文件进行处理时,FRN 等于 0,因此:

awk '(0==FRN){print ""} {print}' files_*.raw >> All

注意,这一切都是在一个 awk 进程中完成的。性能应该接近问题中的 cat 命令。

Each time awk opens another file to process, the FRN equals 0, so:

awk '(0==FRN){print ""} {print}' files_*.raw >> All

Note, it's all done in one awk process. Performance should be close to the cat command from the question.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文