Grep 数据并输出到文件

发布于 12-04 19:29 字数 657 浏览 2 评论 0原文

我正在尝试从日志文件中提取数据并系统地组织它。我有大约 9 个日志文件,每个文件大小约为 100mb。

我想做的是:从每个日志文件中提取多个块,对于提取的每个块,我想创建一个新文件并将提取的数据保存到其中。每个块都有明确的起点和终点。

基本上,我已经取得了一些进展,并且能够提取我需要的数据,但是,我在尝试弄清楚如何为每个匹配的块创建新文件时遇到了困难。

由于环境的限制,我无法使用 Python 或 Perl 等编程语言。所以请原谅混乱的命令。

到目前为止我的命令:

find Logs\ 13Sept/Log_00000000*.log -type f -exec \
sed -n '/LRE Starting chunk/,/LRE Ending chunk/p' {} \; | \
grep -v -A1 -B1 "Starting chunk" > Logs\ 13Sept/Chunks/test.txt

LRE 起始块LRE 结束块 是我的边界。现在我的命令可以工作,但它将所有匹配的块保存到一个文件中(其大小变得过大)。

如何为每场比赛创建一个文件并向其中添加匹配的内容?请记住,每个文件可以容纳多个块,并且不限于每个文件一个块。

I'm attempting to extract data from log files and organise it systematically. I have about 9 log files which are ~100mb each in size.

What I'm trying to do is: Extract multiple chunks from each log file, and for each chunk extracted, I would like to create a new file and save this extracted data to it. Each chunk has a clear start and end point.

Basically, I have made some progress and am able to extract the data I need, however, I've hit a wall in trying to figure out how to create a new file for each matched chunk.

I'm unable to use a programming language like Python or Perl, due to the constraints of my environment. So please excuse the messy command.

My command thus far:

find Logs\ 13Sept/Log_00000000*.log -type f -exec \
sed -n '/LRE Starting chunk/,/LRE Ending chunk/p' {} \; | \
grep -v -A1 -B1 "Starting chunk" > Logs\ 13Sept/Chunks/test.txt

The LRE Starting chunk and LRE Ending chunk are my boundaries. Right now my command works, but it saves all matched chunks to one file (whose size is becoming exessive).

How do I go about creating a new file for each match and add the matched content to it? keeping in mind that each file could hold multiple chunks and is not limited to one chunk per file.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

从此见与不见2024-12-11 19:29:08

可能需要比 sed 更可编程的东西:我假设 awk 可用。

awk '
    /LRE Ending chunk/ {printing = 0}
    printing {print > "chunk" n ".txt"}
    /LRE Starting chunk/ {printing = 1; n++}
' *.log

Probably need something more programmable than sed: I'm assuming awk is available.

awk '
    /LRE Ending chunk/ {printing = 0}
    printing {print > "chunk" n ".txt"}
    /LRE Starting chunk/ {printing = 1; n++}
' *.log
淡淡の花香2024-12-11 19:29:08

尝试这样的操作:

find Logs\ 13Sept/Log_00000000*.log -type f -print | while read file; do \
sed -n '/LRE Starting chunk/,/LRE Ending chunk/p' "$file" | \
grep -v -A1 -B1 "Starting chunk" > "Logs 13Sept/Chunks/$file.chunk.txt";
done

这会循环查找结果并针对每个文件执行,然后为每个文件创建一个 $file.chunk.txt。

Try something like this:

find Logs\ 13Sept/Log_00000000*.log -type f -print | while read file; do \
sed -n '/LRE Starting chunk/,/LRE Ending chunk/p' "$file" | \
grep -v -A1 -B1 "Starting chunk" > "Logs 13Sept/Chunks/$file.chunk.txt";
done

This loops over the find results and executes for each file and then create one $file.chunk.txt for each of the files.

轻拂→两袖风尘2024-12-11 19:29:08

也许是这样的?

find Logs\ 13Sept/Log_00000000*.log -type f -exec \
sed -n '/LRE Starting chunk/,/LRE Ending chunk/{;/LRE .*ing chunk/d;w\
'"{}.chunk"';}' {} \;

这使用 sed 的 w 命令写入名为 (inputfile).chunk 的文件。如果这是不可接受的,也许您可​​以使用 sh -c '...' 传入一个小 shell 脚本来包装 sed 命令。 (或者 shell 脚本是否也因某种原因被禁止?)

Something like this perhaps?

find Logs\ 13Sept/Log_00000000*.log -type f -exec \
sed -n '/LRE Starting chunk/,/LRE Ending chunk/{;/LRE .*ing chunk/d;w\
'"{}.chunk"';}' {} \;

This uses sed's w command to write to a file named (inputfile).chunk. If that is not acceptable, perhaps you can use sh -c '...' to pass in a small shell script to wrap the sed command with. (Or is a shell script also prohibited for some reason?)

留蓝2024-12-11 19:29:08

也许您可以使用 csplit 进行分割,然后在块末尾截断输出文件。

Perhaps you could use csplit to do the splitting, then truncate the output files at the chunk end.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文