当前位置： 文江博客话题详情

如何按日期对多个日志文件的输出进行排序

发布于 2024-09-11 07:48:08 字数 1013 浏览 3 评论 0原文

我从几个不同的日志文件中获得了输出：

logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx

logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx

logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx

我想按日期对此输出进行排序，但将日志文件的名称保留在日志行上方，因此它应该如下所示：

logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx

logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx

logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx

Do you haveany idea how to sort out such with bash 命令、sed 还是 awk？ 多谢！

更新： 的来源

for i in $( find log/ -iname *debug*.log -size +0 );do
if [ `grep -c 'ERROR' $i` -gt 0 ];then
 echo -e "\n$i"
 grep 'ERROR' --color=auto -A 5 -B 5 $i
fi
done

这是输出Martin

原文

I have got output from several different log files:

logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx

logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx

logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx

I would like to sort this output by date, but keep the name of the logfile above the log lines, so it should look like:

logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx

logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx

logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx

Do you have any idea how to sort output like this with bash commands, sed or awk?
Thanks a lot!

UPDATE:
This is the source of the output

for i in $( find log/ -iname *debug*.log -size +0 );do
if [ `grep -c 'ERROR' $i` -gt 0 ];then
 echo -e "\n$i"
 grep 'ERROR' --color=auto -A 5 -B 5 $i
fi
done

Martin

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（6）

思念满溢 2024-09-18 07:48:08

您也许能够从中获得满意的结果（只要您的文件名都不包含冒号）：

grep -C 5 --recursive 'ERROR' log/* | sort --field-separator=: --key=2

每一行都会以文件名开头。您的输出将如下所示：

logfile2:2010/07/21 13:28:52 INFO xxx
logfile2:2010/07/21 13:31:25 INFO xxx
logfile2:2010/07/21 13:31:25 DEBUG xxx

logfile3:2010/07/21 15:28:52 INFO xxx
logfile3:2010/07/21 15:31:25 INFO xxx
logfile3:2010/07/21 15:31:25 DEBUG xxx
etc.

您可以使用 AWK 将其重新格式化为示例中显示的格式：

grep -C 5 --recursive 'ERROR' log/* | sort --field-separator=: --key=2 |
    awk '{colon = match($0,":"); file = substr($0,1,colon - 1); 
    if (file != prevfile) {print "\n" file; prevfile = file}; 
    print substr($0,colon+1)}'

以下是对脚本的几项改进，以防您仍然使用它：

find log/ -iname "*debug*.log" -size +0 | while read -r file
do
    if grep -qsm 1 'ERROR' "$file"
    then
        echo -e "\n$file"
        grep 'ERROR' --color=auto -C 5 "$file"
    fi
done

You may be able to get satisfactory results from this (as long as none of your filenames contain colons):

grep -C 5 --recursive 'ERROR' log/* | sort --field-separator=: --key=2

Each line will be prepended by the filename. Your output will look something like this:

logfile2:2010/07/21 13:28:52 INFO xxx
logfile2:2010/07/21 13:31:25 INFO xxx
logfile2:2010/07/21 13:31:25 DEBUG xxx

logfile3:2010/07/21 15:28:52 INFO xxx
logfile3:2010/07/21 15:31:25 INFO xxx
logfile3:2010/07/21 15:31:25 DEBUG xxx
etc.

You can use AWK to reformat that into the format that you show in your example:

grep -C 5 --recursive 'ERROR' log/* | sort --field-separator=: --key=2 |
    awk '{colon = match($0,":"); file = substr($0,1,colon - 1); 
    if (file != prevfile) {print "\n" file; prevfile = file}; 
    print substr($0,colon+1)}'

Here are several improvements to your script, in case you still use it:

find log/ -iname "*debug*.log" -size +0 | while read -r file
do
    if grep -qsm 1 'ERROR' "$file"
    then
        echo -e "\n$file"
        grep 'ERROR' --color=auto -C 5 "$file"
    fi
done
回复 原文
幸福不弃 2024-09-18 07:48:08
Nicholas-Knights-MacBook-Pro:~/logtest$ ls
logfile1 logfile2 logfile3
Nicholas-Knights-MacBook-Pro:~/logtest$ cat logfile*
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx

2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx

2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx

Nicholas-Knights-MacBook-Pro:~/logtest$ for i in `ls logfile*` ; do printf "$i"; sort -n $i; printf '\n'; done
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 19:31:25 INFO xxx

logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 13:31:25 INFO xxx

logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
2010/07/21 15:31:25 INFO xxx

Nicholas-Knights-MacBook-Pro:~/logtest$ 

Nicholas-Knights-MacBook-Pro:~/logtest$ ls
logfile1 logfile2 logfile3
Nicholas-Knights-MacBook-Pro:~/logtest$ cat logfile*
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx

2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx

2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx

Nicholas-Knights-MacBook-Pro:~/logtest$ for i in `ls logfile*` ; do printf "$i"; sort -n $i; printf '\n'; done
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 19:31:25 INFO xxx

logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 13:31:25 INFO xxx

logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
2010/07/21 15:31:25 INFO xxx

Nicholas-Knights-MacBook-Pro:~/logtest$
回复 原文
开始看清了 2024-09-18 07:48:08

如果你已经在文件（或脚本输出）中输出了，我会选择 Perl：

$/=undef;
$t=<>;
@t=split(/\s*\n*(logfile.*)$/m,$t);
foreach $f (@t) {
    next unless $f;
    if($f =~ /^logfile/) {
      print $f;
    } else {
        print join("\n",sort (split(/\n/,$f))) . "\n\n";
   }
}

或者，更干净一点：

@lines = ();
while($t=<>) {
    if($t!~ /^2\d\d\d/) {
        print sort @lines if(scalar(@lines));
        @lines = ();
        print $t;
    }
    else {
      push @lines,$t;
   }
}
print sort @lines if(scalar(@lines));

If you have you output already in a file (or script output) I'd go Perl:

$/=undef;
$t=<>;
@t=split(/\s*\n*(logfile.*)$/m,$t);
foreach $f (@t) {
    next unless $f;
    if($f =~ /^logfile/) {
      print $f;
    } else {
        print join("\n",sort (split(/\n/,$f))) . "\n\n";
   }
}

Or, a little more clean:

@lines = ();
while($t=<>) {
    if($t!~ /^2\d\d\d/) {
        print sort @lines if(scalar(@lines));
        @lines = ();
        print $t;
    }
    else {
      push @lines,$t;
   }
}
print sort @lines if(scalar(@lines));
回复 原文
同展鸳鸯锦 2024-09-18 07:48:08
$ awk 'FNR==1{$NF=$NF" "FILENAME;}1' logfile*|sort -t" " -k1 -k2|awk 'NF==5{ h=$NF;$NF="";$0=h"\n"$0 }1'
logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 13:31:25 INFO xxx
logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
2010/07/21 15:31:25 INFO xxx
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 19:31:25 INFO xxx

$ awk 'FNR==1{$NF=$NF" "FILENAME;}1' logfile*|sort -t" " -k1 -k2|awk 'NF==5{ h=$NF;$NF="";$0=h"\n"$0 }1'
logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 13:31:25 INFO xxx
logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
2010/07/21 15:31:25 INFO xxx
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 19:31:25 INFO xxx
回复 原文
老娘不死你永远是小三 2024-09-18 07:48:08

谢谢大家。

我改进了 Dennis Williamson 的脚本，以按日期对错误进行排序。每个内部有错误的日志文件都保存在以上次发生错误的时间戳命名的文件中。这些文件随后被分类并放在一起。可能有比使用临时文件更干净的解决方案。

find log/ -iname "*debug*.log" -size +0 | while read -r file
do
    if grep -qsm 1 'ERROR' "$file"
    then
        echo -e "$i \t$file"
        errors=$(grep 'ERROR' --color=auto -C 5 "$file")
        #get the timestamp of last error occured
        time=$(echo $errors | head -n 1 | awk '{print $1" "$2}')
        timestamp=$(date -d "$time" +%s)
        #save it to temp file
        echo -e "\n$file\n$errors" > tmp/logs/$timestamp.$i
    fi
    let i++
done

#put files together
rm -f output.txt
for i in `ls tmp/logs/*|sort`;do cat $i >> output.txt ; rm  $i; done

感谢意见和改进建议！

Thank you all.

I improved script from Dennis Williamson to sort errors by date. Each log file with error inside is saved in file named by the timestamp of last error occured. These files are later sorted and put together. There may be cleaner solutions for that than to use of temp files.

find log/ -iname "*debug*.log" -size +0 | while read -r file
do
    if grep -qsm 1 'ERROR' "$file"
    then
        echo -e "$i \t$file"
        errors=$(grep 'ERROR' --color=auto -C 5 "$file")
        #get the timestamp of last error occured
        time=$(echo $errors | head -n 1 | awk '{print $1" "$2}')
        timestamp=$(date -d "$time" +%s)
        #save it to temp file
        echo -e "\n$file\n$errors" > tmp/logs/$timestamp.$i
    fi
    let i++
done

#put files together
rm -f output.txt
for i in `ls tmp/logs/*|sort`;do cat $i >> output.txt ; rm  $i; done

Opinions and suggestions for improvement appreciated!

回复 原文
月下凄凉 2024-09-18 07:48:08

您可以尝试基于 Rust 的工具超快速系统日志搜索器

（假设您有rust 安装）

cargo install super_speedy_syslog_searcher

然后

s4 logfile1 logfile2 logfile3

但是...

但将日志文件的名称保留在日志行上方

s4 无法输出该方法。

s4 可以在日志消息之前写入日志文件的名称。

$ s4 -wp logfile1 logfile2 logfile3
logfile2:2010/07/21 13:28:52 INFO xxx
logfile2:2010/07/21 13:31:25 INFO xxx
logfile2:2010/07/21 13:31:25 DEBUG xxx
logfile3:2010/07/21 15:28:52 INFO xxx
logfile3:2010/07/21 15:31:25 INFO xxx
logfile3:2010/07/21 15:31:25 DEBUG xxx
logfile1:2010/07/21 19:28:52 INFO xxx
logfile1:2010/07/21 19:31:25 INFO xxx
logfile1:2010/07/21 19:31:25 DEBUG xxx

You could try rust-based tool Super Speedy Syslog Searcher

(assuming you have rust installed)

cargo install super_speedy_syslog_searcher

then

s4 logfile1 logfile2 logfile3

HOWEVER...

but keep the name of the logfile above the log lines

s4 cannot output that approach.

s4 can write the name of the logfile to precede the log message.

$ s4 -wp logfile1 logfile2 logfile3
logfile2:2010/07/21 13:28:52 INFO xxx
logfile2:2010/07/21 13:31:25 INFO xxx
logfile2:2010/07/21 13:31:25 DEBUG xxx
logfile3:2010/07/21 15:28:52 INFO xxx
logfile3:2010/07/21 15:31:25 INFO xxx
logfile3:2010/07/21 15:31:25 DEBUG xxx
logfile1:2010/07/21 19:28:52 INFO xxx
logfile1:2010/07/21 19:31:25 INFO xxx
logfile1:2010/07/21 19:31:25 DEBUG xxx
回复 原文
~没有更多了~

关于作者

我一向站在原地

暂无简介

0 文章
0 评论
22 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

謌踐踏愛綪

文章 0 评论 0

开始看清了

文章 0 评论 0

高速公鹿

文章 0 评论 0

alipaysp_PLnULTzf66

文章 0 评论 0

热情消退

文章 0 评论 0

白色月光

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文