如何“grep”连续的流?
是否可以在连续流上使用 grep
?
我的意思是有点像 tail -f
命令,但在输出上使用 grep
以便仅保留我感兴趣的行。
我已经尝试过 tail -f
但似乎 grep
只能在 tail
完成后执行,也就是说永远不会执行。
Is that possible to use grep
on a continuous stream?
What I mean is sort of a tail -f <file>
command, but with grep
on the output in order to keep only the lines that interest me.
I've tried tail -f <file> | grep pattern
but it seems that grep
can only be executed once tail
finishes, that is to say never.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
![扫码二维码加入Web技术交流群](/public/img/jiaqun_03.jpg)
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(13)
使用BSD grep(FreeBSD,Mac OS X等)时打开
grep
的行缓冲模式不久前看起来
--line-buffered
并不重要对于 GNU grep(几乎在任何 Linux 上使用),因为它默认刷新(YMMV 对于其他类 Unix 系统,如 SmartOS、AIX 或 QNX)。然而,截至 2020 年 11 月,需要--line-buffered
(至少在 openSUSE 中的 GNU grep 3.5 中是这样,但根据下面的评论,似乎通常需要)。Turn on
grep
's line buffering mode when using BSD grep (FreeBSD, Mac OS X etc.)It looks like a while ago
--line-buffered
didn't matter for GNU grep (used on pretty much any Linux) as it flushed by default (YMMV for other Unix-likes such as SmartOS, AIX or QNX). However, as of November 2020,--line-buffered
is needed (at least with GNU grep 3.5 in openSUSE, but it seems generally needed based on comments below).我使用
tail -f; | grep
始终。它将等到 grep 刷新,而不是完成(我使用的是 Ubuntu)。
I use the
tail -f <file> | grep <pattern>
all the time.It will wait till grep flushes, not till it finishes (I'm using Ubuntu).
我认为你的问题是 grep 使用一些输出缓冲。尝试
将 grep 的输出缓冲模式设置为无缓冲。
I think that your problem is that grep uses some output buffering. Try
it will set output buffering mode of grep to unbuffered.
如果您想在整个文件(而不仅仅是尾部)中查找匹配项,并且希望它坐下来等待任何新匹配项,那么这很有效:
-c +0< /code> 标志表示输出应从文件开头 (
+
) 开始0
字节 (-c
)。If you want to find matches in the entire file (not just the tail), and you want it to sit and wait for any new matches, this works nicely:
The
-c +0
flag says that the output should start0
bytes (-c
) from the beginning (+
) of the file.在大多数情况下,您可以
tail -f /var/log/some.log |grep foo
,它会正常工作。如果您需要在正在运行的日志文件上使用多个 grep 并且发现没有输出,则可能需要将
--line-buffered
开关粘贴到中间 grep(s),像这样:In most cases, you can
tail -f /var/log/some.log |grep foo
and it will work just fine.If you need to use multiple greps on a running log file and you find that you get no output, you may need to stick the
--line-buffered
switch into your middle grep(s), like so:你可能会认为这个答案是增强..通常我使用
-F 在文件旋转的情况下更好(如果文件旋转,-f 将无法正常工作)
-A 和 -B 对于获取模式出现之前和之后的行很有用..这些块将出现在虚线分隔符之间,
但对我来说,我更喜欢执行以下操作
,如果您想在流式日志中搜索,这非常有用。我的意思是回顾过去并深入观察
you may consider this answer as enhancement .. usually I am using
-F is better in case of file rotate (-f will not work properly if file rotated)
-A and -B is useful to get lines just before and after the pattern occurrence .. these blocks will appeared between dashed line separators
But For me I prefer doing the following
this is very useful if you want to search inside streamed logs. I mean go back and forward and look deeply
没有看到有人提供我通常的做法:
我更喜欢这个,因为您可以使用
ctrl + c
随时停止并浏览文件,然后只需按shift + f
返回实时流式搜索。Didn't see anyone offer my usual go-to for this:
I prefer this, because you can use
ctrl + c
to stop and navigate through the file whenever, and then just hitshift + f
to return to the live, streaming search.sed 将是更好的选择(流编辑器)
tail -n0 -f; | sed -n '/search string/p'
然后如果您希望 tail 命令在找到特定字符串后退出:
tail --pid=$(($BASHPID+1)) -n0 -f <文件> | sed -n '/搜索字符串/{p; q}'
显然是一个 bashism:$BASHPID 将是 tail 命令的进程 ID。 sed 命令位于管道中 tail 之后的下一个命令,因此 sed 进程 ID 将为 $BASHPID+1。
sed would be a better choice (stream editor)
tail -n0 -f <file> | sed -n '/search string/p'
and then if you wanted the tail command to exit once you found a particular string:
tail --pid=$(($BASHPID+1)) -n0 -f <file> | sed -n '/search string/{p; q}'
Obviously a bashism: $BASHPID will be the process id of the tail command. The sed command is next after tail in the pipe, so the sed process id will be $BASHPID+1.
关于这个问题有点晚了,考虑到这种工作是监控工作的重要组成部分,这是我的答案(不是那么短)...
使用 bash
1. 命令
tail
这个命令稍微详细一点比阅读已经发布的答案
之间的区别遵循选项
tail -f
和tail -F
,来自手册页:<块引用>
这意味着:通过使用
-F
而不是-f
,tail
将在删除时重新打开文件(在日志上)旋转,用于示例)。这对于观察多天的日志文件很有用。
能够同时关注多个文件
我已经用过:
对于通过数百个文件跟踪事件...(考虑此答案的其余部分以了解如何使其可读...;)
使用开关
-n
(Don请勿使用-c
进行行缓冲!)。默认情况下
tail
将显示最后 10 行。可以调整:将跟随文件,但仅打印新行
在跟踪他的进度之前将打印整个文件。
2.管道传送时的缓冲问题:
如果您计划过滤输出,请考虑缓冲!请参阅
sed
的-u
选项、grep
的--line-buffered
或stdbuf
选项code> 命令:(比使用 grep 更有效)比在 sed 中不使用 -u 开关更具反应性> 命令。
3. 最近的日志系统
在最近的系统上,您必须以几乎相同的方式运行
journalctl -xf
,而不是tail -f /var/log/syslog
...但是阅读
手册页
,该工具是为日志分析而构建的!4. 将其集成到 bash 脚本中
两个文件(或更多)的彩色输出
以下是监视多个文件的脚本示例,第一个文件的输出颜色与其他文件不同:
<前><代码>#!/bin/bash
尾-F“$@”|
sed-une“
/^==> /{h;};
//!{
克;
s/^\\(.*\\)\\n==>.*${1//\//\\\/}.*<==/\\o33[47m\\1\\ o33[0米/;
s/^\\(.*\\)\\n==> .* <==/\\o33[47;31m\\1\\o33[0m/;
p;}”
它们在我的主机上运行良好,运行:
交互式脚本
您可能正在查看日志以对事件做出反应?
这是一个小脚本,当某些 USB 设备出现或消失时播放一些声音,但相同的脚本可以发送邮件或任何其他交互,例如打开咖啡机电源...
<前><代码>#!/bin/bash
执行{tailF} < <(tail -F /var/log/kern.log)
尾部Pid=$!
同时:;做
读取-rsn 1 -t .3键盘
[ "${键盘,}" = "q" ] &&休息
如果读取 -ru $tailF -t 0 _ ;则
读取 -ru $tailF 行
案例 $line in
*新\ USB\ 设备\ 已找到* ) 播放/some/sound.ogg ;;
*USB\ 断开连接* ) 播放 /some/othersound.ogg ;;
埃萨克
printf "\r%s\e[K" "$line"
菲
完毕
回声
exec {tailF}<&-
杀死 $tailPid
您可以按Q键退出。
Coming some late on this question, considering this kind of work as an important part of monitoring job, here is my (not so short) answer...
Following logs using bash
1. Command
tail
This command is a little more porewfull than read on already published answer
Difference between follow option
tail -f
andtail -F
, from manpage:This mean: by using
-F
instead of-f
,tail
will re-open file(s) when removed (on log rotation, for sample).This is usefull for watching logfile over many days.
Ability of following more than one file simultaneously
I've already used:
For following events through hundreds of files... (consider rest of this answer to understand how to make it readable... ;)
Using switches
-n
(Don't use-c
for line buffering!).By default
tail
will show 10 last lines. This can be tunned:Will follow file, but only new lines will be printed
Will print whole file before following his progression.
2. Buffer issues when piping:
If you plan to filter ouptuts, consider buffering! See
-u
option forsed
,--line-buffered
forgrep
, orstdbuf
command:Is (a lot more efficient than using
grep
) a lot more reactive than if you does'nt use-u
switch insed
command.3. Recent journaling system
On recent system, instead of
tail -f /var/log/syslog
you have to runjournalctl -xf
, in near same way...But read
man page
, this tool was built for log analyses!4. Integrating this in a bash script
Colored output of two files (or more)
Here is a sample of script watching for many files, coloring ouptut differently for 1st file than others:
They work fine on my host, running:
Interactive script
You may be watching logs for reacting on events?
Here is a little script playing some sound when some USB device appear or disappear, but same script could send mail, or any other interaction, like powering on coffe machine...
You could quit by pressing Q key.
是的,这实际上效果很好。
Grep
和大多数 Unix 命令在流上一次运行一行。从 tail 出来的每一行都会被分析,如果匹配则继续传递。Yes, this will actually work just fine.
Grep
and most Unix commands operate on streams one line at a time. Each line that comes out of tail will be analyzed and passed on if it matches.这个命令对我有用(Suse):
收集邮件服务的登录信息
This one command workes for me (Suse):
collecting logins to mail service
您肯定不会成功
当您使用“colortail”作为 tail 的别名时, ,例如。在 bash 中
你可以检查
类型别名
如果这输出类似
tail 是
colortail -n 30
的别名。那么你就有了罪魁祸首:)
解决方案:
删除别名并
确保你通过此命令使用“真正的”尾部二进制文件,
该命令应该输出类似以下内容:
然后你可以运行你的命令
祝你好运。
you certainly won't succeed with
when you use "colortail" as an alias for tail, eg. in bash
you can check by
type alias
if this outputs something like
tail isan alias of
colortail -n 30
.then you have your culprit :)
Solution:
remove the alias with
ensure that you're using the 'real' tail binary by this command
which should output something like:
and then you can run your command
Good luck.
如果没有行缓冲选项,请使用 awk(另一个很棒的 bash 实用程序)代替 grep!它将持续从尾部传输您的数据。
这就是你如何使用 grep
这就是你如何使用 awk
Use awk(another great bash utility) instead of grep where you dont have the line buffered option! It will continuously stream your data from tail.
this is how you use grep
This is how you would use awk