FileBeat未从嵌套目录读取日志
我对Elk Stack是相对较新的，我正在尝试将Linux服务器的日志发送到Elasticsearch。我选择的道路是 - 我已经在Linux Server上安装了FileBeat，在该服务器上生成了应用程序日志 - ＆gt;通过logstash解析它们，然后 - ＆gt;将它们发送到
我遇到的Elasticsearch问题 - 具有应用程序日志的Linux服务器会根据一天中的哪个日/月/月份在目录中生成并存储在目录中的日志，例如，我的目录结构在06/10/2022上的我的目录结构在上午11:45上午11:45看起来像 -
- / var/log/2022/06/10/11/abc.txt
我希望我的filebeat输入路径在filebeat..yml中动态采用路径，这样我就不必继续更改路径并重新启动文件beat服务，所以我尝试了要使用 - /var/log/2022/*/*/*/* .txt之类的东西，
但是当我用诸如/var/log/2022/2022/2022/*/*/*/* /* .txt的通配符指定文件路径时，我没有登录发货和FileBeat服务运行良好，但收割机总是显示0个文件，并且没有运输日志，但是当我将其更改为在不使用通配符的情况下将其更改为任何文件夹时日志被运送了，我可以在Elasticsearch中看到它们。因此，我想知道我应该做什么，以使这种动态路径工作，并且我使用的文件键版本为7.17.0。 如果你们有任何想法，请告诉我。
（注意： - 2022年内有12个文件夹，持续数月，例如-01、02、03等。 在这些文件夹中几个月内有子文件夹以支持日期，具体取决于几个月中的几天-01，02 ... 29,30 ...等 在这些人中，一天中的几个小时有23个子文件夹，例如-00,01,02 ... 23）
- 我遇到的另一个问题是，每当我得到日志发货时，我都会看到潜伏期，就像我理想地希望日志立即看到潜伏期出现在Elasticsearch中，一旦出现在Linux服务器中，该应用程序正在运行并生成日志，但是我总是看到像日志这样的延迟至少在Elasticsearch中出现，至少延迟了5-15分钟，所以我如何能尽快使它出现在当他们像实时出现一样？
I am relatively new to ELK stack and I am trying to send logs from a linux servers to elasticsearch. The path I am choosing is -
I have installed the filebeat on linux server where my application logs are getting generated - > parsing them via logstash and then - > sending them to elasticsearch
Questions I have are -
The linux server having application logs generates and stored logs in directories dynamically based on what day/month/time of the day its running for example my directory structure for logs on 06/10/2022 at 11:45 am will look like -
-/var/log/2022/06/10/11/abc.txt
I want my filebeat input path for logs in filebeat.yml to take paths dynamically so that I do not have to keep changing the paths and restarting the filebeat service so I tried to use something like - /var/log/2022/*/*/*/*.txt
But when I specify file path with wildcards like /var/log/2022/*/*/*/*.txt I get no logs shipped and the filebeat service runs fine but harvester always shows 0 files and no logs gets shipped however when I changes that to specifically point to any folder without using wildcard like -/var/log/2022/06/10/11/abc.txt the logs get shipped and I can see them in elasticsearch. So, I want to know what I should do in order to make this dynamic path work and the filebeat version I am using is 7.17.0.
Please let me know if you guys have any ideas.
(Note: - There are 12 folders inside 2022 for months like - 01, 02, 03 etc ..
Inside those folders for months there are sub folders to support dates depending on how many days in the months like - 01, 02...29,30 ...etc
Inside those there are 23 subfolders for hours of the day like - 00,01,02...23 )
- Another question I have is whenever I get the logs to get shipped I see latency like I ideally want logs to immediately appear in Elasticsearch as soon as they appear in linux server where the application is running and generating the logs but I always see a latency like logs appears in elasticsearch at least with a delay of 5-15 mins so how can I make it appear as soon as they show up like real time ?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您是否尝试过使用双星号？ https://github.com/elastic/elastic/beats/beats/pulls/pull/3980 实施了一个更改（FileBeat 6.0，因此应该在7.x中起作用）扩展**至16个级别。
Have you tried using a double asterisk? https://github.com/elastic/beats/pull/3980 implemented a change (FileBeat 6.0, so it should work in 7.x) to expand ** up to sixteen levels.