将filebeat中的文件路径提取到logstash中，然后提取到弹性索引中
我正在尝试获取路径的一部分并将其作为我的elasticsearch 索引的索引。
我的logstash配置文件看起来像这样（注意：配置文件可能是错误的，因为我尝试了100种不同的东西）
# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
beats {
port => "5044"
}
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
grok {
match => ["[log][file][path]", "c:\\PL_Logs\\%{GREEDYDATA:index_name}\\%{GREEDYDATA}" ]
}
}
#output {
# stdout { codec => rubydebug }
#}
output {
elasticsearch {
hosts => [ "localhost:9222" ]
index => "%{index_name}-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
我的日志是这样归档的 C:\Incoming_Logs\requested_part\somesubfolder_doesnt_matter\2022\03\18\2022031814.txt
我的 filebeat 配置是这样的：
filebeat.inputs:
- type: log
enabled: true
paths:
- c:\Incoming_Logs\**\*.txt
我的问题是我该如何采取我的路径中的 requested_part 部分并将其设为索引？ 我尝试了很多事情，比如用 [] (%[{index_name}]) 覆盖索引我尝试匹配“path”而不是“[log][file][path]”我已将 grok 条件更改为“c:\PL_Logs\%{GREEDYDATA:index_name}\**”，我也尝试使用“/”而不是双“\”
有人能指出我在这里做错了什么吗？
I'm trying to take a part of my path and give it as index to my elasticsearch index.
my logstash config file looks like this (Note: the config file might be wrong cuz I tried 100 different things)
# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
beats {
port => "5044"
}
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
grok {
match => ["[log][file][path]", "c:\\PL_Logs\\%{GREEDYDATA:index_name}\\%{GREEDYDATA}" ]
}
}
#output {
# stdout { codec => rubydebug }
#}
output {
elasticsearch {
hosts => [ "localhost:9222" ]
index => "%{index_name}-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
my logs are filed like this C:\Incoming_Logs\requested_part\somesubfolder_doesnt_matter\2022\03\18\2022031814.txt
my filebeat config is this:
filebeat.inputs:
- type: log
enabled: true
paths:
- c:\Incoming_Logs\**\*.txt
My question is how can I take the requested_part part from my path and make it an index?
I've tried many things like covering the index with [] (%[{index_name}]) I tried to match with "path" instead of "[log][file][path]" I've changed the grok condition to "c:\PL_Logs\%{GREEDYDATA:index_name}\**" and I've also tried to use "/" instead of double "\"
Can someone point me out what I'm doing wrong here?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我从这个问题中找到了适合我的 grok 匹配 如何在 Logstash 中获取部分 Filebeat 源文件名
我是这样制作的.*)(\\|\/).*.*(\\|\/).*(\\|\/).*(\ \|\/).*(\\|\/)" }
<代码> 匹配 => { "[日志][文件][路径]" =>; ".*(\\|\/)(?
它有效，但如果有人可以提供帮助，我更喜欢更优雅的正则表达式。
I found the grok match that works for me from this question How to get parts of Filebeat source filename in Logstash
I made mine like this
match => { "[log][file][path]" => ".*(\\|\/)(?<myIndex>.*)(\\|\/).*.*(\\|\/).*(\\|\/).*(\\|\/).*(\\|\/)" }It works but I prefer a more elegance regex if someone can help.