根据唯一的跟踪 ID 组合 Filebeat 日志以发送到 Elastic Search（无需 Logstash）

Question

我已经配置了 filebeat，以便将 Spring Boot 应用程序的日志直接发送到 AWS Opensearch Service，而无需配置 Logstash。我想配置 filebeat 多行正则表达式，使其可以将所有日志合并到具有相同 transactionId 的单个记录中。有人可以建议吗？ 这里日志配置的部分如下 filebeat.yml 所示：

filebeat.inputs:
- type: log
  paths:
    - E://filebeat//*.log
  reload.enabled: true
  reload.period: 300s
  enabled: true
  multiline:
    type: pattern
    pattern: '^\['
    negate: true
    match: after
     
Log Sample:

2022-03-07 07:08:44 [ERROR] | transactionId=6af42925-48df-4f49-95a4-aa0e43b152ed | c.example.utility.advice.LoggingAdvice - Method Signature:  Boolean com.example.config.service.impl.ServiceImpl.testLogger() - Line No : 1345 - Exception :  null
2022-03-07 07:08:50 [ERROR] | transactionId=51a2d445-574b-4992-b867-a7fc5ee4b473 | c.example.utility.advice.LoggingAdvice - Method Signature:  Boolean com.example.config.service.impl.ServiceImpl.testLogger() - Line No : 1345 - Exception :  null
2022-03-07 05:26:08 [ERROR] | transactionId=17f4f815-ee74-4658-a8d7-347fea6dd9d1 | o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause
java.lang.NullPointerException: null
    at com.example.config.service.impl.ServiceImpl.testLogger(EventServiceImpl.java:1345) ~[classes/:na]
    at com.example.config.service.impl.ServiceImpl$$FastClassBySpringCGLIB$$983a4fb2.invoke(<generated>) ~[classes/:na]
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.9.jar:5.3.9]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) ~[spring-aop-5.3.9.jar:5.3.9]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.9.jar:5.3.9]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.9.jar:5.3.9]
    at 

Answer 1

您可以在 filebeat.yml 文件的多行部分中使用这些行，

multiline.type: pattern
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after

有关更多参考，请访问 官方网站。