Grafana Loki LogQL:如何使用不同的日志格式解析日志线?

发布于 2025-01-18 03:34:27 字数 840 浏览 2 评论 0原文

对于同一个应用程序,我们有不同类型的日志。有些来自我们的应用程序,以 JSON 格式记录,有些则来自不同类型的日志消息。

例如,这 3 行日志:

"{\"written_at\": \"2022-03-30T07:51:04.934Z\", \"written_ts\": 1648626664934052000, \"msg\": \"Step 'X' started at 2022-03-30 07:51:04\", \"type\": \"log\", \"logger\": \"my-logger\", \"thread\": \"MainThread\", \"level\": \"DEBUG\", \"module\": \"my.module\", \"line_no\": 48}\n"
"                    ERROR    Data processing error: Did not work       \n"
"FileNotFoundError: [Errno 2] No such file or directory: '/local.json'\n"

为了解析我们的应用程序 JSON 日志,我们执行以下 LogQL 查询:

| json log="log" 
| line_format "{{.log}}"
| json | line_format "{{.msg}}"
| __error__ != "JSONParserErr"

正如我们的查询已经指出的那样,我们无法解析其他行日志,因为它们不是 JSON 格式。

我们能否根据条件定义不同的解析和格式? 或者作为 JSONParserErr 发生时的后备?

We have different type of logs for one and the same application. Some are coming from our application which logs in a JSON format and others are different types of log messages.

For example these 3 log lines:

"{\"written_at\": \"2022-03-30T07:51:04.934Z\", \"written_ts\": 1648626664934052000, \"msg\": \"Step 'X' started at 2022-03-30 07:51:04\", \"type\": \"log\", \"logger\": \"my-logger\", \"thread\": \"MainThread\", \"level\": \"DEBUG\", \"module\": \"my.module\", \"line_no\": 48}\n"
"                    ERROR    Data processing error: Did not work       \n"
"FileNotFoundError: [Errno 2] No such file or directory: '/local.json'\n"

To parse our application JSON logs we perform the following LogQL query:

| json log="log" 
| line_format "{{.log}}"
| json | line_format "{{.msg}}"
| __error__ != "JSONParserErr"

As our query already states, we can not parse the other line logs since they are not in JSON format.

Can we define different parsing and formatting depending on conditions?
Or as fallback when the JSONParserErr happens?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

ˉ厌 2025-01-25 03:34:28

不确定您是否设法获得了答案,因为我希望在一个查询中查看这是否可以,但是您可以使用多个查询来完成此操作……

JSON行…

| json log="log" 
| line_format "{{.log}}"
| json
| line_format "{{.msg}}"
| __error__ != "JSONParserErr"
# … more processing

对于非json行的

{swarm_stack="apiv2-acme", swarm_service="apiv2-acme_tenant-import"}
| json
| __error__ = "JSONParserErr"
| drop __error__
# … more processing

Not sure if you managed to get an answer to this, as I'm looking to see if this is possible in a single query, however you can do this with multiple queries…

For the JSON rows

| json log="log" 
| line_format "{{.log}}"
| json
| line_format "{{.msg}}"
| __error__ != "JSONParserErr"
# … more processing

For the non-JSON rows…

{swarm_stack="apiv2-acme", swarm_service="apiv2-acme_tenant-import"}
| json
| __error__ = "JSONParserErr"
| drop __error__
# … more processing
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文