如何将 webhook json 负载发送到 elasticsearch 索引

发布于 2025-01-11 11:59:26 字数 1575 浏览 0 评论 0原文

我的目标是每当我的 webhook 收到 POST 请求时，我都会将该 json 负载发送到 ElasticSearch 索引。

为此，我使用了带有 HTTP 插件的 Logstash 管道。当我使用来自端口 9000 上本地配置的 Webhook 的输入时，当我的 Webhook 在本地配置时，我能够发送 Webhook 消息，

input {
  http {
    host => "0.0.0.0"
    port => "9000"
    codec => "json"
  }
}
filter {
    mutate {
        add_field => {
          "documentType" => "productionlogs"
          "deleted" => "false"
          "created" => "%{@timestamp}"
          "modified" => "%{@timestamp}"
          }
    }
}
output {
  elasticsearch {
    hosts    => [ '${ES_HOST_ADDRESS}' ]
    user     => '${ES_USER}'
    password => '${ES_PASSWORD}'
    ssl => "true"
    index => "production_logs"
  }
  stdout {
    id => "Pipeline_logs"
    codec => "json"
  }
}

但当我使用来自“https://requestinspector.com/”的外部 Webhook 端点时，例如，

input {
  http {
    host => "requestinspector.com/inspect/xxxxx"
    port => "443"
    ssl ==> "true"
    codec => "json"
  }
}
filter {
...

No documents are added to elasticsearch index whi this error
[ERROR] 2022-03-02 08:21:07.004 [Converge PipelineAction::Create<pipeline-dev-webhook>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:pipeline-dev-webhook, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"=>\" at line 5, column 9 (byte 114) after input {\n  http {\n    host =>...

有什么帮助吗？谢谢

原文

My goal is whenever my webhook receives a POST request, I'd like to send that json payload to an ElasticSearch Index.

For this purpose, I used Logstash pipeline with HTTP plugin.
When I am using input from a webhook configured locally on port 9000, I am able to send webhook messages when my webhook is configured locally

input {
  http {
    host => "0.0.0.0"
    port => "9000"
    codec => "json"
  }
}
filter {
    mutate {
        add_field => {
          "documentType" => "productionlogs"
          "deleted" => "false"
          "created" => "%{@timestamp}"
          "modified" => "%{@timestamp}"
          }
    }
}
output {
  elasticsearch {
    hosts    => [ '${ES_HOST_ADDRESS}' ]
    user     => '${ES_USER}'
    password => '${ES_PASSWORD}'
    ssl => "true"
    index => "production_logs"
  }
  stdout {
    id => "Pipeline_logs"
    codec => "json"
  }
}

but when I am using an external webhook endpoint from "https://requestinspector.com/" for example,

input {
  http {
    host => "requestinspector.com/inspect/xxxxx"
    port => "443"
    ssl ==> "true"
    codec => "json"
  }
}
filter {
...

No documents are added to elasticsearch index whi this error
[ERROR] 2022-03-02 08:21:07.004 [Converge PipelineAction::Create<pipeline-dev-webhook>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:pipeline-dev-webhook, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"=>\" at line 5, column 9 (byte 114) after input {\n  http {\n    host =>...

any help?
Thanks

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

纸伞微斜 2025-01-18 11:59:27

您只需在第 5 行编写 ==> 而不是 => ;-)

    ssl ==> "true"

应该是：

    ssl => "true"

以 Expected one of ... 通常是拼写错误/缺少括号/等等。

You simply wrote ==> instead of => at line 5 ;-)

    ssl ==> "true"

Should be :

    ssl => "true"

Errors beginning with Expected one of ... are usually typos / missing brackets / etc.

回复收藏 0 原文

蓝海 2025-01-18 11:59:27

@RedaE，您好，您可以参考这篇博文：

https://ashish。 one/blogs/elastic/receive-webhook-requests-using-elk/

webhook-receiver.conf

input {
  http {
    port => 9000
  }
}

filter {
  json {
    source => "message"
  }
}

output {   
  elasticsearch {
    hosts => ["https://es01:9200"]
    cacert => '/usr/share/logstash/pipeline/certs/ca.crt'
    user => 'elastic'
    password => 'pass@123'
    index => 'webhook'
  }
}

Hi @RedaE you can refer this blogpost:

https://ashish.one/blogs/elastic/receive-webhook-requests-using-elk/

webhook-receiver.conf

input {
  http {
    port => 9000
  }
}

filter {
  json {
    source => "message"
  }
}

output {   
  elasticsearch {
    hosts => ["https://es01:9200"]
    cacert => '/usr/share/logstash/pipeline/certs/ca.crt'
    user => 'elastic'
    password => 'pass@123'
    index => 'webhook'
  }
}

回复收藏 0 原文

~没有更多了~