日志不会发送到elasticsearch

Question

一段时间以来，我一直在尝试将简单的日志发送到 Elasticsearch，在尝试了一个非常简单的示例后，日志没有从 Logstash 发送到 Elasticsearch。

服务：在同一服务器上进行此测试 操作系统：Centos 7 Logstash版本是：7.17.1 Elasticsearch 版本是： 7.17.1

/etc/logstash/conf.d

input {
  file {
    path => "/var/log/Elasticsearch/Elasticsearch.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
}
}
filter {

}
output {
  Elasticsearch {
  hosts => ["localhost:9200"]
}
}

/var/log/logstash/logstash-plain.log

[2022-03-18T11:33:30,690][INFO ][org.reflections.Reflections] Reflections took 118 ms to scan 1 urls, producing 119 keys and 417 values
[2022-03-18T11:33:32,042][INFO ][logstash.outputs.Elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::Elasticsearch", :hosts=>["//localhost:9200"]}
[2022-03-18T11:33:32,540][INFO ][logstash.outputs.Elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://localhost:9200/]}}
[2022-03-18T11:33:32,889][WARN ][logstash.outputs.Elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2022-03-18T11:33:32,908][INFO ][logstash.outputs.Elasticsearch][main] Elasticsearch version determined (7.17.1) {:es_version=>7}
[2022-03-18T11:33:32,913][WARN ][logstash.outputs.Elasticsearch][main] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2022-03-18T11:33:33,037][INFO ][logstash.outputs.Elasticsearch][main] Config is not compliant with data streams. data_stream => auto resolved to false
[2022-03-18T11:33:33,113][INFO ][logstash.outputs.Elasticsearch][main] Config is not compliant with data streams. data_stream => auto resolved to false
[2022-03-18T11:33:33,311][INFO ][logstash.outputs.Elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2022-03-18T11:33:33,337][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/etc/logstash/conf.d/logstash.conf"], :thread=>"#<Thread:0x15acb961 run>"}
[2022-03-18T11:33:34,573][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.23}
[2022-03-18T11:33:34,664][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-03-18T11:33:34,761][INFO ][filewatch.observingtail ][main][2b6c69038f817ebf29690e5d479fe4c6e56f482b9d6cc052978d217447903269] START, creating Discoverer, Watch with file and sincedb collections
[2022-03-18T11:33:34,771][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}

/var/log/Elasticsearch/Elasticsearch.log

[2022-03-18T01:30:00,079][INFO ][o.e.x.m.MlDailyMaintenanceService] [ip-.eu-west-2.compute.internal] Successfully completed [ML] maintenance task: triggerDeleteExpiredDataTask
[2022-03-18T10:23:44,010][INFO ][o.e.c.m.MetadataIndexTemplateService] [ip-.eu-west-2.compute.internal] adding template [logstash] for index patterns [logstash-]
[2022-03-18T10:23:44,189][INFO ][o.e.c.m.MetadataCreateIndexService] [ip-.eu-west-2.compute.internal] [logstash-2022.03.18-000001] creating index, cause [api], templates [logstash], shards [1]/[1]
[2022-03-18T10:23:44,522][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [ip-.eu-west-2.compute.internal] adding index lifecycle policy [logstash-policy]
[2022-03-18T10:23:44,603][INFO ][o.e.x.i.IndexLifecycleTransition] [ip-.eu-west-2.compute.internal] moving index [logstash-2022.03.18-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [logstash-policy]
[2022-03-18T10:23:44,671][INFO ][o.e.x.i.IndexLifecycleTransition] [ip-.eu-west-2.compute.internal] moving index [logstash-2022.03.18-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] in policy [logstash-policy]
[2022-03-18T10:23:44,726][INFO ][o.e.x.i.IndexLifecycleTransition] [ip-.eu-west-2.compute.internal] moving index [logstash-2022.03.18-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [logstash-policy]
[2022-03-18T10:23:55,371][INFO ][o.e.c.r.a.DiskThresholdMonitor] [ip-*.eu-west-2.compute.internal] low disk watermark [85%] exceeded on [r51WwHrKTE-VK6UCAaR4IA][ip-*8.eu-west-2.compute.internal][/var/lib/Elasticsearch/nodes/0] free: 1.1gb[14.1%], replicas will not be assigned to this node

任何帮助将不胜感激:)

Answer 1

您需要在输出中指定索引。
请注意，Elasticsearch 中存在可用空间问题：超出了低磁盘水位线 [85%]。