Logstash 无法执行操作

Question

我正在尝试在 docker 中设置logstash。 我正在使用logstash:8.0.0 图像。

这是我的logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.enabled: false

这是我的pipeline.conf

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["http://10.135.95.164:9200"]
    index => "instameister"
    username => "elastic"
    password => ""
  }
  stdout { codec => rubydebug }
}

这是我收到的错误：

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:120)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:85)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:333)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:376)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:829)"]}

我看到的只是无法配置插件：（ConfigurationError）您的配置有问题。但我不知道出了什么问题。

Answer 1

您可以覆盖环境中的变量，而不是修改 logstash.yml。您的 pipeline.conf 似乎没问题。默认情况下，为 stdout 启用 rubydebug 编解码器。

因此，假设您有一个 docker compose 文件，配置将如下所示：

  logstash:
    image: docker.elastic.co/logstash/logstash
    container_name: logstash
    restart: always
    user: root
    volumes:
      - ./logstash/pipeline:/usr/share/logstash/pipeline:ro
      - ./logstash/logs/:/logstash/logs/:rw
    environment:
      - xpack.monitoring.enabled=false 
      - outputs.elasticsearch=http://elasticuser:elasticuserpassword@elasticsearch:9200
    depends_on:
      - elasticsearch

在 ./logstash/pipeline 目录中，有一个 logstash.conf 文件，其中包含

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => "${outputs.elasticsearch}"
  }

  stdout {
  }
}

：满足您的需求。

Answer 2

问题是关键“用户名”应该是“用户”
这是工作配置：

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["http://10.135.95.164:9200"]
    user => "elastic"
    password => ""
    index => "instameister"
    manage_template => false
  }
  stdout { codec => json_lines }
}