elasticsearch+logstash+jdbc

Question

logstash配置：

output {
   elasticsearch {
      hosts => ["192.168.8.8:9200"]
      index => "jdbc_index"
      document_id => "%{id}"
   }

   stdout {
      codec => json_lines
   }
}

上面配置两种方式输出的，为什么使用kibana显示ES的记录只有一条呢

stdout输出的就是正常的：

搞了好久了，有前辈知道原因的吗？

配置文件代码：

input {
   stdin {
   }
   jdbc {
      jdbc_connection_string => "jdbc:mysql://192.168.12.222:3306/main"
      jdbc_user => "admin"
      jdbc_password => "123456"
      jdbc_driver_library => "/usr/local/logstash/mysql-connector-java-5.1.3.jar"
      jdbc_driver_class => "com.mysql.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      statement_filepath => "/usr/local/logstash/config/jdbc.sql"
      #statement => "select inquiryId, inquiryNo, carTypeId, orgId, `status`, createTime, publishTime, failTime from qp_inquiry limit 500"
      schedule => "* * * * *"
      type => "jdbc_type"
      lowercase_column_names => "false"
   }
}

filter {
   json {
      source => "message"
      remove_field => ["message"]
   }
}

output {
   elasticsearch {
      hosts => ["192.168.8.8:9200"]
      index => "jdbc_index"
      document_id => "%{id}"
   }

   stdout {
      codec => json_lines
   }
}

sql语句：

SELECT
*
FROM
 (
  SELECT
   inquiryId,
   inquiryNo,
   carTypeId,
   orgId,
   `status`,
   createTime,
   publishTime,
   failTime
  FROM
   qp_inquiry
  LIMIT 50
 ) as `t1`
LIMIT 50000 OFFSET 0

Answer 1

看你的原始字段是没有 id 这个字段的，你应该指定一个存在的字段