filebeat-logstash -es 结合实战案例

111. 配置

filebeat.inputs:
- type: log
  paths: ["/var/log/nginx/access.log"]
  processors:
  - decode_json_fields:
      fields: ["message"]
      target: ""
  - add_fields:
      target: ""
      fields:
        apps: nginx

output.logstash:
  hosts: ["10.0.0.113:8888"]

启动filebeat

filebeat -e -c /root/config/hm.yaml

112 配置

filebeat.inputs:
- type: log
  paths: ["/baimei/softwares/tomcat/logs/localhost_access_log*.txt"]
  processors:
  - decode_json_fields:
      fields: ["message"]
      target: ""
  - add_fields:
      target: ""
      fields:
        apps: tomcat

- type: log
  paths: ["/tmp/app.log"]
  processors:
  - add_fields:
      target: ""
      fields:
        apps: generate

output.logstash:
  hosts: ["10.0.0.113:8888"]

启动 filebeat

filebeat -e -c /root/config/hm-tomcat.yaml 

113 logstash配置

[root@baimeidashu-elk113 ~/config]#cat  nginx-filebeat-logstash-es.conf
input {
  beats {
    port => 8888
  }
  
}


filter {
   mutate {
       remove_field => [ "@version","agent","host","input","ecs","log","tags" ]
   }
}


output { 
   stdout {
     codec => rubydebug 
   } 

#  if [apps] == "nginx" {
#    elasticsearch {
#      hosts => ["10.0.0.101:19200","10.0.0.102:19200","10.0.0.103:19200"]
#      index => "baimei-logstash-if-nginx-%{+yyyy.MM.dd}"
#    } 
#  } else if [apps] == "tomcat" {
#    elasticsearch {
#      hosts => ["10.0.0.101:19200","10.0.0.102:19200","10.0.0.103:19200"]
#      index => "baimei-logstash-if-tomcat-%{+yyyy.MM.dd}"
#    } 
#  }

}

启动 logstash:

logstash -rf /root/config/nginx-filebeat-logstash-es.conf 

终端测试成功

可以打开 logstash  的输出端为es l

  if [apps] == "nginx" {
    elasticsearch {
      hosts => ["10.0.0.111:19200","10.0.0.112:19200","10.0.0.113:19200"]
      index => "baimei-logstash-if-nginx-%{+yyyy.MM.dd}"
    } 
  } else if [apps] == "tomcat" {
    elasticsearch {
      hosts => ["10.0.0.111:19200","10.0.0.112:19200","10.0.0.113:19200"]
      index => "baimei-logstash-if-tomcat-%{+yyyy.MM.dd}"
    } 
  }

}

pipeline

欢迎来撩 ： 汇总all