Logstash配置详解

一、配置文件

Logstash配置文件位于Logstash安装目录下bin/logstash.conf

启动命令:

logstash -f logstash.conf

logstash.yml

logstash.yml 配置 es 数据推送的地址

logstash.con

logstash.con 文件

input {
  file {
    path => "/home/test/logstash/ml-25m/movies.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {

    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

  mutate {
    convert => {
      "year" => "integer"
    }

二 、插件

2.1 logstash-input-file配置

1. Input Plugin

1.1 从文件输入
从文件读取数据，如常见的日志文件。文件读取通常要解决几个问题：