Logstash -> Input配置如下:
file { type => "test_aaa" path => ["/usr1/app/logs/aaa"] sincedb_path => "/home/jfy/soft/logstash-2.0.0/sincedb_aaa.access" start_position => "beginning" codec => multiline { max_bytes => "10MiB" max_lines => 500 charset => "GBK" pattern => "^(?!.*?=== >>>>>>>> ===).*$" what => "previous" } }
multiline合并,默认最大合并500行,最大大小10M
昨天由于程序出错导致一个请求出现了50000多行Err Log,由于Logstash multiline合并导致出现了一个大文档,在kibana显示时显示不出来,可以通过适当调节这两个参数来限制合并日志的行数与大小