Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [logstash]

logstash is a tool for collecting and distributing log events.

logstash is a free and open source tool (apache 2.0 license) for managing events and logs. It can be used to collect and parse logs, and distribution to other indexing systems. It has a web interface for searching and drilling into the logs.

261 questions
12
votes
2 answers

Configuring Logstash when installed as a service

I have installed logstash as a service using the logstash APT repository on Ubuntu 13.10. So now I can run: dpkg -s logstash And it outputs: Package: logstash Status: install ok installed Priority: extra Section: default Installed-Size:…
Binyomin Trager
  • 1,018
8
votes
1 answer

What is the significance of the @ prefix in logstash field names?

The following logstash configuration is used to accept Windows Event Logs as json over a TCP connection and then after some filtering forward the result to Elastic search (source: https://gist.github.com/robinsmidsrod/4215337): input { tcp { …
Kev
  • 7,897
6
votes
2 answers

logstash alert after 1000 occurences

I am trying to make Logstash to alert me only after it receives over 1000 items within 10 minutes. I need alerts in both Hipchat and PagerDuty. My config seems reasonable, but does not work as expected. filter { if…
Sart
  • 63
5
votes
6 answers

Logstash binding to a port already in use

This is the output when I try and run logstash. With Redis and ElasticSearch disabled it still says address already in use. Any suggestions? As far as I can tell this was fixed in 1.1.8 but I seem to still have this problem.…
David Neudorfer
  • 171
4
votes
3 answers

logstash: is there a way to view the complete running config when using multiple config files in /etc/logstash/conf.d?

I am running logstash 1.5.0.1 and I have multiple config files in my /etc/logstash/conf.d folder: 01-input-source-one.conf 02-input-source-two.conf 10-filter-one.conf 11-filter-two.conf 20-output-one.conf 21-output-two.conf After modifying a…
Peter M
  • 973
4
votes
3 answers

logstash failing to parse syslog input

I've configured logstash (v1.5.0), with a simple syslog input, as follows: input { syslog { type => syslog port => 5514 } } filter { kv {} } output { elasticsearch { cluster => "logs" host => "0.0.0.0" protocol =>…
Olly
  • 449
4
votes
1 answer

Logstash Date Has the Wrong Year?

I'm parsing Nginx logs into logstash with the following config: input { stdin { type => "nginx"}} filter { grok { type => nginx pattern => "%{COMBINEDAPACHELOG}" } date { type => nginx match => [ …
Brian Hicks
  • 185
3
votes
1 answer

Logstash output-http plugin error 500 when trying to send to slack webhook

I'm running a docker container with an ELK stack. Everythings working dandy and fine. I'd like to utilize the https://www.elastic.co/guide/en/logstash/current/plugins-outputs-http.html plugin. I install the plugin it: root@elk:/#…
Karl Morrison
  • 1,661
3
votes
1 answer

logstash timestamp on year rollover

We use logstash to store/search logs from our mail servers. I noticed today that we didn't have any indices from this year (2015). Quick investigation showed that current logs were being stored as 2014.01.05 (ie same day but last year) and these…
Paul Haldane
  • 4,547
3
votes
1 answer

Logstash-forwarder is throwing SSL errors

I got this task handed over to by my colleage and this is the background. He got ELK (Elasticsearch, Logstash and Kibana) stack working with our RHEL 6.2 servers, by using the regular method of configuring the Logstash on the server and…
Sreeraj
  • 464
3
votes
3 answers

How to send ip address to logstash using input as "file"?

I am sending few logs to logstash central server using another logstash on client as shipper. The input type is "file". The messages are received fine on server but it does not reflect the IP address of the client. It sends the hostname instead in…
Aditya Patawari
  • 1,085
2
votes
0 answers

Logstash cannot find logstash.yml although it exists

Testing logstash as instructed here. This is the command I execute (it doesn't work without the --path.settings too): /usr/share/logstash/bin/logstash -f logstash-simple.conf --path.settings /etc/logstash/logstash.yml But I am getting this…
kev
  • 261
  • 1
  • 6
  • 13
2
votes
1 answer

logstash multiline log for a mysql query

I'm looking to push logs from mysql-proxy lua's script into lostash. An example log might be [2015-03-09 11:13:47] USER:username IP:10.102.51.134:41420 DB:dbName Query: -- One Pager Trends -- params: SELECT date, SUM(t.rev) revenue, …
Mike
  • 22,510
2
votes
0 answers

Why logstash multiline filter is not matching the logs?

I am sending logs to logstash using beaver. These logs are multiline in nature so I am using multiline filter but somehow the filter is not able to match the log. There are no errors in the output of the logstash. Here is what I see in…
Aditya Patawari
  • 1,085
2
votes
1 answer

Repair logstash checkpoint file

We had an OutOfMemoryException in our logstash 5.6.6 which lead to the checkpoint. head file not to be written properly: ls -l /var/lib/logstash/queue/main/ total 266424 -rw-r--r-- 1 logstash logstash 34 Mar 1 20:33…
Dero
  • 185
1
2