Logstash

IRC: #logstash at freenode

Grok debugger: http://grokdebug.herokuapp.com/

Nginx config example: https://github.com/elasticsearch/kibana/blob/master/sample/nginx.conf

Elasticsearch

You may want to increase the maximum open file limit.

To show limits for a running process:

egrep 'files|Limit' /proc/$pid/limits

Increasing File Limits

Global file handle limit (300k on a server, 800k on my desktop):

cat /proc/sys/fs/file-max

Upstart

Add the following to your job:

limit nofile 64000 64000

See http://upstart.ubuntu.com/wiki/Stanzas#limit

Per User

/etc/security/limits.conf:

elasticsearch    hard  nofile         64000
elasticsearch    soft  nofile         64000

And to make these settings apply for java in /etc/pam.d/common-session:

# nofile limit should apply for "java" executable too (elasticsearch)
session required        pam_limits.so

Check what elasticsearch got:

curl localhost:9200/_nodes?process

http://stackoverflow.com/questions/34588/how-do-i-change-the-number-of-open-files-limit-in-linux Wildcard syntax seems not to work (http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-exception-even-after-raising-the-open-file-limit-td2994940.html)