A whole year of development has gone into this new Graylog2 version. It is now multiple times faster in both throughput and searching. The CPU and RAM footprint has reduced and the new ElasticSearch multi-index model together with the possiblity to run multiple graylog2-server nodes in a cluster allows easy scaling. The new help pages are online and TORCH has launched, offering commercial services for Graylog2. Read about notable new features on the right side.
632 files changed, 32862 insertions(+), 15954 deletions(-)
So, what's new?
ElasticSearch multi-index storage
Old versions were storing all messages in one index which was hard to scale. The new Graylog2 version dynamically creates new indices when they reach a given size. You can define how many indices you want to keep in total and old indices are deleted automatically. There also is a recent index now that keeps only the most recent messages and allows to build fast overview pages. First setups just crossed the 1.000.000.000 messages mark.
Searches have been optimized in many ways. The server now keeps metadata about all indices it created and can limit timespan searches (which is the default now) to indices that actually contain messages in that timespan. A search on all messages of the last 24 hours will not take longer no matter if your overall message base keeps growing. The new version also introduces the universal search bar that accepts the full Lucene query syntax.
A brand new graylog2-server
Yes, graylog2-server has been completely rewritten and has a dramatically reduced CPU and RAM footprint. It is possible to run multiple graylog2-server instances in parallel to spread the load. You can just add as many graylog2-server as you wish on the fly. Stability has also been greatly improved
You can now extend graylog2-server with your own plugins! The first plugins are already available from the plugin directory. The following plugin types are supported: Initializer (called on startup), Message Input, Message Filter, Message Output (per stream), Alarm Callback (per stream - called when an alarm is triggered, useful to alert services like PagerDuty or chat rooms), Alarm Transport (per stream - a way to send alarms to users). All plugins are configurable via the web interface automatically and installed with just one command.
Improved AMQP subsystem
AMQP is the way to go if you need reliable delivery of huge amounts of messages. The new graylog2-server now brings fully featured AMQP support. You define which topic exchanges to bind on using which routing key in the web interface and the server will do the rest for you automatically. If you stop all graylog2-server instances the messages will just queue up until the first graylog2-server instance is started again. (You can define a TTL if you want) The full documentation is available here.
...and much more
Just to note a few more things: Writing internal metrics and stream counts to Graphite and Librato Metrics, Jabber/XMPP alarm transport now built in, alarms are now sent from the server automatically without having to call a rake task, dynamic search result histogram graphs, better tokenizer to allow easier searching, improved server health page, GELF HTTP input, support for uncompressed GELF, LDAP logins ...