Ataxo Social Insider
Ataxo Social Insider is a web application to monitor and evaluate communication in social networks such as Twitter, Facebook, blogs and other online media in Central Europe. The back-end system, implemented with RabbitMQ and custom Python libraries, retrieves, stores and indexes data from various online services. The front-end system, a Ruby On Rails application, continuously retrieves these „mentions“, based on user preferences („keywords“), from the back-end.
ElasticSearch is used both at the back-end (migrating from Solr) and the front-end (which previously relied on CouchDB-Lucene integration; see our CouchDB case study). All the front-end application logic: filtering by keywords, sources, dates, fulltext searching, computing aggregated metrics and displaying records is powered by ElasticSearch (via the Tire gem for Ruby). In fact, we're using ElasticSearch as a searchable database with powerful aggregation features. The data are being duplicated in CouchDB for durability and redundancy.
ElasticSearch has enabled us to implement some unique and competitive features, which would be hard or impossible to achieve otherwise:
- Users can use any valid Lucene query syntax for defining the „keyword”, such as
apple AND NOT iphone
. They're able to fine-tune already defined keywords in this way, and see the impact of their changes immediately.
- We're able to display real-time, rich, interactive visualization of various metric for currently displayed data, without having to write custom map/reduce functions. See the article on the ElasticSearch's blog for details.
- Using the percolator feature, we can offer users real-time alerts, where notifications are triggerred by a certain amount of mentions retrieved within certain period.
- Last, but not least, we're able to retrieve data very fast. Most searches take around 5 milliseconds in ElasticSearch and most pages load under 1 second, for the data set in range of millions of documents and tens of gigabytes.