Elasticsearch is an allocated search and analytics engine based on the Lucene library. Logstash and Beats help to get, aggregate, and improve your data and store it in Elasticsearch. Kibana allows you to interactively search, see, and distribute insights into your data and control and watch the stack: Elasticsearch is where the indexing, search, and analysis magic happen.

1- How does Elasticsearch work?

Elasticsearch provides real-time search and analytics for all sorts of data. Whether you have structured or unstructured text, numerical data, or geospatial data, Elasticsearch can efficiently store and index it in a way that helps fast searches. You can go far beyond simple data retrieval and aggregate information to discover trends and patterns in your data. And as your data and query volume grows, the distributed nature of Elasticsearch enables your deployment to grow seamlessly right along with it.

While not all problems are search problem, Elasticsearch offers speed and flexibility to handle data in a wide variety of use cases:

Add a search box to an app or website

Store and analyze logs, metrics, and security event data

Use machine learning to automatically model the behaviour of your data in real-time

Automate business workflows using Elasticsearch as a storage engine

Manage, integrate, and analyze spatial information using Elasticsearch as a geographic information system (GIS)

Store and process genetic data using Elasticsearch as a bioinformatics research tool

We’re continually amazed by the novel ways people use to search. But whether your use case is similar to one of these, or you’re using Elasticsearch to tackle a new problem, the way you work with your data, documents, and indices in Elasticsearch is the same.

2- Keepnet Labs Threat Intelligence and Elasticsearch – Keepnet Labs Elasticsearch

Collecting threat intelligence to prevent future crimes has been an important method in the information security industry. Elasticsearch gives you an opportunity to manage these intelligence data for threat intelligence purposes. It is possible to utilise Logstash, Elasticsearch, and Kibana in working with threat intelligence.

One of the benefits is to make use of translate filters in logstash and alert on data that corresponds to data in blacklists. You can also ingest the data into Elasticsearch and then output it in CSV and make use of the CSV file in a translate filter. This is where the translate filter checks the CSV if it includes a value and then, performs an action if the CSV includes the value. Reasonably, you just want to fill Elasticsearch with possible email addresses that could be sending you phishing emails, blacklisted IPs etc. This would allow you to search for an IP and help make an informed decision whether connections to or from the IP could potentially be malicious. Elasticsearch has been important for Keepnet Labs for in this aspect.