Elastic Search

Technical abilities –  Literally any coding/scripting language that does the CURD operations with the Elastic search REST API. Good understanding of JSON.

Elastic search, as the name indicates, this is a search engine which provides us with an opportunity to organize the data and later search that using Query Domain Specific Language.

Terminology –

  • Cluster – A group of nodes combines to form a cluster.
  • Node – Think about it as a single server.

Legend –

Elastic Search – ES

Firstly, let’s get through the organizing data part. All the data in the elastic search is formatted in JSON. So let’s say you have vivid logs you want to monitor, example – Web Server logs, application server logs, HVLM logs, etc. So, the first thing you do here is to create an index named logs in the elasticsearch using the elastic search API and then, you would be creating the web server logs, app server logs, hvlm logs documents which are formatted in JSON under the doctype name, ex- region name here, the whole thing would be under the index – logs. By default, the ES creates an index with 5 shards and 1 replica. When you have too many documents on a single index in real time, ES splits the documents on to individual nodes, which are called as shards. All the doc’s under the index have at least one replica, for high availability and performance.

Now, when you have your index ready with the doc’s under it. It’s time for you to perform the searches. Elasticsearch uses the Term Frequency/Inverse Document Frequency relevance algorithm to do the search based on the relevance score. It uses a Domain Specific Language Query model which is in JSON format to do the search. Using the wide variety of function provided by ES you can do your search based on the relevance score. If you are not keen on the relevance score, then you could simply use the filter context search.

Why is Elastic search preferred?

  • Open source
  • High performance and availability
  • Provides a service in the cloud
  • Easy to use
  • Provides searches based on relevance algorithm

Security –

At the inception of elastic search there was no security feature that was in-built within the product. So, many firms started to put a web server in front of the ES and started to filter it based on IP’s, users, etc. But, this has been tedious as the infrastructure was growing too large in size, which is not recommended.

Elastic then released a security feature for their product called SHIELD. You just have to install this plugin to use it. ‘elasticsearch.yml’ is the configuration file that handles all the security for you. Here you could mention the IP filtering, integrate it with LDAP for user id authentications and even do SSL authentication. The roles for the users is maintained in a file called ‘roles.yml’.

But, if you want to try this on your home PC’s, try using ‘esusers’, a simple config file where you can have the users list.

Published by Ritesh Kumar Reddy

I(Ritesh) work as a Sr. Cloud Engineer for a living. Learning new technologies has always been my hobby. Why not share it? Here is the brainchild – blogging to share the knowledge. This blog is for those who wish to start or already into the Cloud field. Each article briefly talks about a tool/technology that is used in the Cloud model. Once you read the article, I hope, you get a kick start regarding the specific tool/technology.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: