Technical abilities – Literally any coding/scripting language that does the CURD operations with the Elastic search REST API. Good understanding of JSON.
Elastic search, as the name indicates, this is a search engine which provides us with an opportunity to organize the data and later search that using Query Domain Specific Language.
Terminology –
- Cluster – A group of nodes combines to form a cluster.
- Node – Think about it as a single server.
Legend –
Elastic Search – ES
Firstly, let’s get through the organizing data part. All the data in the elastic search is formatted in JSON. So let’s say you have vivid logs you want to monitor, example – Web Server logs, application server logs, HVLM logs, etc. So, the first thing you do here is to create an index named logs in the elasticsearch using the elastic search API and then, you would be creating the web server logs, app server logs, hvlm logs documents which are formatted in JSON under the doctype name, ex- region name here, the whole thing would be under the index – logs. By default, the ES creates an index with 5 shards and 1 replica. When you have too many documents on a single index in real time, ES splits the documents on to individual nodes, which are called as shards. All the doc’s under the index have at least one replica, for high availability and performance.
Now, when you have your index ready with the doc’s under it. It’s time for you to perform the searches. Elasticsearch uses the Term Frequency/Inverse Document Frequency relevance algorithm to do the search based on the relevance score. It uses a Domain Specific Language Query model which is in JSON format to do the search. Using the wide variety of function provided by ES you can do your search based on the relevance score. If you are not keen on the relevance score, then you could simply use the filter context search.
Why is Elastic search preferred?
- Open source
- High performance and availability
- Provides a service in the cloud
- Easy to use
- Provides searches based on relevance algorithm
Security –
At the inception of elastic search there was no security feature that was in-built within the product. So, many firms started to put a web server in front of the ES and started to filter it based on IP’s, users, etc. But, this has been tedious as the infrastructure was growing too large in size, which is not recommended.
Elastic then released a security feature for their product called SHIELD. You just have to install this plugin to use it. ‘elasticsearch.yml’ is the configuration file that handles all the security for you. Here you could mention the IP filtering, integrate it with LDAP for user id authentications and even do SSL authentication. The roles for the users is maintained in a file called ‘roles.yml’.
But, if you want to try this on your home PC’s, try using ‘esusers’, a simple config file where you can have the users list.
TechnologyTickles 