Elasticsearch

Elasticsearch

At Factweavers, we have been using this way right from its inception and have been one of the earliest adopters of Elasticsearch. We have seen Elasticsearch growing from a simple Lucene based search software to a critical log parsing platform along with the new members in its stack and also later on evolving to become one of the most used and sought out utility when it comes to search, even in the enterprise circles.

Our new initiatives on Elasticsearch provides you to start learning Elasticsearch right from its basics to advanced through a series of informative and hands-on blogs. So let us start our journey to learn and master Elasticsearch

 

What is meant by Elasticsearch

In layman’s terms, Elasticsearch is nothing but a database, with a plethora of search operations supported out of the box itself. To dig a little more deeper, Elasticsearch is an open source search engine, first developed by Shay Banon, on top of the search library Lucene. Elasticsearch is built in Java, and its main highlighting features includes

  • amazingly fast search capabilities within a wide range of use cases
  • distributed search on top of Lucene
  • Http based REST interface
  • Schema free
  • Near real time search capability

Due to these features Elasticsearch has seen an exponential increase on the interests, deployment and use cases in the last few years since its inception. Also the community has been growing with the same pace and has developed additional components like Kibana, Logstash, Beats to the Elasticsearch platform making the stack grow bigger and more capable over the years.

Elasticsearch Install

I have given you a brief introduction into what Elasticsearch is about and what are its capabilities in a very high level way. Now I think it is better to get down and get our hands dirty by doing some hands-on in Elasticsearch, meanwhile learning the very basics of Elasticsearch.

For this setup I have used an Ubuntu machine and Elasticsearch 6.x

Download the setup file from here. Since I am using Ubuntu, which is provided with two kinds of installation, the .deb file one and the .zip/tar kind, I prefer to use the later, since it is more convenient for us for development and trial purposes, also we can switch to any version according to our choice.
The version of ES, I am going to download is 6.4.0. For that you can use the below command too:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.0.tar.gz.sha512

 

Now unzip the above tar file using the following command

tar -xvzf elasticsearch-6.4.0.tar.gz.sha512

 

Now change the directory to the unzipped directory and run the following command.

./bin/elasticsearch

 

This will start ES in the port 9200, which happens to be its default port.
You can test whether ES is up and running by simply doing a curl request to the 9200 port from other terminal like below:

curl localhost:9200

The above command will return a response like below, which means that everything went fine and our ES is up and running.

 

{
 “name” : “node-1”,
 “cluster_name” : “elasticsearch”,
 “cluster_uuid” : “EY9YhLBnRb6RsrdzuOSA1w”,
 “version” : {
   “number” : “6.4.0”,
   “build_hash” : “260387d”,
   “build_date” : “2017-06-30T23:16:05.735Z”,
   “build_snapshot” : false,
   “lucene_version” : “7.4.0”
 },
 “tagline” : “You Know, for Search”
}

 

There are also other ways to try out ES like:

Elasticsearch AWS

Elasticsearch AWS, solution includes a half managed service by AWS which comes at a price. Here a lot of settings for fine tuning can be managed in the AWS console itself, but unlike most of the other services, Elasticsearch AWS is a paid one. For those interested can try it out here .More about AWS and the advantages it offer can be read from our blogs here

Elasticsearch Docker

Elasticsearch docker, comes in a variety of combination, it can be a docker instance which only has Elasticsearch, or it can come in with other elements of the stack like with Kibana and Logstash. Elasticsearch docker world is pretty flexible and one can easily get the version and the combination dockers from Docker hub. You can find the reasons to switch to dockers here and also get a basic introduction to dockers in this blog

What is Kibana and Elasticsearch

As mentioned in one of the sections above (What is meant by Elasticsearch), the Elasticsearch community has been very active with adding much useful components to Elasticsearch. And one of the addendums is Kibana, which has become one of the indispensable part of the Elastic stack. Let us have a brief look into what Kibana is on a very high level.

Kibana is the de-facto visualisation tool for Elasticsearch. Suppose we have some data in Elasticsearch and we need to quickly visualise it for analytics. It would be ideal, if this can be explained with an example. Suppose we have the apache logs parsed into Elasticsearch, which would give us a lot of information about the traffic of our website.

Also it would be not ideal to waste time on checking and going through each entry of the log to get an idea about the requests as it primarily take too much time and there are very good chances we might miss a lot of details.

So if we connect Kibana to Elasticsearch and load the default dashboard provided for the Apache logs, it would show us visualised statistics of the traffic to our website and from this graphical interface, it is easy for us to come into conclusions than merely going out and checking the logs directly.

What is the use of Kibana

The most important and the primary intent which was in mind when developing Kibana is that of a visualisation tool. As I have mentioned earlier, Kibana can be used to easily visualise a wide range of logs and it has default dashboards for the very common log formats such as Apache, Nginx etc. A sample Kibana dashboard for Apache logs would look like below:

During the evolution of Kibana, the capabilities of debugging was also added to it. The main debugging tools involve the “dev tools” console and the “Discover” tab. While the “dev tools” would provide us a way to query the data in the Elasticsearch cluster, the “Discover” dashboard can be extremely useful when we are dealing with new data sets. It can be used for simple querying and finding out the terms, phrases etc we are looking for are there in the database.

Also, apart from the Dev tools and the visualisations, Kibana is also used to host some part of the X-pack suite (which is an paid version offered by Elastic, which provides monitoring capabilities along with Role based access control for Elasticsearch).

Conclusion

In this blog, we have seen the what is meant by Elasticsearch, how to install it and also a brief introduction to its most sought out utility, the Kibana. In the coming blogs, we will see how ES works, the common queries, data ingestion and also use Kibana for debugging and querying exercises. In detail.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>