Elasticsearch: Tutorial & Best Practices

Search and analytics engine

Elasticsearch is a free and open-source search and analytics engine based on the Lucene library. It is designed to provide fast and scalable search and analytics capabilities for large volumes of data.

Elasticsearch is distributed, meaning that it can scale horizontally across multiple servers or nodes. It also has a flexible, JSON-based query language and a rich set of APIs for indexing, searching, and managing data.

Elasticsearch is widely used for a variety of purposes, including full-text search, real-time analytics, and data visualization. It is often used in combination with other tools such as Logstash and Kibana to build powerful data processing and visualization pipelines.

Elasticsearch is a powerful and flexible tool for managing and analyzing large datasets, and it is widely used in organizations and businesses to improve the performance and scalability of their search and analytics systems.

To install and configure Elasticsearch under Linux, you will need to follow these steps:

• Install Elasticsearch: To install Elasticsearch on a Debian-based system (such as Ubuntu), you can use the following command:

  wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
  echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
  sudo apt-get update && sudo apt-get install elasticsearch
  

  To install Elasticsearch on a Red Hat-based system (such as CentOS), you can use the following command:

  sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
  echo "[elasticsearch-7.x]
  name=Elasticsearch repository for 7.x packages
  baseurl=https://artifacts.elastic.co/packages/7.x/yum
  gpgcheck=1
  gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
  enabled=1
  autorefresh=1
  type=rpm-md" | sudo tee /etc/yum.repos.d/elasticsearch.repo
  sudo yum install elasticsearch
  

• Configure Elasticsearch: The main configuration file for Elasticsearch is located at /etc/elasticsearch/elasticsearch.yml. This file contains a large number of options that you can use to configure the behavior of Elasticsearch.

  Here are a few basic options that you may want to configure:

  • cluster.name: This specifies the name of the Elasticsearch cluster.
  • node.name: This specifies the name of the Elasticsearch node.
  • network.host: This specifies the IP address or hostname that Elasticsearch will bind to.
  • http.port: This specifies the port number that Elasticsearch will listen on for HTTP requests.

  To configure Elasticsearch to start automatically when the system boots, you can use the following command on a Debian-based system:

  sudo systemctl enable elasticsearch
  

  To configure Elasticsearch to start automatically when the system boots on a Red Hat-based system, you can use the following command:

  sudo systemctl enable elasticsearch.service
  

• Start Elasticsearch: Once you have finished configuring Elasticsearch, you can start the service using the following command:

  sudo service elasticsearch start
  

  On some systems, you may need to use a different command to start Elasticsearch, such as systemctl start elasticsearch or /etc/init.d/elasticsearch start.

• Test Elasticsearch: To test that Elasticsearch is working correctly, you can send an HTTP request to the Elasticsearch API. To do this, you can use a tool such as curl.

  For example, to retrieve the Elasticsearch version information, you can use the following command:

  curl -X GET "localhost:9200"
  

  This will send a GET request to the Elasticsearch API, and it should return a JSON document containing the version information.

  If you encounter any errors or issues, you may need to check the Elasticsearch log files (located in /var/log/elasticsearch) for more information.

Add Elasticsearch repository:

Trust the repository key:

wget -qO - https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

Add the repository to /etc/apt/sources.list:

deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main

Update and install:

apt-get update
apt-get install elasticsearch

Configure /etc/elasticsearch/elasticsearch.yml and change the following entries:

cluster.name: mycluster
node.name: "mynode"
node.master: true
node.data: true
node.rack: myrack
network.host: 10.0.0.1
discovery.zen.ping.unicast.hosts: [ "myotherhost1", "myotherhost2" ]

Install elasticsearch-HQ plugin:

cd /usr/share/elasticsearch/bin
./plugin -install royrusso/elasticsearch-HQ

Start Elasticsearch:

/etc/init.d/elasticsearch start

Address in your browser http://myhost:9200/_plugin/HQ/

Enable IP-Forwarding in /etc/sysctl.conf to access the address from another network:

net.ipv4.ip_forward = 1