How to Install and Configure Elasticsearch on Ubuntu 16.04 or 18.04
This post will guide you how to install, configure, secure, and use an Elasticsearch server on your Ubuntu Linux 16.04 or 18.04. How do I install one of the most popular text indexing and search engine, Elasticsearch on Ubuntu system.
- What is ElasticSearch?
- Prerequisites
- Step1: Installing JAVA
- Step2: Downloading Elasticsearch Package
- Step3: Installing Elasticsearch
- Step4: Configuring Elasticsearch
- Step5: Securing Elasticsearch with UFW Tool
- Step6: Using Elasticsearch
What is ElasticSearch?
Elasticsearch is an open source search engine based on the Lucene library. And it provides a distributed, multitenant-capable full-text search engine with an HTTP web interface (RESTful) and schema-free JSON documents. So you can use HTTP methods (GET, POST, PUT, DELETE, etc.) in combination with an HTTP URI to operate your data.
Elasticsearch is the most popular enterprise search engine followed by Apache Solr, also based on Lucene. And it can be used to search for all kind of text documents or analyzing big volumes of data in real time.
Elasticsearch is also one of the most popular NoSQL databases which is used to store and search for text based data.
Elasticsearch is developed in JAVA. It is freely available under the Apache 2 license, so it provide the most flexibility.
Prerequisites
You need to install the latest JDK version on your Ubuntu Linux, because Elasticsearch is developed in JAVA program language, and it need to have a JAVA Runtime Environment.
You will need to be logged in as a user with sudo privileges, so that you can install packages with dpkg or apt command on your Ubuntu system.
Step1: Installing JAVA
As we talked above, you need to firstly install JDK 8 package or high version on your Ubuntu system for JAVA Runtime Environment (JRE). You can install either the Oracle JAVA Standard Edition 8 or OpenJDK 8 on the system.
The OpenJDK 8 is already available in the default Ubuntu APT repository. So we try to install the OpenJDK 8 on this guide, just type the following command to update the packages index and installing OpenJDK 8:
$ sudo apt update $ sudo apt install openjdk-8-jdk
Outputs:
devops@devops:~$ sudo apt install openjdk-8-jdk Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: ca-certificates-java libxt-dev openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless Suggested packages: libxt-doc openjdk-8-demo openjdk-8-source visualvm icedtea-8-plugin fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei fonts-wqy-zenhei The following NEW packages will be installed: ca-certificates-java libxt-dev openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 0 upgraded, 6 newly installed, 0 to remove and 45 not upgraded. Need to get 37.5 MB/37.5 MB of archives. After this operation, 143 MB of additional disk space will be used. Do you want to continue? [Y/n] Y
After JDK package is installed, you can check the installation of JDK package by print the version of JDK installed, type:
$ java -version
Outputs:
devops@devops:~$ java -version openjdk version "1.8.0_191" OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12) OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
From the above outputs, you would see that the OpenJDK Runtime Environment is available. You can also install Oracle JDK 8 to set up JRE.
Note: If you want to install the Oracle JAVA 8 on your Ubuntu system, just issue the following commands:
$ sudo add-apt-repository ppa:webupd8team/java $ sudo apt update $ sudo apt install oracle-java8-installer –y
Step2: Downloading Elasticsearch Package
Once JAVA is installed on your system, and then you can download installation package of Elasticsearch to your local disk.
You can download the latest Elasticsearch archive file from official download page of elastic. Or you can also download directly Deb package of Elasticsearch from the official download page for your Ubuntu system.
Using one of the following command to get the Elasticsearch package, type:
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-linux-x86_64.tar.gz
Or
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-amd64.deb .
Outputs:
[devops@devops ~]$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-linux-x86_64.tar.gz
--2019-04-23 06:41:12-- https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-linux-x86_64.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.110.222, 2a04:4e42:1a::734
Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.110.222|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 346760090 (331M) [application/x-gzip]
Saving to: ‘elasticsearch-7.0.0-linux-x86_64.tar.gz’
100%[======================================================================================================================>] 346,760,090 6.35MB/s in 58s
2019-04-23 06:42:11 (5.72 MB/s) - ‘elasticsearch-7.0.0-linux-x86_64.tar.gz’ saved [346760090/346760090]
devops@devops:~$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-amd64.deb . --2019-04-24 17:53:39-- https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-amd64.deb Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.110.222, 2a04:4e42:1a::734 Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.110.222|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 346791426 (331M) [application/octet-stream] Saving to: ‘elasticsearch-7.0.0-amd64.deb’ elasticsearch-7.0.0-amd64.deb 100%[==========================================================================>] 330.73M 3.06MB/s in 77s 2019-04-24 17:54:57 (4.31 MB/s) - ‘elasticsearch-7.0.0-amd64.deb’ saved [346791426/346791426] --2019-04-24 17:54:57-- http://./ Resolving . (.)... failed: Name or service not known. wget: unable to resolve host address ‘.’ FINISHED --2019-04-24 17:54:57-- Total wall clock time: 1m 18s Downloaded: 1 files, 331M in 1m 17s (4.31 MB/s) devops@devops:~$ ls elasticsearch-7.0.0-amd64.deb elasticsearch-7.0.0-amd64.deb
Note: At the time of writing this post, the latest version of Elasticsearch is 7.0.
Step3: Installing Elasticsearch
You can install Elasticsearch through either archive file or deb package downloaded in the above step. For Ubuntu, it is best to use the deb package which will install everything you need to run Elasticsearch.
For Elasticsearch archive file, you just need to extract it on your system with the following command:
$ tar -zxvf elasticsearch-7.0.0-linux-x86_64.tar.gz
For Elasticsearch Deb package, you can install it with dpkg command, type:
$ sudo dpkg –i elasticsearch-7.0.0-amd64.deb
Ouputs:
devops@devops:~$ sudo dpkg -i elasticsearch-7.0.0-amd64.deb
[sudo] password for devops:
Selecting previously unselected package elasticsearch.
(Reading database ... 322964 files and directories currently installed.)
Preparing to unpack elasticsearch-7.0.0-amd64.deb ...
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Unpacking elasticsearch (7.0.0) ...
Setting up elasticsearch (7.0.0) ...
Created elasticsearch keystore in /etc/elasticsearch
Processing triggers for systemd (237-3ubuntu10.19) ...
Processing triggers for ureadahead (0.100.0-20) ...
Elasticsearch will be installed in /usr/share/elasticsearch directory, and its configuration files locates in /etc/elasticsearch directory. And its init script is added into /etc/init.d/ directory.
Once the installation process is complete, you need to start and enable elaticsearch service by the following command:
$ sudo systemctl start elasticsearch.service $ sudo systemctl enable elasticsearch.service
Outputs:
devops@devops:~$ sudo systemctl start elasticsearch.service devops@devops:~$ sudo systemctl enable elasticsearch.service Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable elasticsearch Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /usr/lib/systemd/system/elasticsearch.service.
Then you can send an HTTP GET request to the default port 9200 of Elasticsearch service using Curl command, type:
$ curl -X GET "localhost:9200/"
Outputs:
devops@devops:~$ curl -X GET "localhost:9200/"
{
"name" : "devops",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "sGNB30MGRCKvJnKKaREKXg",
"version" : {
"number" : "7.0.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "b7e28a7",
"build_date" : "2019-04-05T22:55:32.697037Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
If you get the output similar to the above, it indicates that Elasticsearch server is successfully installed on your Ubuntu or Debian Linux.
Step4: Configuring Elasticsearch
Now that Elasticsearch and its Java dependencies have been installed. So you can try to configure Elasticsearch, such as: changing node.name or cluster.name through modifying the Elasticsearch configuration files elasticsearch.yml located in /etc/elasticsearch/ directory.
If you do not change those two variables (node.name and cluster.name), node.name will be assigned automatically with the default hostname. And the cluster.name variable will be automatically set to the name of default cluster as “elasticsearch”.
To start editing the main elasticsearch.yml configuration file with vi/vim text editor to change node.name as osetcnode, and changing cluster name as osetccluster:
$ sudo vim /etc/elasticsearch/elasticsearch.yml
You need to remove the # character at the beginning of the lines for cluster.name and node.name variables to uncomment them, and then update their values as you need. Like as below:
cluster.name: osetccluster node.name: osetcnode
Save and close the file. And then restart the elasticsearch service by the following command:
$ sudo systemctl restart elasticsearch.service
Then you can test it again by sending a HTTP GET request with curl command to check if those two variables are changed, type:
$ curl -X GET 'http://localhost:9200'
Outputs:
devops@devops:~$ curl -X GET "localhost:9200/"
{
"name" : "osetcnode",
"cluster_name" : "osetccluster",
"cluster_uuid" : "sGNB30MGRCKvJnKKaREKXg",
"version" : {
"number" : "7.0.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "b7e28a7",
"build_date" : "2019-04-05T22:55:32.697037Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
From the above outputs, you would see that node name and cluster name have been changed.
Note: the elasticsearch service will take at least 5 seconds to fully start. So if you see the following error message: curl: (7) Failed connect to localhost:9200; Connection refused, and you just wait a few seconds and try it again.
Step5: Securing Elasticsearch with UFW Tool
As Elasticsearch do not provide any security, so it can be accessed by anyone who can access the HTTP API. So if you want to allow the specified remote hosts to access your Elasticsearch server, you need to change network.host variable so that Elasticsearch service can only be accessed from those remote hosts.
Using vi/vim text editor or your favorite text editor to edit elasticsearch.yml configuration file, and finding the line that contains network.host, and uncomment it by removing the # character at the beginning of the line, and change its value as IP address of your remote host.
network.host: 192.168.3.29 #listening on specified remote host
Note: if you want Elasticsearch listens on all interfaces and bound IPs, you just need to specify 0.0.0.0 value for network.host variable.
Save and close the file, restart the Elasticsearch service for the changes to take effect, type:
$ sudo systemctl restart elasticsearc.service
You still need to allow access the default port 9200 from the trusted clients, so you need to add the trusted hosts into your firewall policy with Ubuntu’s default firewall, UFW (Uncomplicated Firewall).
By default, UFW firewall is already installed on your Ubuntu system, but it is not enabled. Before enabling the UFW firewall, you need to firstly create a rule to allow any needed services, such as: ssh service.
To allow incoming SSH connections, you need to add port 22 into whitelist. Type:
$ sudo ufw allow 22
Outputs:
devops@devops:~$ sudo ufw allow 22 [sudo] password for devops: Rules updated Rules updated (v6)
Then you need to create another new rule to allow access to the default service port of Elasticsearch 9200 for the trusted remote host (assuming that the ip address is 192.168.3.22 ), running the following command:
$ sudo ufw allow from 192.168.3.22 to any port 9200
Outputs:
devops@devops:~$ sudo ufw allow from 192.168.3.22 to any port 9200 Rules updated
Enable UFW with the following command:
$ sudo ufw enable
Outputs:
devops@devops:~$ sudo ufw enable Command may disrupt existing ssh connections. Proceed with operation (y|n)? y Firewall is active and enabled on system startup
Finally, you can try to check the status of UFW to see if those new rules take effect, type:
$ sudo ufw status
Outputs:
devops@devops:~$ sudo ufw status Status: active To Action From -- ------ ---- 22 ALLOW Anywhere 9200 ALLOW 192.168.3.22 22 (v6) ALLOW Anywhere (v6)
From the above outputs, you would see that the trusted remote host 192.168.3.22 to access port 9200 is in the Allow action list.
Step6: Using Elasticsearch
So far, Elasticsearch should be running on default port 9200 successfully. And You can try to add, read,delete or update the data to your Elasticsearch server using RESTful API with Curl command.
Adding Data to Elasticsearch server
You can run the following Curl command to add your first data into Elasticsearch server, type:
$ curl -H "Content-Type: application/json" -X POST 'http://localhost:9200/osetc/howto/1' -d '{
"Title" : "Installing Elasticsearch",
"Date" : "Jan 2020",
"Tag" : "server,CentOS,Linux"
}'
Outputs:
devops@devops:~$ curl -H "Content-Type: application/json" -X POST 'http://localhost:9200/osetc/howto/1' -d '{
> "Title" : "Installing Elasticsearch",
> "Date" : "Jan 2020",
> "Tag" : "server,CentOS,Linux"
> }'
{"_index":"osetc","_type":"howto","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
The URI of the request was /osetc/howto/1 with several parameters:
- Osetc is the index of the data in Elasticsearch server.
- Howto is the type.
- 1 is the id of entry under the above index and type.
Reading Data from Elasticsearch
You can try to retrieve the above entry with an HTTP GET request using Curl command, type:
$ curl -X GET 'http://localhost:9200/osetc/howto/1’
Outputs:
devops@devops:~$ curl -X GET 'http://localhost:9200/osetc/howto/1'
{"_index":"osetc","_type":"howto","_id":"1","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{
"Title" : "Installing Elasticsearch",
"Date" : "Jan 2020",
"Tag" : "server,CentOS,Linux"
}}
Removing Data from Elasticsearch
If you want to remove above data entry from Elasticsearch, just type the following curl command:
$ curl -X DELETE 'http://localhost:9200/osetc/howto/1'
Outputs:
devops@devops:~$ curl -X DELETE 'http://localhost:9200/osetc/howto/1'
{"_index":"osetc","_type":"howto","_id":"1","_version":2,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}
Conclusion
You should know that how to install and configure ElasticSearch to set up a distributed search engine on your Ubuntu 16.04 or 18.04 or Debian Linux. If you want to get more detail document about Elasticsearch, you can directly go to its official web site.
