In this post we will Install Elasticsearch on Ubuntu 18.04.1 including Logstash and Kibana. Elasticsearch lets you search and visualize your data.
Prepare
We will begin by starting with a fresh installation of Ubuntu Server 18.04.1 and running all the updates.
I created a virtual machine with 8 vCPUs, 4 GB of memory, and 200 GB of drive space. I also setup an A record on my internal DNS that set the logging host to 192.168.1.15.
If you would like to know how to install Ubuntu Server 18.04.1 I have a video available on my YouTube Channel: AdminTome Blog TV – Installing Ubuntu Server 18.04.1 Tutorial Video
During the install I set the host name as logging.admintome.lab and set the static IP to the IP mentioned above.
After Ubuntu Server is finished installing, we need to install all the updates.
apt update && apt upgrade -y
When it is completed updating, go ahead and reboot the system.
reboot
Elasticsearch requires the Java SDK 8 be installed.
Run these commands to install the Java 8 SDK.
add-apt-repository ppa:webupd8team/java apt install -y oracle-java8-set-default
You can verify that we have Java 8 installed by running this command:
# java -version java version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
We are now ready to download and Install Elasticsearch.
Install Elasticsearch
We are going to install the latest version of Elasticsearch which as of this writing is 6.3.2.
If you want to check for the latest version go to the Elastic Downloads page.
We will want to download the .DEB package.
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.deb
When it is done downloading, install it using DPKG:
dpkg -i elasticsearch-6.3.2.deb
After it is done installing open /etc/elasticsearch/elasticsearch.yml and edit this line:
# network.host: 192.168.0.1
Uncomment it and set the IP to your server.
network.host: 192.168.1.15
Save and exit the file.
Finally, start and enable the Elasticsearch service.
systemctl enable elasticsearch.service systemctl start elasticsearch.service
Verify that everything is working by browsing to this url:
http://:9200/_cat/health?v
You should see a page similar to this.
We can see that our node status is green which is great.
Next we will install Kibana which will give us a graphical font-end.
Install Kibana
We will download and install Kibana exactly the same way we installed Elasticsearch.
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.3.2-amd64.deb dpkg -i kibana-6.3.2-amd64.deb
Next, open /etc/kibana/kibana.yml and update these two lines. Uncommenting as necessary.
server.host: "192.168.1.15" elasticsearch.url: "http://192.168.1.15:9200"
Save and exit.
Configure the VM Heap Size for JVMs. We only have to do this now so we don’t have to reboot.
After a reboot this setting will be configured for us already.
sysctl -w vm.max_map_count=262144
Finally, start and enable the Kibana service.
systemctl enable kibana.service systemctl start kibana.service
Browse to this URL:
http://:5601
And you will see the Kibana Dashboard.
Our last step is to install Logstash.
Install Logstash
Logstash is the agent that will put our data into Elasticsearch and displayed using Kibana.
As with the other applications, we will download and install Logstash in the same manner.
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.3.2.deb dpkg -i logstash-6.3.2.deb
Open the vim /etc/logstash/logstash.yml file and change this setting:
http.host: "192.168.1.15"
Save the file and exit.
Start and enable the Logstash service.
systemctl enable logstash.service systemctl start logstash.service
Logstash is now installed. Next we will put some data into Elasticsearch using FileBeat.
Installing FileBeats
First, lets change to the logstash installation directory.
cd /usr/share/logstash/
Run the following command to install the FileBeat Logstash plugin:
bin/logstash-plugin install logstash-input-beats
Now we will configure Logstash to receive FileBeat data.
Create /etc/logstash/conf.d/beats.conf and add the following contents.
input {
beats {
port => "5044"
}
}
output {
elasticsearch {
hosts => ["192.168.1.15:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
Be sure to set your actual IP address in the hosts line.
Save and exit the file.
Sending Logs
For this tutorial, we are going to configure the Elasticsearch server to send it’s own logs to demonstrate how to configure your systems to send logs to the server.
If you want to configure sending logs from another server, follow these steps on the other server.
Download and install the FileBeat agent just like we have done for the other applications.
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-amd64.deb dpkg -i filebeat-6.3.2-amd64.deb
Next, we need to configure it.
Open up /etc/filebeat/filebeat.yml and configure it to look like this:
filebeat.prospectors:
- input_type: log
paths:
- /var/log/*/*.log
output.logstash:
hosts: ["192.168.1.15:5044"]
This will configure FileBeat to send logs from /var/log/ to our Elasticsearch server on port 5044 ( The port we configured in the last section).
Start and enable the FileBeat service.
systemctl enable filebeat.service systemctl start filebeat.service
Finally, we need to install the FileBeat template to Kibana. You only have to do this once if you are setting up logging for multiple servers using FileBeat.
filebeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'
Wait a couple minutes for the logs to end up in Elasticsearch then go back to Kibana.
Click on the Management menu item on the left.
Next click on Index Patterns.
You should see our filebeat index pattern listed.
In the index pattern type in filebeat-* as shown here.
Click the Next step button.
In the next step, select @timestamp for the Time Filter field name.
Finally, click on the Create index pattern button.
Click on the Discover menu item on the left and you will see your logs.
Follow these same steps to get logs from other systems into Elasticsearch / Kibana.
Conclusion
In this post, we learned how to install Elasticsearch on Ubuntu 18.04.1.
I hope that you enjoyed this post and it was helpful.
If it was then please share this post and comment below.
Be sure to subscribe to the AdminTome Blog Newsletter and the Admintome Blog TV YouTube Channel.
The post Install Elasticsearch on Ubuntu 18.04.1 appeared first on AdminTome Blog.
