How to Install Elasticsearch, Logstash, Filebeat, & Kibana on Ubuntu

ELK stack is the collection of open source products developed by the elastic. The following three things are performed:

  • Collection of data: The user collects the data from different sources
  • Analyze the data: processing the collected data
  • Visualization: denotes the representation of data

Prerequisites

  • Ubuntu Server 20.04 LTS
  • Java(JDK)
  • 2 CPU and 4 GB RAM
  • Ports 9200, 5601, 5044.

Install the required packages:

apt-get update
apt-get install openjdk-11-jdk wget apt-transport-https curl gnupg2 -y

Check Java version.

java -version

Here is the command output.

Fig 1

Install & Configure ElasticSearch

  • Add elasticsearch signing key & repository.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch --no-check-certificate
 | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" 
| sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

Here is the command output.

Fig 2

Update the  repository.

apt-get update

Install ElasticSearch

apt-get install elasticsearch -y

Open elesticsearch configuration file using vim editor.

vim /etc/elasticsearch/elasticsearch.yml

Uncomment & Provide the follwoing values:

network.host: localhost/0.0.0.0
http.port: 9200
discovery.type: single-node

Start & Enable ElasticSearch Service

systemctl start elasticsearch
systemctl enable elasticsearch

Check ElasticSearch Status.

systemctl status elasticsearch

Here is the command output.

Fig. 4

Check elasticsearch service pid using command line.

ss -antpl | sudo grep 9200

Here is the command output.

Fig. 3

Check elasticsearch is running by sending HTTP request.

Fig. 6

We can also check elasticsearch is running status using web browser(http://localhost:9200) or (http://server-ip:9200)

Fig. 6

 

Install and Configure Kibana

  • Install kibana on Ubuntu.
apt-get install kibana

Open kibana configuration file using editor.

vim /etc/kibana/kibana.yml

Uncomment & Edit the config file.Provide the follwoing values.

server.port: 5601
server.host: "0.0.0.0" or "localhost"
elasticsearch.hosts: ["http://localhost:9200"] or ["http://0.0.0.0:9200"]

Start & Enable Kibana Service

systemctl start kibana
systemctl enable kibana

Check the Kibana Status

systemctl status kibana

Here is the command output.

Fig. 7

Install and Configure Logstash

  • Install logstash on ubuntu.
apt-get install logstash
  • Create config file.
vim /etc/logstash/conf.d/02-beats-input.conf
  • Enter the follwoing lines.
input {

  beats {

    port => 5044

  }

}
  • Create the logstash configuration file to send the logs.
vim /etc/logstash/conf.d/30-elasticsearch-output.conf
  • Enter the follwoing lines.
output {

  elasticsearch {

    hosts => ["localhost:9200"] or ["0.0.0.0:9200"]

    manage_template => false

    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

  }

}
  • Start & Enable Logstash service.
systemctl start logstash
systemctl enable logstash
  • Check Logstash status.
systemctl status logstash

Here is the command output.

Fig. 9

 

Install and Configure Filebeat

  • Install Filebeat on ubuntu.
apt-get install filebeat
  • Open filebeat configuration file.
vim /etc/filebeat/filebeat.yml
  • Comment the below lines.
#output.elasticsearch:
  # Array of hosts to connect to.
#  hosts: ["localhost:9200"]
  • Uncomment the below lines.
output.logstash:
hosts: ["localhost:5044"] or ["0.0.0.0:5044"]
  • Start & Enable filebeat service.
systemctl start filebeat
systemctl enable filebeat
  • Check Filebeat status.
systemctl status filebeat

Here is the command output.

Fig 10

  • Enable filebeat system module.
filebeat modules enable system

Here is the command output.

Fig 11

  • Check that ElasticSearch is receiving datalog from filebeat using below command.
curl -XGET http://localhost:9200/_cat/indices?v

Here is the command output.

Fig 12

Access Kibana Web Interface

  • Access Kibana Web Interface by using the URL.
http://your-server-ip:5601

Fig. 8

 

Leave a Reply