How to Install and Configure ELK stack on Fedora 32 using RPM method

Table of Contents

Test Environment

Fedora 32 installed

ELK Stack

ELK Stack is used to analyze and index large datasets of logs and provide insights. Elasticsearch manages the data, Logstash reads the data from different sources, Kibana makes fine visualization of it. ELK stack now as evolved as Elastic Stack. Elastic stack helps with pipeline and other tools to add like security, notification and monitoring capabilities to the setup.

LogStash

– Has plugins to read data from multiple sources and output plugins to submit data to various destinations
– Elasticsearch plugin helps to send data to elasticsearch
– Collects raw data and modifies or transforms it to structured, formatted and meaniful data
– Allows for custom plugins to be built

Elasticsearch

– Helps in indexing the data read from Logstash
– Its a full text search engine
– Provides tools to query, access and aggregate the data using the API’s
– Its based on the Apache Lucene search engine

Kibana

– Reads/queries data from elasticsearch indices using its API’s
– visualizes and generates graphs and charts for the data

Beats

– These are ligthweight and are installed as agents
– Reads data, parses it and ships it to either elasticsearch or logstash
– Metricsbeat, Filebeat and Packetbeat are some of the beats available
– libbeat is the library which can be used to write custom beat

Step1: Verify Java JDK installated or not

Make sure JDK is installed if not install the openjdk development package available on the linux systems.

javac -version
java -version

Step2: Install Elasticsearch

Import Elastcisearch GPG key, add the below elasticsearch repository and install the package.

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

File: /etc/yum.repos.d/elasticsearch.repo

[elasticsearch]
name=Elasticsearch repository for 7.x packages 
baseurl=https://artifacts.elastic.co/packages/7.x/yum 
gpgcheck=1 
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch 
enabled=0 
autorefresh=1 
type=rpm-md

sudo dnf install --enablerepo=elasticsearch elasticsearch

Step3: Enable and Start the Elasticsearch service

Let’s start and enable Elasticsearch service.

sudo systemctl daemon-reload
sudo systemctl  enable elasticsearch.service
sudo systemctl start elasticsearch.service

Step4: Verify that Elasticsearch is up and running with a GET request

Once the installation is completed and services are restarted, we should be able to get the response from elasticsearch running on port 9200 as shown below.

curl -X GET "localhost:9200/?pretty"

Output:

{
  "name" : "fedser32.stack.com",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "kmsW5BHgSc6r-1z5_hwZGA",
  "version" : {
    "number" : "7.13.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "4d960a0733be83dd2543ca018aa4ddc42e956800",
    "build_date" : "2021-06-10T21:01:55.251515791Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Step5: Install Kibana

Here let’s setup the kibana repository and install the respective package from it as shown below.

File: /etc/yum.repos.d/kibana.repo

[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

sudo dnf install kibana

Step6: Enable and Start Kibana service

Let’s enable and start the kibana service as shown below.

sudo systemctl daemon-reload
sudo systemctl enable kibana.service
sudo systemctl start kibana.service

Step7: Verify the Kibana service by launching the below URL

Once the installation is completed and services restarted, we should be able to launch the Kibana web portal with the following URL.

URL - http://FQDN:5601

Step8: Install Logstash

Let’s add the logstash repository and install the package for it.

File: /etc/yum.repos.d/logstash.repo

[logstash-7.x]
name=Elastic repository for 7.x packages 
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

sudo dnf install logstash

Step9 : Enable and Start logstash service

Here we will enabled and start the logstash service.

sudo systemctl daemon-reload
sudo systemctl enable logstash
sudo systemctl start logstash

Step10: Verify logstash installation

Verify the logstash installation by checking the version that is installed as shown below.

/usr/share/logstash/bin/logstash --version

Output:

Using bundled JDK: /usr/share/logstash/jdk
logstash 7.13.2

Step11: Install, Start and Enable Filebeat service

Here we will install, enable and start up the filebeat service as shown below.

sudo dnf install filebeat

sudo systemctl enable filebeat
sudo systemctl start filebeat

Verify the installation of filebeat as shown below.

/usr/share/filebeat/bin/filebeat version

Output:

filebeat version 7.13.2 (amd64), libbeat 7.13.2 [686ba416a74193f2e69dcfa2eb142f4364a79307 built 2021-06-10 21:04:13 +0000 UTC]

Hope you enjoyed reading this article. Thank you..

How to Install and Configure ELK stack on Fedora 32 using RPM method