How to install and configure Fluent Bit using Ansible

Here in this article we will try to install and configure fluent-bit using ansible playbook and demonstrate the default cpu metrics collection and processing that is carried out by the fluent-bit engine.

Table of Contents

Test Environment

Fedora 39 workstation
Ansible 2.16.2

What is Fluent Bit

As stated in its documentation, Fluent Bit is a super fast, lightweight, and highly scalable logging and metrics processor and forwarder. It is the preferred choice for cloud and containerized environments. Fluent Bit is an Cloud Native Computing Foundation graduated project under the umbrella of Fluentd.

It is an open-source telemetry agent specifically designed to efficiently handle the challenges of collecting and processing telemetry data across a wide range of environments, from constrained systems to complex cloud infrastructures. Fluent Bit has been designed with performance and low resource consumption in mind.

Features of Fluent Bit

Scalable – Proven across distributed cloud and container environments. Highly available with I/O handlers to store data for disaster recovery
Flexible – Granular management of data parsing and routing. Filtering and enrichment to optimize security and minimize cost
Efficient – Lightweight, asynchronous design optimizes resource usage: CPU, memory, disk I/O, network. No more OOM errors
Extensible – Integration with all your technology – cloud native services, containers, streaming processors, and data backends

Fluent Bit always handles every Event message as a structured message. For performance reasons, we use a binary serialization data format called MessagePack.

Fluent Bit Data Pipeline

Input – Fluent Bit provides different Input Plugins to gather information from different sources, some of them just collect data from log files while others can gather metrics information from the operating system
Parser – Fluent Bit provides different Parser Plugins which help in converting the unstructured event data into a strucutred format
Filtering – It is an important feature that allows us to alter the data before delivering it to some destination
Buffer – The buffer phase in the pipeline aims to provide a unified and persistent mechanism to store your data, either using the primary in-memory model or using the filesystem based mode
Routing – Routing is a core feature that allows to route your data through Filters and finally to one or multiple destinations. It is implemented based on Tags and Matching rules
Output – The output plugins acts as an interface to our destination where we want to send the processed events. Common destinations are remote services, local file system or standard interface with others

Here is the project structure for fluent-bit setup.

admin@fedser:fluentbit$ tree .
.
├── inventory
│   ├── hosts
│   └── hosts_template
├── linux_setup_fluentbit.yml
├── README.md
└── roles
    ├── linux_configure_fluentbit
    │   ├── defaults
    │   │   └── main.yml
    │   ├── tasks
    │   │   └── main.yml
    │   └── templates
    │       └── fluent-bit.conf
    ├── linux_install_fluentbit
    │   └── tasks
    │       └── main.yml
    ├── linux_ping
    │   └── tasks
    │       └── main.yml
    ├── linux_restart_fluentbit
    │   └── tasks
    │       └── main.yml
    ├── linux_start_fluentbit
    │   └── tasks
    │       └── main.yml
    └── linux_stop_fluentbit
        └── tasks
            └── main.yml

NOTE: Role “linux_ping” can be used to validate the ssh connectivity with the managed hosts as per the inventory file. Here are the details.

admin@fedser:fluentbit$ cat roles/linux_ping/tasks/main.yml 
- name: ansible ping pong validation
  ping:

admin@fedser:fluentbit$ cat inventory/hosts
[fluent-bit]
fedres.stack.com

If you are interested in watching the video. Here is the YouTube on the same step by step procedure outlined below.

Update 1: By default fedora provides us with a fluent-bit package to install from the default configured repositories. There is no need to add the fluent-bit repo separately as done in the below playbook

Procedure

Step1: Install fluent-bit

The following role “linux_install_fluentbit” will setup the fluent-bit repository and accept the gpg key for the repository. Once repository setup is done it will carry out the fluent-bit package installation as shown below.

admin@fedser:fluentbit$ cat roles/linux_install_fluentbit/tasks/main.yml 
---
#- name: Add fluentbit repository
#  yum_repository:
#    name: fluent-bit
#    description: fluent-bit repo
#    baseurl: https://packages.fluentbit.io/centos/$releasever/
#    gpgkey: https://packages.fluentbit.io/fluentbit.key
#    gpgcheck: true
#    repo_gpgcheck: true
#    enabled: true

- name: Install fluent-bit
  dnf:
    name: fluent-bit
    state: present

Step2: Configure fluent-bit

As a part of this step the following role “linux_configure_fluentbit” will help upload the template file “fluent-bit.conf” to the following default location “/etc/fluent-bit/fluent-bit”. Here are the details.

admin@fedser:fluentbit$ cat roles/linux_configure_fluentbit/templates/fluent-bit.conf 
[SERVICE]
    flush        1
    daemon       Off
    log_level    info
    parsers_file parsers.conf
    plugins_file plugins.conf
    
    http_server  Off
    http_listen  0.0.0.0
    http_port    2020

    storage.metrics on

[INPUT]
    name cpu
    tag  cpu.local

    # Read interval (sec) Default: 1
    interval_sec 1

[OUTPUT]
    name  stdout
    match *

admin@fedser:fluentbit$ cat roles/linux_configure_fluentbit/defaults/main.yml 
---
fluentbit_config_directory: /etc/fluent-bit
fluentbit_config_template: fluent-bit.conf

admin@fedser:fluentbit$ cat roles/linux_configure_fluentbit/tasks/main.yml 
- name: "upload fluent-bit config"
  template:
    src: "{{fluentbit_config_template}}"
    dest: "{{fluentbit_config_directory}}/fluent-bit.conf"
    owner: "root"
    group: "root"
    mode: 0644

Step3: Start fluent-bit

Here we are going to use the following role “linux_start_fluentbit” to start up the fluent-bit service.

admin@fedser:fluentbit$ cat roles/linux_start_fluentbit/tasks/main.yml 
- name: "ensure fluent-bit service started"
  service:
    name: fluent-bit
    state: started

Step4: Stop fluent-bit

The following role “linux_stop_fluentbit” helps to stop the fluent-bit service.

admin@fedser:fluentbit$ cat roles/linux_stop_fluentbit/tasks/main.yml 
- name: "ensure fluent-bit service stopped"
  service:
    name: fluent-bit
    state: stopped

Step5: Restart fluent-bit

The following role “linux_restart_fluentbit” to restart the fluent-bit service.

admin@fedser:fluentbit$ cat roles/linux_restart_fluentbit/tasks/main.yml 
- name: "ensure fluent-bit service running"
  service:
    name: fluent-bit
    state: restarted

Step6: Fluent Bit Ansible Playbook

Here is the main playbook from which the required roles can be triggered based on the tag information that we pass to the ansible playbook. The instructions to execute are provided in the README.md file below.

admin@fedser:fluentbit$ cat linux_setup_fluentbit.yml 
---
- hosts: "fluent-bit"
  serial: 1
  become: true
  become_user: root
  roles:
  - { role: "linux_ping", tags: "linux_ping" }
  - { role: "linux_install_fluentbit", tags: "linux_install_fluentbit" }
  - { role: "linux_configure_fluentbit", tags: "linux_configure_fluentbit" }
  - { role: "linux_restart_fluentbit", tags: "linux_restart_fluentbit" }
  - { role: "linux_stop_fluentbit", tags: "linux_stop_fluentbit" }
  - { role: "linux_start_fluentbit", tags: "linux_start_fluentbit" }

Step7: README Instructions

admin@fedser:fluentbit$ cat README.md 
# Instructions for execution

ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts --tags "linux_ping" -v
ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts --tags "linux_install_fluentbit" -v 
ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts --tags "linux_configre_fluentbit" -v
ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts --tags "linux_stop_fluentbit" -v
ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts --tags "linux_start_fluentbit" -v
ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts --tags "linux_restart_fluentbit" -v

Step8: Execute Playbook

We can execute specific role by mentioning the “–tags” option as shown in the instructions. But if we want to execute all the roles in a playbook we can run the playbook command without providing the “–tags” option as shown below.

admin@fedser:fluentbit$ ansible-playbook linux_setup_fluentbit.yml -i inventory/hosts
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

PLAY [fluent-bit] ***********************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************
ok: [fedres.stack.com]

TASK [linux_ping : ansible ping pong validation] ****************************************************************************************************************************
ok: [fedres.stack.com]

TASK [linux_install_fluentbit : Add fluentbit repository] *******************************************************************************************************************
ok: [fedres.stack.com]

TASK [linux_install_fluentbit : Install fluent-bit] *************************************************************************************************************************
ok: [fedres.stack.com]

TASK [linux_configure_fluentbit : upload fluent-bit config] *****************************************************************************************************************
ok: [fedres.stack.com]

TASK [linux_restart_fluentbit : ensure fluent-bit service running] **********************************************************************************************************
changed: [fedres.stack.com]

TASK [linux_stop_fluentbit : ensure fluent-bit service stopped] *************************************************************************************************************
changed: [fedres.stack.com]

TASK [linux_start_fluentbit : ensure fluent-bit service started] ************************************************************************************************************
changed: [fedres.stack.com]

PLAY RECAP ******************************************************************************************************************************************************************
fedres.stack.com           : ok=8    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The order of stop/start roles can be changed as per your requirements.

Step9: Validate results

By default fluent-bit is configured to capture the cpu metrics using the cpu input plugin and the statistics are output to console as shown below.

admin@fedres:~$ journalctl -fu fluent-bit
Dec 22 17:37:42 fedres.stack.com fluent-bit[2161]: [0] cpu.local: [1703246861.874238578, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>0.000000, "cpu2.p_user"=>0.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>0.000000, "cpu3.p_user"=>0.000000, "cpu3.p_system"=>0.000000, "cpu4.p_cpu"=>0.000000, "cpu4.p_user"=>0.000000, "cpu4.p_system"=>0.000000, "cpu5.p_cpu"=>0.000000, "cpu5.p_user"=>0.000000, "cpu5.p_system"=>0.000000, "cpu6.p_cpu"=>0.000000, "cpu6.p_user"=>0.000000, "cpu6.p_system"=>0.000000, "cpu7.p_cpu"=>0.000000, "cpu7.p_user"=>0.000000, "cpu7.p_system"=>0.000000}]
Dec 22 17:37:43 fedres.stack.com fluent-bit[2161]: [0] cpu.local: [1703246862.874265104, {"cpu_p"=>0.250000, "user_p"=>0.125000, "system_p"=>0.125000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>0.000000, "cpu2.p_user"=>0.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>0.000000, "cpu3.p_user"=>0.000000, "cpu3.p_system"=>0.000000, "cpu4.p_cpu"=>1.000000, "cpu4.p_user"=>0.000000, "cpu4.p_system"=>1.000000, "cpu5.p_cpu"=>0.000000, "cpu5.p_user"=>0.000000, "cpu5.p_system"=>0.000000, "cpu6.p_cpu"=>0.000000, "cpu6.p_user"=>0.000000, "cpu6.p_system"=>0.000000, "cpu7.p_cpu"=>0.000000, "cpu7.p_user"=>0.000000, "cpu7.p_system"=>0.000000}]
Dec 22 17:37:44 fedres.stack.com fluent-bit[2161]: [0] cpu.local: [1703246863.874168108, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>0.000000, "cpu2.p_user"=>0.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>0.000000, "cpu3.p_user"=>0.000000, "cpu3.p_system"=>0.000000, "cpu4.p_cpu"=>0.000000, "cpu4.p_user"=>0.000000, "cpu4.p_system"=>0.000000, "cpu5.p_cpu"=>0.000000, "cpu5.p_user"=>0.000000, "cpu5.p_system"=>0.000000, "cpu6.p_cpu"=>0.000000, "cpu6.p_user"=>0.000000, "cpu6.p_system"=>0.000000, "cpu7.p_cpu"=>0.000000, "cpu7.p_user"=>0.000000, "cpu7.p_system"=>0.000000}]
Dec 22 17:37:45 fedres.stack.com fluent-bit[2161]: [0] cpu.local: [1703246864.874245701, {"cpu_p"=>0.250000, "user_p"=>0.000000, "system_p"=>0.250000, "cpu0.p_cpu"=>1.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>1.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>0.000000, "cpu2.p_user"=>0.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>0.000000, "cpu3.p_user"=>0.000000, "cpu3.p_system"=>0.000000, "cpu4.p_cpu"=>0.000000, "cpu4.p_user"=>0.000000, "cpu4.p_system"=>0.000000, "cpu5.p_cpu"=>0.000000, "cpu5.p_user"=>0.000000, "cpu5.p_system"=>0.000000, "cpu6.p_cpu"=>0.000000, "cpu6.p_user"=>0.000000, "cpu6.p_system"=>0.000000, "cpu7.p_cpu"=>1.000000, "cpu7.p_user"=>0.000000, "cpu7.p_system"=>1.000000}]

Hope you enjoyed reading this article. Thank you..

How to install and configure Fluent Bit using Ansible