How to generate SOS report for Linux Operating System

Often, we will have a situation where in we want to capture diagnostic information from a linux system which is having issue and provide this information to the product owner for further review and analysis. Here in this article we will see how we can generate SOS report for an ansible managed fedora linux system. We will be preparing an ansible playbook with a role to be executed on the managed node for generating and capturing the diagnostic information in this guide.

Table of Contents

Test Environment

Fedora 36 server

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

Procedure

Step1: Make sure the affected node is managed using ansible

Here i am going to use the following machine ‘fedser36.stack.com’ as an affected node from which we would like to capture the diagnostic information. Make sure this server FQDN or IP address is available in the ansible list of managed hosts inventory file as shown below.

[admin@fedser learnansible]$ grep -i -C 1 "fedser36.stack.com" /etc/ansible/hosts 
[sandbox]
fedser36.stack.com

Here as you can see i have my ‘fedser36.stack.com’ under the ‘sandbox’ group managed by ansible.

Step2: Prepare the ansible playbook with role directory structure

Here is the below directory structure that i have prepared where in under ‘sos_report’ folder we have a playbook named ‘linux_sos_report.yml’ from which we are going to execute the role ‘sos_report’ with the respective tasks underneath. Also there are some variables that we have defined in the vars folder.

[admin@fedser learnansible]$ tree sos_report/
sos_report/
├── linux_sos_report.yml
└── roles
    └── sos_report
        ├── tasks
        │   └── main.yml
        └── vars
            └── main.yml

Step3: Create the ansible playbook definition

So here is my very basic ansible playbook which executes the role ‘sos_report’ on my sandbox group of managed hosts. As sos report generation required escalated privileges we need to run this playbook with sudo privilege user or a root user.

[admin@fedser sos_report]$ pwd
/home/admin/middleware/stack/github_space/learnansible/sos_report

[admin@fedser sos_report]$ ls -ltr
total 8
drwxrwxr-x. 3 admin admin 4096 Jul  8 10:24 roles
-rw-rw-r--. 1 admin admin  141 Jul  8 10:45 linux_sos_report.yml

[admin@fedser sos_report]$ cat linux_sos_report.yml 
---
- name: Linux SOS report generation playbook
  hosts: sandbox
  become: true
  become_user: root
  roles:
    - { role: 'sos_report' }

Step4: Create the ansible role vars main.yml definition file

This file contains our variable and arguments to our ‘sos report’ command. We are defining the report generation folder variable, case variable and the arguments that we want to pass to the ‘sos report’ command.

[admin@fedser vars]$ pwd
/home/admin/middleware/stack/github_space/learnansible/sos_report/roles/sos_report/vars
[admin@fedser vars]$ cat main.yml 
---
report_dir: '/var/tmp/reports'
report_case: 'test'
report_args:
  - --all-logs
  - --batch
  - --quiet
  - --verify

Step5: Create the ansible role tasks main.yml definition file

Here in the roles main.yml definition file we have the following tasks defined. First, we are installing the sos package and then creating a reports directory on managed node to store the generated sos report. Finally we are executing the ‘sos report’ with the arguments that we defined in the vars main.yml file as shown below.

[admin@fedser tasks]$ pwd
/home/admin/middleware/stack/github_space/learnansible/sos_report/roles/sos_report/tasks
[admin@fedser tasks]$ ls -ltr
total 4
-rw-rw-r--. 1 admin admin 73 Jul  8 10:48 main.yml
[admin@fedser tasks]$ cat main.yml 
---
- name: install sos package
  dnf:
    name: sos
    state: present
- name: create reports directory
  file:
    path: "{{ report_dir }}"
    state: directory
    mode: '0755'
- name: generate sos report
  shell: /usr/sbin/sos report
        {% for arg in report_args %}
        {{ arg }}
        {% endfor %} 
        --tmp-dir "{{ report_dir }}" --case-id "{{ report_case }}"

Step6: Execute the ansible playbook

Navigate to the folder where you have your ansible playbook definition available as shown below.

[admin@fedser sos_report]$ pwd
/home/admin/middleware/stack/github_space/learnansible/sos_report
[admin@fedser sos_report]$ ls -ltr
total 8
drwxrwxr-x. 3 admin admin 4096 Jul  8 10:24 roles
-rw-rw-r--. 1 admin admin  141 Jul  8 10:45 linux_sos_report.yml

Now, we can execute the ansible playbook as shown below. We need to pass the sudo privilege user password for the execution to proceed.

[admin@fedser sos_report]$ ansible-playbook linux_sos_report.yml -K
BECOME password: 

PLAY [Linux SOS report generation playbook] **************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************************************
ok: [fedser36.stack.com]

TASK [sos_report : install sos package] ******************************************************************************************************************************************************
changed: [fedser36.stack.com]

TASK [sos_report : create reports directory] *************************************************************************************************************************************************
changed: [fedser36.stack.com]

TASK [sos_report : generate sos report] ******************************************************************************************************************************************************
changed: [fedser36.stack.com]

PLAY RECAP ***********************************************************************************************************************************************************************************
fedser36.stack.com         : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Step7: Validate SOS report generation

Currently i am logged into my managed node and as you can see the sos package is installed and it has generated the compressed format report file in the mentioned folder as per the playbook task execution.

[admin@fedser36 ~]$ rpm -qa | grep sos
sos-4.3-2.fc36.noarch
[admin@fedser36 ~]$ ls -ltr /var/tmp/reports/
total 11860
-rw-------. 1 root root 12137612 Jul  8 12:13 sosreport-fedser36-test-2022-07-08-rydzkzt.tar.xz
-rw-r--r--. 1 root root       65 Jul  8 12:13 sosreport-fedser36-test-2022-07-08-rydzkzt.tar.xz.sha256

This report file can further be copied onto your ansible controller or it can be upload into some repository for later reference as per your requirements.

Hope you enjoyed reading this article. Thank you..

How to generate SOS report for Linux Operating System