How to manage and troubleshoot SELinux issues

How to manage and troubleshoot SELinux issues

fedora_selinux_management

Here in this article we will try to see how SELinux security when enabled in Linux will provide additional layer of security and also try to troubleshoot some of the issues that we might encounter if SELinux is enabled.

Test Environment

Fedora 39 server

What is DAC

In Linux everything is referred to as file. We have different types of files in linux. They are regular files, directory files, special files, block or character files, socket files, named pipe files etc.

Discretionary Access Control also known as DAC is a security model that most of the operation systems implement. In this model users control the permissions of files that they own. Sensitive information could be leaked if the file owners do not implement adequate restrictions through permissions. It does not implement any centralised security policy administered by a security administrator which can be imposed on the system. It does not provide any protection against malicious software program if they are compromised.

What is MAC

MAC security enables a system to adequately defent itself and offers critical support for application security by protecting againts tampering with and bypassing of secured applications. It also provides a strong isolations of application and permits safe execution of untrustworthy applications. MAC access control decisions are based on security relevant information that is attached files and process in the form of labels.

What is SELinux

Security Enhanced Linux also known as SELinux is additional layer of security implementing the Mandatory Access Control (MAC) model. It referes to files such as directories and devices as objects and processes such as user initiated commands or application as subjects. Mandatory Access Control enforces an administratively set security policy over all processes and files in the system, basing access control decisions on labels containing a variety of security relevant information.

SELinux policy rules which implement the MAC model are not used if DAC rules deny access first.

SELinux Implementation in Fedora

Targeted policy is the default SELinux policy used in Fedora. In this implementation processes are targeted and run in either confined domain or unconfined domain. Almost every service that listens on a network runs in a confined domain in Fedora. Processes that required privilege access for execution run in confined domain. With SELinux policy configuration, an attacker’s access to resources and the possible damage they can do is limited. SELinux is based on the least level of access required for a service to run.

SELinux booleans that allow parts of SELinux policy to be changed at runtime, without any knowledge of SELinux policy writing. Changes to the service to run on non default port number and access a non default directory to store files for services required policy configuration to be updated using tools such as semanage.

Type Enforcement is the main permission control used in SELinux targeted policy. All files and processes are labeled with a type: types define a domain for processes and a type for files.

Now let’s try to understand more about SELinux using two Fedora Server 39 system. One of them is SELinux enabled and other is SELinux disabled and see the behaviour of processes interacting with files and other processes.

SELinux Disabled System

Ensure SELinux Disabled

SELinux “sestatus” utility provides us with the status of the SELinux if its enabled or disabled in the linux operating system as shown below.

$ /usr/sbin/sestatus
SELinux status:                 disabled

Install and Start Nginx

The default configuration of nginx listens on port 80 which is a privileged port. For any service to be started on privilege port we need to start up the service with any user who has root privileges.

$ sudo dnf install nginx
$ sudo systemctl start nginx.service 
$ ps -efZ | grep nginx
kernel                          root        2078       1  0 03:35 ?        00:00:00 nginx: master process /usr/sbin/nginx
kernel                          nginx       2079    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2080    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2081    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2083    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2084    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2085    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2086    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          nginx       2087    2078  0 03:35 ?        00:00:00 nginx: worker process
kernel                          admin       2309    1067  0 04:20 pts/0    00:00:00 grep --color=auto nginx

Let’s also stop the firewall service to just see the effect of SELinux security.

$ sudo systemctl stop firewalld.service 

Update default port and restart service

Now’s let’s update our service to start on a non default port “2222” by updating the nginx.conf as shown below.

$ grep listen /etc/nginx/nginx.conf
        listen       2222;
        listen       [::]:2222;

Let’s restart the service now and see if there are any issues starting up the service.

$ sudo systemctl restart nginx.service 

With SELinux disabled, there is no restriction on port on which a service is supposed to be listening. As long as the user initiating the service and having access to the configuration file they are able to update the configuration as per their need and restart the service.

Update default docroot and restart service

Create a new doc root for holding the html content for nginx service.

# mkdir -p /opt/nginx_data
# echo "Testing nginx instance" >> /opt/nginx_data/fedunr.html
# chown -R admin:admin /opt/nginx_data/
# chmod -R 005 /opt/nginx_data/

Update nginx.conf to point to the new doc root as shown below.

$ grep "root" /etc/nginx/nginx.conf
        root         /opt/nginx_data;

$ sudo systemctl restart nginx.service 

Nginx servie restarts without any issues and you shold be able to access the html content as shown below.

Irrespective the ownership of the doc root. The nginx service is able to access the html content from a non default doc root if the required permissions are statisified to acces the files. The access control purely works on the level of permissions that are granted to those files and procesess for that any user, user group or others.

SELinux Enabled System

Ensure SELinux Enabled

SELinux “sestatus” utility provides us with the status of the SELinux if its enabled or disabled in the linux operating system as shown below.

$ /usr/sbin/sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33

Install and Start Nginx

The default configuration of nginx listens on port 80 which is a privileged port. For any service to be started on privilege port we need to start up the service with any user who has root privileges.

$ sudo dnf install nginx
$ sudo systemctl start nginx.service 
$ ps -efZ | grep nginx
system_u:system_r:httpd_t:s0    root        1298       1  0 20:19 ?        00:00:00 nginx: master process /usr/sbin/nginx
system_u:system_r:httpd_t:s0    nginx       1299    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1300    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1301    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1302    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1303    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1304    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1305    1298  0 20:19 ?        00:00:00 nginx: worker process
system_u:system_r:httpd_t:s0    nginx       1306    1298  0 20:19 ?        00:00:00 nginx: worker process

Let’s also stop the firewall service to just see the effect of SELinux security.

$ sudo systemctl stop firewalld.service 

Update default port and restart service

Now’s let’s update our service to start on a non default port “2222” by updating the nginx.conf as shown below.

Also restart the service and see if there are any issues starting up the service.

$ grep 2222 /etc/nginx/nginx.conf
        listen       2222;
        listen       [::]:2222;

$ sudo systemctl restart nginx.service 
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.

Now let’s have a look athe logs of nginx service using journalctl as shown below.

$ journalctl -fxeu nginx
Jan 22 20:23:58 fedres.stack.com nginx[1408]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Jan 22 20:23:58 fedres.stack.com nginx[1408]: nginx: [emerg] bind() to 0.0.0.0:2222 failed (13: Permission denied)
Jan 22 20:23:58 fedres.stack.com nginx[1408]: nginx: configuration file /etc/nginx/nginx.conf test failed
Jan 22 20:23:58 fedres.stack.com systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Jan 22 20:23:58 fedres.stack.com systemd[1]: nginx.service: Failed with result 'exit-code'.

This permission denied error is basically due to selinux policy constraints that are imposed on the confined process using targeted policy. To further troubleshoot on this error we need to look at the audit logs as shown below.

# tail -f /var/log/audit/audit.log | grep nginx
type=AVC msg=audit(1705935470.588:247): avc:  denied  { name_bind } for  pid=1521 comm="nginx" src=2222 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket permissive=0

Analyze selinux issue

Ensure the policycoreutils-python-utils and setroubleshoot-server packages are installed on your system.

$ sealert -l "*"

SELinux is preventing nginx from name_bind access on the tcp_socket port 2222.

*****  Plugin bind_ports (92.2 confidence) suggests   ************************

If you want to allow nginx to bind to network port 2222
Then you need to modify the port type.
Do
# semanage port -a -t PORT_TYPE -p tcp 2222
    where PORT_TYPE is one of the following: http_cache_port_t, http_port_t, jboss_management_port_t, jboss_messaging_port_t, ntop_port_t, puppet_port_t.

*****  Plugin catchall_boolean (7.83 confidence) suggests   ******************

If you want to allow nis to enabled
Then you must tell SELinux about this by enabling the 'nis_enabled' boolean.

Do
setsebool -P nis_enabled 1

*****  Plugin catchall (1.41 confidence) suggests   **************************

If you believe that nginx should be allowed name_bind access on the port 2222 tcp_socket by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'nginx' --raw | audit2allow -M my-nginx
# semodule -X 300 -i my-nginx.pp


Additional Information:
Source Context                system_u:system_r:httpd_t:s0
Target Context                system_u:object_r:unreserved_port_t:s0
Target Objects                port 2222 [ tcp_socket ]
Source                        nginx
Source Path                   nginx
Port                          2222
Host                          fedres.stack.com
Source RPM Packages           
Target RPM Packages           
SELinux Policy RPM            selinux-policy-targeted-39.3-1.fc39.noarch
Local Policy RPM              selinux-policy-targeted-39.3-1.fc39.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     fedres.stack.com
Platform                      Linux fedres.stack.com 6.6.8-200.fc39.x86_64 #1
                              SMP PREEMPT_DYNAMIC Thu Dec 21 04:01:49 UTC 2023
                              x86_64
Alert Count                   2
First Seen                    2024-01-22 20:23:58 IST
Last Seen                     2024-01-22 20:27:50 IST
Local ID                      5f40212e-a76a-477c-8710-15c935ee8183

Raw Audit Messages
type=AVC msg=audit(1705935470.588:247): avc:  denied  { name_bind } for  pid=1521 comm="nginx" src=2222 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket permissive=0


Hash: nginx,httpd_t,unreserved_port_t,tcp_socket,name_bind

By default selinux policy allow http service ie. nginx to bind on below ports only.

$ sudo semanage port --list | grep http_port_t
http_port_t                    tcp      80, 81, 443, 488, 8008, 8009, 8443, 9000

We need to modify the port type as shown below to allow http to bind on port 2222.

$ sudo semanage port -a -t http_port_t -p tcp 2222
$ sudo semanage port --list | grep http_port_t
http_port_t                    tcp      2222, 80, 81, 443, 488, 8008, 8009, 8443, 9000
$ sudo systemctl restart nginx.service 
URL - http://fedres.stack.com:2222/

Update default docroot and restart service

Create a new doc root for holding the html content for nginx service.

# mkdir -p /opt/nginx_data
# echo "Testing nginx instance" >> /opt/nginx_data/fedunr.html
# chown -R admin:admin /opt/nginx_data/
# chmod -R 005 /opt/nginx_data/

Update nginx.conf to point to the new doc root as shown below.

$ grep "root" /etc/nginx/nginx.conf
        root         /opt/nginx_data;
$ sudo systemctl restart nginx.service 
# ls -ltrZ /opt/nginx_data/fedunr.html
-------r-x. 1 admin admin unconfined_u:object_r:usr_t:s0 23 Jan 22 20:45 /opt/nginx_data/fedunr.html

Nginx service restarts without any issues and you should be able to access the html content as the file is labelled with “usr_t”.

As the root user, run the following command to change the type to a type used by Samba.

# chcon -t samba_share_t /opt/nginx_data/fedunr.html
# chcon -t samba_share_t /opt/nginx_data/fedunr.html
# ls -ltrZ /opt/nginx_data/fedunr.html
-------r-x. 1 admin admin unconfined_u:object_r:samba_share_t:s0 23 Jan 22 20:45 /opt/nginx_data/fedunr.html

Now let’s try to access a file with the following context.

# tail -f /var/log/audit/audit.log 
type=AVC msg=audit(1705937058.112:380): avc:  denied  { read } for  pid=1826 comm="nginx" name="fedunr.html" dev="dm-0" ino=8391516 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:samba_share_t:s0 tclass=file permissive=0

Analyze selinux issue

# sealert -l "*"

SELinux is preventing nginx from read access on the file fedunr.html.

*****  Plugin public_content (89.3 confidence) suggests   ********************

If you want to treat fedunr.html as public content
Then you need to change the label on fedunr.html to public_content_t or public_content_rw_t.
Do
# semanage fcontext -a -t public_content_t 'fedunr.html'
# restorecon -v 'fedunr.html'

*****  Plugin catchall (11.6 confidence) suggests   **************************

If you believe that nginx should be allowed read access on the fedunr.html file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'nginx' --raw | audit2allow -M my-nginx
# semodule -X 300 -i my-nginx.pp


Additional Information:
Source Context                system_u:system_r:httpd_t:s0
Target Context                unconfined_u:object_r:samba_share_t:s0
Target Objects                fedunr.html [ file ]
Source                        nginx
Source Path                   nginx
Port                          <Unknown>
Host                          fedres.stack.com
Source RPM Packages           
Target RPM Packages           
SELinux Policy RPM            selinux-policy-targeted-39.3-1.fc39.noarch
Local Policy RPM              selinux-policy-targeted-39.3-1.fc39.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     fedres.stack.com
Platform                      Linux fedres.stack.com 6.6.8-200.fc39.x86_64 #1
                              SMP PREEMPT_DYNAMIC Thu Dec 21 04:01:49 UTC 2023
                              x86_64
Alert Count                   2
First Seen                    2024-01-22 20:53:33 IST
Last Seen                     2024-01-22 20:54:18 IST
Local ID                      fff37532-e3fd-4e3a-a350-e3ae98289bf2

Raw Audit Messages
type=AVC msg=audit(1705937058.112:380): avc:  denied  { read } for  pid=1826 comm="nginx" name="fedunr.html" dev="dm-0" ino=8391516 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:samba_share_t:s0 tclass=file permissive=0


Hash: nginx,httpd_t,samba_share_t,file,read

Relabel the content

# semanage fcontext -a -t public_content_t '/opt/nginx_data/fedunr.html'
# restorecon -v '/opt/nginx_data/fedunr.html'
Relabeled /opt/nginx_data/fedunr.html from unconfined_u:object_r:usr_t:s0 to unconfined_u:object_r:public_content_t:s0

Now if we try to access the html context it should be allowed.

$ curl http://fedres.stack.com:2222/fedunr.html
Testing nginx instance

Hope you enjoyed reading this article. Thank you..