Linux container system management and monitoring using Podman

Here in this article we will see how we can manage and monitor the overall system using Podman. There might be a need for System Administrators to know about detailed information about the system where the containers are running.

Table of Contents

Test Environment

Fedora 32
Podman

Here are some of the information that may be relevant for System Adminstrators.

Gather overall information about the host machine running Linux containers
Real time Statistics for all the running Linux containers
Disk utilization by Images and Containers
Runtime events that were triggered on the Host machine managing the Linux containers

Lets see how we can capture all these information with Podman and day to day management and monitoring of the System hosting Linux containers.

If you are interested in watching video. Here is the YouTube on the same step by step procedure outlined below.

Procedure

Step1: Gather overall information about the host machine running Linux containers

The following command will provide important information about the host machine where Podman is installed. It containes information related to Memory and Swap utilization. Also it provide information about how many images are available on the host machine and number of containers available in different states. There is also information regarding the registries that are currently available from where Podman try to search, pull and push the images. It also contains information related to configuration files like storage.conf which provide information related to the directories where volumes, containers and images are stored.

[root@fed32 ~]# podman info

Step2: Real time Statistics for all the running Linux containers

The following command give real time information related to CPU, Memory and Network IO utilization and number of PIDs that are currently running within a container. This information will be very useful like top command information to identify which container process is taking up more resources and take corrective actions by scaling them.

[root@fed32 ~]# podman stats -a
ID            NAME         CPU %   MEM USAGE / LIMIT  MEM %   NET IO             BLOCK IO           PIDS
49c9dae3a751  sweet_dirac  0.12%   30.38MB / 2.047GB  1.48%   28.11kB / 12.14kB  40.83MB / 2.703MB  278
a74a040b886f  busyman2     --      753.7kB / 2.047GB  0.04%   1.258kB / 10.08kB  1.753MB / 0B       1

Step3: Disk Ulilization by Images and Containers

The following command will provide overall space utilized by Images, Containers and Volumes that might have been created. It also contains information related to containers and their respective volumes that are mapped.

[root@fed32 ~]# podman system df -v

Images space usage:

REPOSITORY                         TAG     IMAGE ID      CREATED  SIZE     SHARED SIZE  UNIQUE SIZE  CONTAINERS
registry.fedoraproject.org/fedora  latest  e6b57c0d9f53  2 weeks  208MB    208MB        0B           0
                               d8168a416638  3 days   457.4MB  457.4MB      0B           0
                               e1d959ef7379  3 days   457.4MB  457.4MB      0B           0
                               8fb1a4f15a0c  3 days   457.4MB  457.4MB      0B           0
localhost/hello                    1.0     b6611cf6b4a9  3 days   457.4MB  457.4MB      136B         1
docker.io/library/busybox          latest  f0b02e9d092d  12 days  1.455MB  0B           1.455MB      1

Containers space usage:

CONTAINER ID  IMAGE         COMMAND                       LOCAL VOLUMES  SIZE     CREATED   STATUS   NAMES
49c9dae3a751  b6611cf6b4a9  /usr/sbin/httpd -DFOREGROUND  0              7.073kB  3 days    running  sweet_dirac
a74a040b886f  f0b02e9d092d  sh                            1              92B      32 hours  running  busyman2

Local Volumes space usage:

VOLUME NAME  LINKS   SIZE
stackvol     0       14B
static       1       15B

If there are any uused containers or images are available all these can be captured and pruned to free up space using the below command.

[root@fed32 ~]# podman system prune

WARNING! This will remove:
        - all stopped containers
        - all stopped pods
        - all dangling images
        - all build cache
Are you sure you want to continue? [y/N] y
Deleted Pods
Deleted Containers
Deleted Images

Step4: Runtime events that were triggered on the Host machine managing the Linux containers

This is a very important feature provided by Podman utility which provides information regarding all the events that have been triggered in Linux container management.

[root@fed32 ~]# podman events --filter event=stop --since 48h
2020-10-25 14:17:09.914179712 +0530 IST container stop f5b0c55bad1b9f268d28312a44d898c7b084f4521db73597562e01372438f522 (image=docker.io/library/busybox:latest, name=busyman)
2020-10-25 14:22:42.078107145 +0530 IST container stop f5b0c55bad1b9f268d28312a44d898c7b084f4521db73597562e01372438f522 (image=docker.io/library/busybox:latest, name=busyman)
2020-10-26 05:15:54.104053927 +0530 IST container stop f5b0c55bad1b9f268d28312a44d898c7b084f4521db73597562e01372438f522 (image=docker.io/library/busybox:latest, 

[root@fed32 ~]# podman events --filter event=start --since 48h
2020-10-25 14:16:19.081802639 +0530 IST container start f5b0c55bad1b9f268d28312a44d898c7b084f4521db73597562e01372438f522 (image=docker.io/library/busybox:latest, name=busyman)
2020-10-25 14:19:28.439180422 +0530 IST container start 41f9bcede1a4513990f54d2899f37248d16778e2daba12a02a8f99dc4e7fbdfe (image=docker.io/library/busybox:latest, name=busyman2)
2020-10-25 14:20:47.870894016 +0530 IST container start 340f48ee36c591ca4d53ef14810d268318026798c568f574686fbf482272a19e (image=docker.io/library/busybox:latest, name=busyman2)

[root@fed32 ~]# podman events --filter event=create --since 48h
2020-10-25 14:18:55.511011917 +0530 IST container create 9ddb4e9c870f4cdb318ae1d1c21dca4a532f34a4441eacc297f3d7e3b24f86a1 (image=docker.io/library/busybox:latest, name=busyman2)
2020-10-25 14:19:28.052875886 +0530 IST container create 41f9bcede1a4513990f54d2899f37248d16778e2daba12a02a8f99dc4e7fbdfe (image=docker.io/library/busybox:latest, name=busyman2)
2020-10-25 14:20:47.583994933 +0530 IST container create 340f48ee36c591ca4d53ef14810d268318026798c568f574686fbf482272a19e (image=docker.io/library/busybox:latest, name=busyman2)

[root@fed32 ~]# podman events --filter event=remove --since 48h
2020-10-25 14:19:20.657922937 +0530 IST container remove 9ddb4e9c870f4cdb318ae1d1c21dca4a532f34a4441eacc297f3d7e3b24f86a1 (image=docker.io/library/busybox:latest, name=busyman2)
2020-10-25 14:20:30.728764719 +0530 IST container remove 41f9bcede1a4513990f54d2899f37248d16778e2daba12a02a8f99dc4e7fbdfe (image=docker.io/library/busybox:latest, name=busyman2)
2020-10-25 14:21:22.958907788 +0530 IST container remove 340f48ee36c591ca4d53ef14810d268318026798c568f574686fbf482272a19e (image=docker.io/library/busybox:latest, name=busyman2)

If you want to take a look at the events that were trigger since a particular time we can just pass the –since option by ignoring the –filter as shown below.

[root@fed32 ~]# podman events --since 48h
2020-10-26 23:06:25.780984285 +0530 IST system refresh
2020-10-26 23:06:57.233401534 +0530 IST container init ad2a74bd296c4cdc1752025305bb8933758811570e654057c3069f7114d3d683 (image=docker.io/library/busybox:latest, name=busyman3)
2020-10-26 23:06:57.335876286 +0530 IST container start ad2a74bd296c4cdc1752025305bb8933758811570e654057c3069f7114d3d683 (image=docker.io/library/busybox:latest, name=busyman3)
2020-10-26 23:07:13.683360632 +0530 IST container exec ad2a74bd296c4cdc1752025305bb8933758811570e654057c3069f7114d3d683 (image=docker.io/library/busybox:latest, name=busyman3)
2020-10-26 23:19:58.23759952 +0530 IST container init 49c9dae3a7517dfe8d71104d138dec1ce8cf18116d33deca24e735ba50880396 (image=localhost/hello:1.0, name=sweet_dirac)
2020-10-26 23:19:58.436329641 +0530 IST container start 49c9dae3a7517dfe8d71104d138dec1ce8cf18116d33deca24e735ba50880396 (image=localhost/hello:1.0, name=sweet_dirac)
2020-10-27 10:02:29.050814961 +0530 IST system refresh

Hope you enjoyed reading this article. Thank you..

Linux container system management and monitoring using Podman