How to trigger Automated GitLab CICD pipeline builds using the Gitlab runner

Test Environment

Fedora 32 installed

What is Gitlab

Gitlab is a single application for the entire DevOps lifecycle. GitLab in its core is a Ruby on Rails project. Gitlab can be installed using different methods. Here we are using an Omnibus GitLab docker image which is a way to package different services and tools required to run GitLab. Omnibus GitLab is a customized fork of the Omnibus project from Chef. It follows a batteries-included style of distribution. GitLab in its core is a Ruby on Rails project. There are two product distributions available through different subscriptions (ie. Gitlab CE and Gitlab EE).

What is Gitlab Runner

Gitlab runner is an application that works with GitLab CI/CD to run jobs in a pipeline. Gitlab runner docker images are wrappers around the standard gitlab-runner command, like if GitLab Runner was installed directly on the host. When we register a runner, we are setting up communication between your GitLab instance and the machine where GitLab Runner is installed. There are different types of executors available from which we can select while we are registerring the runner. An executor determines the environment each job runs in. When you install GitLab Runner in a Docker container and choose the Docker executor to run your jobs, it’s sometimes referred to as a “Docker-in-Docker” configuration.

Procedure

Step1: Instantiate gitlab-runner in a docker container

Here we are going to use the docker-compose file to instantiate gitlab-runner within a docker container. Also we need to make sure that we persist the configuration data so that its not lost with container restart.

[admin@fedser32 gitlab-runner-docker]$ cat docker-compose.yml 
version: "3"
services:
  gitlab-runner:
    image: "gitlab/gitlab-runner:latest"
    volumes:
    - "/apps/gitlab-runner/config:/etc/gitlab-runner"
    - "/var/run/docker.sock:/var/run/docker.sock"

Step2: Start the gitlab-runner service and validate

Lets start the docker-compose.yml file corresponding to the gitlab-runner as shown below.

[admin@fedser32 gitlab-runner-docker]$ docker-compose up -d
Creating network "gitlab-runner-docker_default" with the default driver
Pulling gitlab-runner (gitlab/gitlab-runner:latest)...
latest: Pulling from gitlab/gitlab-runner
...
Digest: sha256:9628f2482d9473b7771f92f8d8bcde15d4bd46b2ce60e8a33812f14ba4066aec
Status: Downloaded newer image for gitlab/gitlab-runner:latest
Creating gitlab-runner-docker_gitlab-runner_1 ... done

Get the gitlab-runner help from within the container.

[admin@fedser32 gitlab-runner-docker]$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED          STATUS          PORTS     NAMES
04e6112bca32   gitlab/gitlab-runner:latest   "/usr/bin/dumb-init …"   16 minutes ago   Up 16 minutes             gitlab-runner-docker_gitlab-runner_1

[admin@fedser32 gitlab-runner-docker]$ docker exec -it gitlab-runner-docker_gitlab-runner_1 gitlab-runner --help
NAME:
   gitlab-runner - a GitLab Runner

USAGE:
   gitlab-runner [global options] command [command options] [arguments...]
...

Step3: Register the runner

Registering a runner means we are binding the runner with one or more gitlab instances. The GitLab Runner Container won’t pick up any jobs until it’s registered.

For registerring a shared runner, we need to first generate the token by logging in as Administrator and navigating to Overview > Runners section in Admin area.

Registration URL for runner – http://192.168.29.117
Registration token – CmziosdFsyeAFL1agPYX

Lets try to register the runner using the below command.

[admin@fedser32 gitlab-latest-docker]$ docker exec -it gitlab-runner-docker_gitlab-runner_1 gitlab-runner register
Runtime platform                                    arch=amd64 os=linux pid=92 revision=58ba2b95 version=14.2.0
Running in system-mode.                            
                                                   
Enter the GitLab instance URL (for example, https://gitlab.com/):
http://192.168.29.117
Enter the registration token:
CmziosdFsyeAFL1agPYX
Enter a description for the runner:
[04e6112bca32]: 
Enter tags for the runner (comma-separated):
Registering runner... succeeded                     runner=CmziosdF
Enter an executor: custom, docker, docker-ssh, virtualbox, parallels, shell, ssh, docker+machine, docker-ssh+machine, kubernetes:
docker
Enter the default Docker image (for example, ruby:2.6):
openjdk
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Once the registration is completed successfully. You should be able to see the registered runner in the runners section as shown below once you have logged in as Administrator into gitlab portal.

Step4: Create a Gitlab Project to host the source code

Here is my sample project created in gitlab which just has a README.md file and also .gitlab-ci.yml file as shown below. THe purpose of this sample project with just a .gitlab-ci.yml file it to just show how to build the project automatically using the gitlab runner when ever there is a commit happens in the repository. The .gitlab-ci.yml file is required in the respository to make the project build automatilly using the in build CICD pipeline of gitlab with a gitlab runner.

[admin@fedser32 external]$ git clone http://192.168.29.117/stackapps/hellogitlab.git
[admin@fedser32 external]$ cd hellogitlab/
[admin@fedser32 hellogitlab]$ ls -ahl
total 16K
drwxrwxr-x.  3 admin admin 4.0K Sep 19 14:16 .
drwxrwxr-x. 10 admin admin 4.0K Sep 17 17:36 ..
drwxrwxr-x.  8 admin admin 4.0K Sep 19 14:16 .git
-rw-rw-r--.  1 admin admin  563 Sep 19 14:16 .gitlab-ci.yml
-rw-rw-r--.  1 admin admin    0 Sep 17 17:37 README.md

[admin@fedser32 hellogitlab]$ cat .gitlab-ci.yml 
build-job:
  stage: build
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"

test-job1:
  stage: test
  script:
    - echo "This job tests something"

test-job2:
  stage: test
  script:
    - echo "This job tests somethingttttt, but takes more time than test-job1."
    - echo "After the echo commands complete, it runs the sleep command for 20 seconds"
    - echo "which simulates a test that runs 20 seconds longer than test-job1"
    - sleep 20

deploy-prod:
  stage: deploy
  script:
    - echo "This job deploys something from the $CI_COMMIT_BRANCH branch."

As soon as you create a project and commit the following .gitlab-ci.yml into the project. The CICD pipeline gets triggered which will build each stage defined in the file as shown in screenshot below. The shared runner which we created and register in Step3 will be used to build the pipeline.

Please note that gitlab comes with standard .gitlab-ci.yml templates from which you can select based on the type of projects that you are trying to build. Here ins the screenshot as shown below showing the list of tempates available to select from.

Hope you enjoyed reading this article. Thank you..