How to share Kubernetes cluster agent with multiple GitLab projects

Here in this article we will try to configure a Gitlab CICD agent configuration file such a way that the agent can be shared with other gitlab projects and used to deploy or manage the kubernetes cluster components.

Table of Contents

Test Environment

fedres.stack.com – gitlab v16.9.2-ee
k8master.stack.com – master node – v1.28.5
k8node.stack.com – worker node – v1.28.5

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

Procedure

Step1: Ensure Gitlab agent for Kubernetes cluster installed

As a first step you need to ensure that you have a working Gitlab Server with Gitlab agent for kubernetes cluster installed on your kubernetes cluster.
Follow “How to manage Kubernetes cluster using Gitlab CICD workflow” to complete this setup.

Step2: Create a blank project and clone

Here in this step we are going to create a blank project named “hellojob” which will be used to deploy a job to kubernetes cluster using the CICD workflow.

admin@fedres:~$ git clone git@fedres.stack.com:dev/hellojob.git
admin@fedres:~$ cd hellojob
admin@fedres:hellojob$ git switch –create main

Step3: Update kubernetes_iac project kubernetes agent configuration file

Here in this step we are going to create the following config.yml file and update it such that the “hellojob” project is granted access to this kubernetes cluster agent.

admin@fedres:~/kubernetes_iac$ cat .gitlab/agents/fedreskub/config.yaml 
ci_access:
  projects:
    - id: dev/hellojob

Commit and Push the changes to the “kubernetes_iac” repository.

admin@fedres:~/kubernetes_iac$ git add .gitlab/agents/fedreskub/config.yaml
admin@fedres:~/kubernetes_iac$ git commit -m "update gitlab cicd workflow with hellojob access"
admin@fedres:~/kubernetes_iac$ git push -u origin main

Step4: Update hellojob project with CICD workflow and k8’s job definition

Let us first create a kubernetes job definition file as shown below.

admin@fedres:hellojob$ cat job.yml 
apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  template:
    spec:
      containers:
      - name: pi
        image: perl:5.34.0
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  backoffLimit: 4

Now, let us create the .gitlab-ci.yml which will be used to connect to the shared kubernetes cluster and deploy the kubernetes job.yml file. Here is the .gitlab-ci.yml file.

admin@fedres:hellojob$ cat .gitlab-ci.yml 
stages:
  - deploy

deploy:
  stage: deploy
  script:
    - kubectl config set clusters.gitlab.certificate-authority-data LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQ3RENDQXRTZ0F3SUJBZ0lVWWZOR1IyS3p5OUtvcjNxOEkva1pBeDhZRVlzd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1p6RUxNQWtHQTFVRUJoTUNTVTR4Q3pBSkJnTlZCQWdNQWsxSU1ROHdEUVlEVlFRSERBWk5kVzFpWVdreApEakFNQmdOVkJBb01CWE4wWVdOck1ROHdEUVlEVlFRTERBWmtaWFp2Y0hNeEdUQVhCZ05WQkFNTUVHWmxaSEpsCmN5NXpkR0ZqYXk1amIyMHdIaGNOTWpRd09UTXdNVEEwTXpNNVdoY05Nall3T1RNd01UQTBNek01V2pCbk1Rc3cKQ1FZRFZRUUdFd0pKVGpFTE1Ba0dBMVVFQ0F3Q1RVZ3hEekFOQmdOVkJBY01CazExYldKaGFURU9NQXdHQTFVRQpDZ3dGYzNSaFkyc3hEekFOQmdOVkJBc01CbVJsZG05d2N6RVpNQmNHQTFVRUF3d1FabVZrY21WekxuTjBZV05yCkxtTnZiVENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFNb1EzQklBOFY4MjNiRGMKMFkyRmJad1FCOTNycXRtZU5iWDRKZ0t6dmhVK280NlRxRm9FZGRQY2g1WmdiSmNrSGZMZEZJSm1hQjBFLzFKaQpPS0pQMjdZd1RLdHpZRTBKK2dFVmoxaFVFYUdFbGhvY0p5SHYrK2drcEd1R0RORUc1Ykc3b0s1emcrbjJyVjVuCmtKUXBGZU5KYkU0ZkJVWHAvZm9Zd2FVSGJwUC9ESWNSMzZKN08vU040djNZaXRTeWRRcHdTS09kekJmNU1yelYKNXFWT2hRa1pXT3hvZEdQeUtoSm5lYXZaSGVZa3RCWndlTXNLcnFZNUVTVG9rSUVvSHI4dDVaRFp5eHVCZlhSZApaSkFHNDhGWVFneVRKYjBDd0pHK3luVm5QUFRhYkFXZXRMVHVlRzFTMFlsMGp2SkpxbzdnSVJVNlpsNGlrckZZClNyckpOcU1DQXdFQUFhT0JqekNCakRBTEJnTlZIUThFQkFNQ0JMQXdIUVlEVlIwbEJCWXdGQVlJS3dZQkJRVUgKQXdFR0NDc0dBUVVGQndNQ01EOEdBMVVkRVFRNE1EYUNFR1psWkhKbGN5NXpkR0ZqYXk1amIyMkNEM0JoWjJWegpMbk4wWVdOckxtTnZiWUlSS2k1d1lXZGxjeTV6ZEdGamF5NWpiMjB3SFFZRFZSME9CQllFRlBtMHh6c3lTOXJuCktTMEEyeXE0UHFBUk9SaEZNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUFYZ0tJbWNqcGFNUTlBdm0veHZpejQKNDVFZGtJdy9nODBYVUlyQkpDNWdRbE1lRUFoK2MzUTduaVJldXNKMlhwd0doT0VyZmp5R3NiK3dWQkh4bmMvRgo2VXNFL3lUMDY3NlF0S0g5eWhSTjA2di9HZ2tEenZNbVBvUGlHOFIwVUx5QjFPVWpYS0tsNjBNaTZUQy9FM1IyClo4SndpRzd1VG5yNGh2WVV4aVFoVE5MdkJYdUdoM1NrUjNoVmVZWktkZTlBTzlRRmorcERBTTNuZEZqMUZNY1MKQzF2ekhzdFVNT2hxSHNCZXFMbkI3NHp5NmV6V3hGWEZLMkdDcTlXRm9RSG90cDJxQ3pCeEpiU3psdXN1bm5MSgpZaGdVTGtKS1R6Y1UzVlh3SExrRldaNFJZZ3Vmc3ozRGc1ZGFwUmpoTzhuSXVJb29CNWdnamFtWEVNRlBqNFh6Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    - kubectl config view
    - kubectl config get-contexts
    - kubectl config use-context dev/kubernetes_iac:fedreskub
    - kubectl apply -f job.yml
    - kubectl get jobs
    #- kubectl --insecure-skip-tls-verify get pods

Commit and Push the changes to the “hellojob” repository.

admin@fedres:hellojob$ git add .gitlab-ci.yml job.yml 
admin@fedres:hellojob$ git commit -m "update cicd workflow and job definition"
admin@fedres:hellojob$ git push -u origin main

Step5: Trigger and validate CICD pipeline

Now, its time to validate the CICD pipeline. You can trigger the pipeline execution for “hellojob” manually if auto cicd workflow trigger is not enabled in your environment.

You can also validate that the “fedreskub” kubernetes agent is now shared with the “hellojob” project as shown below.

Step6: Validate Kubernetes cluster

Let us check the jobs list on the kubernetes cluster to check our job.

admin@k8master:~$ kubectl get jobs
NAME   COMPLETIONS   DURATION   AGE
pi     1/1           8s         3m3s

Hope you enjoyed reading this article. Thank you..

How to share Kubernetes cluster agent with multiple GitLab projects