GitLab CI/CD Pipelines for Home Lab

Posted by April 8, 2026 in GIT

The shift toward automation is transforming how system and network engineers manage infrastructure. Recently, I’ve been using GitLab as an accessible, intuitive platform to help engineers demystify CI/CD concepts and build real confidence in adopting GitOps workflows. To help you get hands-on experience, I’ve put together a step-by-step guide to setting up your own GitOps home lab. In this post, we will walk through the entire setup process. Then, to ensure you’ve mastered the concepts, we’ll tackle a practical troubleshooting scenario and finish by deploying a small website. Let’s dive in!


This document outlines the process of implementing GitLab CI/CD pipelines within a lab environment. Leveraging GitLab’s robust CI/CD capabilities allows for automation of various tasks, from building and testing applications to deploying them to local servers or virtual machines.

What is GitLab?

It is a solution that I have been using in the lab for quite some time now. It is, in my opinion, one of the best self-hosted code repositories and makes setting up and learning CI/CD very intuitive and easy to learn. There are many other great solutions you can use as well, such as Jenkins, etc. So the concepts we are learning in this lab will apply across the board. Each solution has a slightly different way of implementing the concepts and terminology that go with a specific solution. So, keep that in mind.

Understanding CI/CD

Continuous Integration (CI) is a software development practice where developers regularly merge their code changes into a central repository. Automated builds and tests are then run. Continuous Delivery (CD) is the logical extension of CI, where code changes are automatically prepared for a release to production.

Automatically test, build and deploy to the end environment (For End Users)

A developer commits new code to the GitLab repository. The code will be released and deployed to the production environment.

Gitlab make CI/CD easy by providing a feature-packed product with the tools you need for hosting repos, deploying and monitoring. 

Prerequisites

Before setting up GitLab CI/CD in your lab, ensure you have the following:

  • GitLab Instance: A self-hosted GitLab instance (Community Edition or Enterprise Edition) running in your lab. (With Public DNS/IP)
  • GitLab Runner: A GitLab Runner installed on a machine within your lab that can execute jobs. This can be the same machine as your GitLab instance or a separate one.
  • Version Control: Your projects are hosted in GitLab repositories.
  • Docker (Recommended): For containerised builds and deployments, Docker is highly recommended.

Setting Up Your GitLab Lab

The GitLab app needs to be installed on the lab server you have deployed. Note that after installing a radon password will be generated and stored in /etc/gitlab/initial_root_password for 24 hours

  1. Install GitLab: Follow the official GitLab documentation.

Example registration command:

sudo systemctl enable --now ssh

sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 5000/tcp
sudo ufw enable

sudo apt install -y curl

curl "https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh" | sudo bash

# Change the example AWS public address to the one for your deployment

sudo EXTERNAL_URL="http://ec2-3-79-45-225.eu-central-1.compute.amazonaws.com" apt install gitlab-ce

sudo cat /etc/gitlab/initial_root_password

# Change the example AWS public address to the one for your deployment

sudo EXTERNAL_URL="http://ec2-3-79-46-215.eu-central-1.compute.amazonaws.com" apt install gitlab-ce

sudo cat /etc/gitlab/initial_root_password

If GitLab can’t detect a valid hostname during installation, reconfigure won’t run automatically. In this case, pass any needed environment variables to your first gitlab-ctl reconfigure command.

  1. Sign in via http://EXTERNAL_URL and use the root password generated during the install.

    Username: root
    Password: See /etc/gitlab/initial_root_password
  2. Activate sign up restictions and save changes



    (Optional) Create user account for yourself via the Admin dashboard > New User

If you choose not to do this, you can continue with the root account in this lab environment.  

GitLab Architecture

GitLab will host your applications and pipelines, which explain what needs to be done for deployment.

The Runners are separate machines that are there to execute the defined pipelines. 

Setting Up Your GitLab Runner

When you install Gitlab, you have to provision what Gitlab calls a GitLab Runner. 

GitLab runners are core components that execute all operations defined in the .gitlab-ci.yml file (the special-purpose file that contains the Gitlab pipeline code).

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

sudo apt install gitlab-runner

apt-cache madison gitlab-runner
sudo apt install gitlab-runner=18.2.1-1 gitlab-runner-helper-images=18.2.1-1

Register a runner

  1. Browse the URL of your gitlab dashboard and click on the Runners button under Admin > CI/CD (http://EXTERNAL_URL/admin/runners)
  2. Under the 3 dots on the right of this page you will find the registration token
  3. Use the token and your URL in the code below to register a runner. The runner will appear on the page when it is successfully registered. 
sudo gitlab-runner register \
  --url http://EXTERNAL_URL/ \
  --registration-token YOUR_REGISTRATION_TOKEN \
  --executor "docker" \
  --description "Lab Docker Runner" \
  --docker-image "alpine:latest" \
  --tag-list "lab,docker"

Install Docker and Docker Compose

At this point, Docker is not installed on your server. It will be needed for the deployment of jobs when you start to develop projects.

  1. Run the following commands on the server terminal
sudo apt update
sudo apt install ca-certificates curl gnupg lsb-release
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

#Verify Docker installation by running the hello-world image
sudo docker run hello-world
sudo docker ps -a

#Manage Docker as a non-root user
sudo usermod -aG docker ubuntu

# Download the latest stable release of Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s | tr '[:upper:]' '[:lower:]')-$(uname -m)" -o /usr/local/bin/docker-compose

# Apply executable permissions to the binary
sudo chmod +x /usr/local/bin/docker-compose

# Verify the installation
docker-compose --version

How to Fix Docker-in-Docker (dind) Permission Errors

When using a GitLab Runner with the Docker executor to build images, you might encounter an error where the docker:dind service fails to start, often with a permission denied message in the job log.

This happens because the dind service (the Docker engine) requires elevated permissions to function, but the runner’s job container is unprivileged by default. The solution is to enable privileged mode for your GitLab Runner.

Step-by-Step Instructions

  1. Connect to Your Runner Server First, SSH into the server that is hosting your GitLab Runner.
  2. Edit the Runner Configuration File Open the config.toml file with a text editor using sudo. This file is almost always located at /etc/gitlab-runner/config.toml.
    Bash
sudo nano /etc/gitlab-runner/config.toml
  1. Enable Privileged Mode Inside the file, find the section for your Docker runner, which starts with [[runners]]. Within that section, under [runners.docker], add the line privileged = true.
    Your configuration should look like this:
    Ini, TOML
[[runners]]
  # ... other settings like name, url, token
  executor = "docker"
  [runners.docker]
    # ... other docker settings
    image = "alpine:latest"
    privileged = true      # <-- Add or change this line to true
  1. Save and Restart the Runner Save the file (in nano, press Ctrl+X, then Y, then Enter) and restart the GitLab Runner service to apply the changes.
    Bash
sudo systemctl restart gitlab-runner

After restarting, the runner will now launch job containers with the necessary privileges, allowing the docker:dind service to start correctly and your Docker build jobs to run without permission errors. ✅

Creating Your First CI/CD Pipeline

CI/CD pipelines are defined in a .gitlab-ci.yml file placed at the root of your project repository.

Basic Build and Test Pipeline

This example demonstrates a simple pipeline that builds a Go application and runs tests.
Check if the Docker builds exist on your server by using the command “sudo docker ps -a” while the build and test stages are in progress.


Add these file to your Git repository

Replace gitlab.com/your-username/your-project-name with your actual GitLab project path.

Create a file named .gitlab-ci.yml in your project root with the following content:

stages:
  - build
  - test

build_job:
  stage: build
  script:
    - apk update
    - apk add go --no-interactive
    - echo "Initializing Go module..."
    - go mod init gitlab.com/your-username/your-project-name
    - echo "Building application..."
    - go build -o myapp .
  tags:
    - lab
    - docker

test_job:
  stage: test
  script:
    - apk update
    - apk add go --no-interactive
    - echo "Initializing Go module..."
    - go mod init gitlab.com/your-username/your-project-name
    - echo "Running tests..."
    - go test ./...
  tags:
    - lab
    - docker

Create a file named main.go in your project root with the following content:

package main

import "fmt"

func main() {
    fmt.Println("Hello, World!")
}

Create a file named main_test.go with the following content:

package main

package main

import "testing"

func TestHelloWorld(t *testing.T) {
    // This is just a placeholder test
    t.Log("Hello, World test executed")
}

Explanation:

  • stages: Defines the sequential stages of your pipeline.
  • build_job / test_job: These are individual jobs within a stage.
  • script: Contains the commands executed by the job.
  • tags: Ensures that only runners with the specified tags pick up these jobs. This is crucial for directing jobs to your home lab runner.

How to Set Up Docker Hub for CI/CD

To automatically build and deploy Docker images, your GitLab pipeline needs a central place to store them. Docker Hub is a popular container registry for this purpose. This guide covers creating an account, a private repository, and an access token for your pipeline.

Step 1: Create a Docker Hub Account

  1. Go to the Docker Hub website: hub.docker.com
  2. Click Sign Up and fill out the form. Your Docker ID is your unique username and will be part of your image names (e.g., your-docker-id/my-app).
  3. Verify your email address to activate your account.

Step 2: Create a Private Repository

A repository is where your specific application images will be stored.

  1. Log into Docker Hub and click the Create Repository button.
  2. Name your repository (e.g., test-flask).
  3. Set the Visibility to Private. This ensures only you and your pipeline can access the images.
  4. Click Create. Your full image name will now be your-docker-id/test-flask.

Step 3: Generate an Access Token

Your CI/CD pipeline needs a token, not your password, to securely log in and push images.

(Web App)Deployment using CI/CD 

  1. In Docker Hub, click on your username in the top-right corner and go to Account Settings.
  2. Navigate to the Security tab and click New Access Token.
  3. Give the token a descriptive name, like gitlab-runner-token.
  4. Set the permissions to Read, Write, Delete, which allows the pipeline to manage images.
  5. Click Generate.
  6. Important: Docker Hub will show you the token only once. Copy it immediately and save it in a secure place, like your GitLab CI/CD variables (DOCKERHUB_TOKEN). 🔐


Learn to troubleshoot: Here are the files needed to deploy a containerised flash web app. There is however, something causing the deployment to not start. Try to deploy with these files to a new project and see if you can troubleshoot the issue. 

Create a file named .gitlab-ci.yml in your project root with the following content:

stages:
  - build
  - test

variables:
  PYTHON_VERSION: "3.9" 
  FLASK_APP: "app.py"   

build_job:
  stage: build
  image: python:${PYTHON_VERSION}-slim-bullseye
  script:
    - echo "Installing Python dependencies..."
    - pip install -r requirements.txt
    - echo "Python web application dependencies installed."
  tags:
    - home-lab
    - docker

test_job:
  stage: test
  image: python:${PYTHON_VERSION}-slim-bullseye
  script:
    - echo "Updating apt and installing curl and procps..."
    - apt-get update -y
    - apt-get install -y curl procps 
    - echo "Curl and procps installed."
    - echo "Running Flask application in background for testing..."
    - pip install -r requirements.txt
    - nohup python -m flask run --host=0.0.0.0 --port=5000 > /dev/null 2>&1 &
    - sleep 5
    - echo "Attempting to access the Flask app..."
    - curl http://localhost:5000 | grep "Hello World!"
    - if [ $? -eq 0 ]; then echo "Flask app is serving 'Hello World!' successfully."; else echo "Flask app did NOT serve 'Hello World!'"; exit 1; fi
    - echo "Killing Flask application process..."
    - pkill -f "flask run" 
  tags:
    - home-lab
    - docker

Create a file named app.py in your project root with the following content:

# app.py
from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_world():
    """
    Renders a simple "Hello World" message when the root URL is accessed.
    """
    return '<h1>Hello World!</h1><p>This is a simple Flask application.</p>'

if __name__ == '__main__':
    # Run the Flask application on all available interfaces (0.0.0.0)
    # and port 5000. debug=True is for development, set to False in production.
    app.run(host='0.0.0.0', port=5000, debug=False)

Create a file named requirements.txt in your project root with the following content:

# requirements.txt
Flask==2.3.3

Prepare For The Deployment Stage:

GitLab CI/CD Variables for Deployment

To configure these variables:

  1. Navigate to your GitLab project.
  2. Go to Settings > CI/CD.
  3. Expand the Variables section.
  4. Click Add variable for each entry below and fill in the details as specified.

1. SERVER_IP

  • Key: SERVER_IP
  • Purpose: Specifies the IP address (or hostname) of your target deployment server. The GitLab Runner will use this address to establish an SSH connection.
  • Recommended Value: The public IP address of your lab server. If your GitLab instance, Runner, and deployment target are all on the same machine, using the public IP ensures the environment.url link in GitLab is externally accessible.
  • Type: Variable
  • Mask variable: ✅ Checked (Recommended to hide the IP in job logs for security, though IPs are often considered less sensitive than credentials).
  • Protected variable: ✅ Checked (Recommended to restrict its availability to pipelines running on protected branches/tags, like main).

2. SERVER_USER

  • Key: SERVER_USER
  • Purpose: Defines the username on your target deployment server that the GitLab Runner will use to log in via SSH. This user must have permissions to run Docker commands (i.e., be part of the docker group on your server).
  • Recommended Value: The actual username on your lab server (e.g., ubuntu or a dedicated deployment user).
  • Type: Variable
  • Mask variable: ✅ Checked (Recommended to hide the username in job logs).
  • Protected variable: ✅ Checked (Recommended for security).

3. SSH_PRIVATE_KEY

  • Key: SSH_PRIVATE_KEY
  • Purpose: Contains the sensitive private key portion of your SSH key pair. This key is used by the GitLab Runner to authenticate with your homelab server via SSH without needing a password.
  • Recommended Value: The entire content of your SSH private key file (e.g., ~/.ssh/gitlab_ci_deploy_key). This must include the —–BEGIN OPENSSH PRIVATE KEY—– and —–END OPENSSH PRIVATE KEY—– header/footer lines. Crucially, ensure you press Enter (or Return) at the very end of the last line after pasting the key content.
  • Type: File (This is critical! Setting it as File type tells GitLab to save the content to a temporary file, which ssh-add can then correctly process.)
  • Mask variable: ❌ Unchecked (File type variables cannot be masked, but GitLab handles their security by saving them to a temporary, job-specific file that is automatically removed.)
  • Protected variable: ✅ Checked (Highly recommended, as this is a direct credential for your server. It restricts its use to protected branches/tags.)

4. DOCKERHUB_USERNAME

  • Key: DOCKERHUB_USERNAME
  • Purpose: Specifies your Docker Hub username (also known as Docker ID). This is used by the pipeline to log into the Docker Hub registry and to correctly tag your Docker images in the required username/repository:tag format.
  • Recommended Value: Your unique Docker Hub username (e.g., DOCKERHUB_USER).
  • Type: Variable
  • Mask variable: ✅ Checked. While usernames are often public, masking them is a good security habit to avoid leaking unnecessary information in job logs.
  • Protected variable: ✅ Checked. Pushing images to your registry is a trusted action that should be restricted to protected branches (like main).

5. DOCKERHUB_TOKEN

  • Key: DOCKERHUB_TOKEN
  • Purpose: Contains the Personal Access Token (PAT) that serves as the password for logging into Docker Hub from an automated system. The CI/CD pipeline uses this token to securely authenticate with the registry before pushing your newly built Docker image.
  • Recommended Value: The full Personal Access Token generated from your Docker Hub account’s Account Settings > Security page.
  • Type: Variable
  • Mask variable: ✅ Checked (Critical). This token is a sensitive credential and must be masked to prevent it from being exposed in job logs.
  • Protected variable: ✅ Checked (Critical). As a direct credential for your image registry, its use should be restricted to pipelines running on protected branches only.

Advanced Pipeline Deployment to a Lab Server

You can extend your pipeline to deploy applications to a server within your home lab. This often involves using SSH for remote execution. Before running this deployment, I recommend experimenting on your local machine with Docker and a virtual Python environment. Here is a good example of a similar project. https://www.freecodecamp.org/news/how-to-dockerize-a-flask-app/

Running this locally will help you understand the concepts of building images in a repository. A fundamental understanding of this will help you grasp the stages in the pipeline.

This is the complete file structure within your Git repository. Each file resides in the root directory.

.

├── .gitlab-ci.yml        # Defines the test, build, and deploy pipeline

├── app.py                # The main Flask web application code

├── Dockerfile            # Instructions to build the production Docker image

├── docker-compose.yml    # Defines the service for persistent deployment

├── requirements.txt      # Lists the Python dependencies (Flask, Gunicorn, etc.)

└── test_app.py           # Contains the automated tests for the ‘test’ stage

After a successful deployment, your server will have the following minimal structure, created and managed automatically by your GitLab pipeline.

/opt/

└── flask-app/

    └── docker-compose.yml  # Transferred by the pipeline to manage the container

The actual Docker container running your Flask app is managed by the Docker daemon on the server but doesn’t reside as a file in this directory. 🚀

.gitlab-ci.yml

stages:
  - test
  - build
  - deploy

variables:
   IMAGE_NAME: REPLACE-WITH-DOCKERHUB-USER/REPLACE-WITH-DOCKERHUB-REPO
   IMAGE_TAG: ${CI_COMMIT_SHORT_SHA}
   IMAGE_FULL_NAME: REPLACE-WITH-DOCKERHUB-USER/REPLACE-WITH-DOCKERHUB-REPO:${CI_COMMIT_SHORT_SHA}
   DOCKER_HOST: tcp://docker:2375
   DOCKER_TLS_CERTDIR: ""

test:
  stage: test
  image: python:3.9-slim-buster
  script:
    - pip install -r requirements.txt
    - pytest
  tags:
    - lab
    - docker

build:
  stage: build
  image: docker:latest
  tags:
    - lab
    - docker
  services:
    - name: docker:dind #Provides a Docker engine (daemon) for this job to use.
      command: ["--tls=false"]  
  script:
    - docker login -u $DOCKERHUB_USERNAME -p $DOCKERHUB_TOKEN
    - docker build -t $IMAGE_FULL_NAME .
    - docker push $IMAGE_FULL_NAME
deploy:
  stage: deploy
  image: alpine:latest
  tags:
    - lab
    - docker
  before_script:
    # Install ssh-client
    - apk add --no-cache openssh-client
    # Set up the SSH key from the CI/CD variable
    - eval $(ssh-agent -s)
    - chmod 400 /builds/root/testproject.tmp/SSH_PRIVATE_KEY
    - ssh -i "/builds/root/testproject.tmp/SSH_PRIVATE_KEY" -o BatchMode=yes -o StrictHostKeyChecking=no -T $SERVER_USER@$SERVER_IP echo "SSH connection successful" || echo "SSH connection failed"
  script:
    - |
      echo "Deploying to the production server..."
      ssh -o StrictHostKeyChecking=no -i "/builds/root/testproject.tmp/SSH_PRIVATE_KEY" ${SERVER_USER}@${SERVER_IP} 'sudo mkdir -p /opt/flask-app && sudo tee /opt/flask-app/docker-compose.yml >/dev/null' < docker-compose.yml
      ssh -o StrictHostKeyChecking=no -i "/builds/root/testproject.tmp/SSH_PRIVATE_KEY" ${SERVER_USER}@${SERVER_IP} "
        docker login -u '${DOCKERHUB_USERNAME}' -p '${DOCKERHUB_TOKEN}'
        export IMAGE_FULL_NAME=${IMAGE_FULL_NAME}
        cd /opt/flask-app && docker compose pull && docker compose up -d
      "
      
      echo "Deployment successful!"

docker-compose.yml

version: '3.8'
services:
 web:
   image: ${IMAGE_FULL_NAME}
   container_name: flask-app-persistent
   restart: always # This makes the container persistent
   ports:
     - "5000:5000"

dockerfile

# Dockerfile

# 1. Use an official Python runtime as a parent image
FROM python:3.9-slim-buster

# 2. Set the working directory in the container
WORKDIR /app

# 3. Copy and install the dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# 4. Copy the application code into the container
COPY . .

# 5. Expose the port Gunicorn will run on
EXPOSE 5000

# 6. Command to run the application using Gunicorn
CMD ["gunicorn", "--bind", "0.0.0.0:5000", "app:app"]

requirements.txt

Flask
gunicorn
pytest

test_app.py

from app import app

def test_hello():
    """
    GIVEN a Flask application
    WHEN the '/' page is requested (GET)
    THEN check that the response is valid
    """
    with app.test_client() as client:
        response = client.get('/')
        assert response.status_code == 200
        assert b"Hello from a persistent Flask & Docker container!" in response.data

app.py

from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_geek():
    return '<h1>Hello from a persistent Flask & Docker container! 🐳</h1>'

Key Additions:

  • artifacts: Specifies files or directories that should be made available to subsequent jobs.
  • deploy_job: A new stage for deployment.
  • only: – main: This job will only run when changes are pushed to the main branch.
  • before_script: Commands executed before the main script of the job. Here, it sets up SSH for passwordless access to your home lab server using an SSH key.
  • SSH_PRIVATE_KEY: This should be set as a CI/CD variable in your GitLab project settings (Settings > CI/CD > Variables). Never hardcode sensitive information in your .gitlab-ci.yml file.

Troubleshooting

  • Runner Not Picking Up Jobs: Ensure your runner is registered, active, and has the correct tags. Check the runner’s logs.
  • SSH Issues: Verify SSH key permissions, known_hosts entries, and that the SSH_PRIVATE_KEY variable is correctly set in GitLab.
  • Job Failures: Review the job logs in GitLab for detailed error messages.

Conclusion

GitLab CI/CD pipelines provide a powerful framework for automating your workflows. By investing time in setting up and configuring these pipelines, you can significantly improve the efficiency and reliability of your personal projects, from development to deployment.

About the Author: Phil

Leave a Reply

Your email address will not be published. Required fields are marked *