In this article, I’m gonna walk you through configuring on-premise GitLab Server, GitLab Runner, and CI/CD for containerized microservices with NGINX configuration as a load balancer. We’ll deploy and run all the containerized microservices on the same machine where GitLab and Runner are set up.

GitLab Server

GitLab allows you to host an on-premise GitLab server(or Git repository) that can be accessed from LAN(or WAN if you have a public IP address).

We can install the GitLab server either on a container environment like docker or on the host machine. In this, I’ll install it on the host machine (i.e. Ubuntu 20.04 machine).

Prerequisites

Machine with at least 2 cores and 4GB of memory.

On-Premise GitLab Server

On-Premise Installation

Step-1: Update System and Install Required Dependencies

Open the terminal on the server(machine) and run the following commands:

$ sudo apt update
$ sudo apt upgrade -y

After running these, let’s install some dependencies by running:

$ sudo apt install -y ca-certificates curl openssh-server tzdata perl

Step-2: Install Postfix for Email Notifications (Optional)

Optionally, if you want to use the same system or server to send email notifications to users, then you can install postfix, an open-source mail transfer agent.

$ sudo apt install postfix -y

While installing Postfix you’ll be asked to set it up, select ‘Internet Site’ option and add your DNS for mail name and configure other required things.

Further, you’ll need to install the mailutils package:

$ sudo apt install mailutils

OR, you can set up SMTP server to send email notifications instead of Postfix. To do this, you need to first install GitLab and edit /etc/gitlab/gitlab.rb.

Step-3: Add GitLab Package Repository and GPG Key

As the GitLab is not available on the base repository of Ubuntu. You need to add the GitLab package repository and GPG key by running the following command:

$ curl -sS <https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh> | sudo bash

Here, I’ve added a package repository for GitLab Enterprise Edition, you can also add for GitLab Community Edition.

After adding the repository package, you can see the repository contents in:

$ cat /etc/apt/sources.list.d/gitlab_gitlab-ce.list

Step-4: Install GitLab EE on Ubuntu (22.04 | 20.04 | 18.04)

Now update the system again and install GitLab EE on the machine:

$ sudp apt update
$ sudp apt install gitlab-ee

You’ll see output similar:

It looks like GitLab has not been configured yet; skipping the upgrade script.

       *.                  *.
      ***                 ***
     *****               *****
    .******             *******
    ********            ********
   ,,,,,,,,,***********,,,,,,,,,
  ,,,,,,,,,,,*********,,,,,,,,,,,
  .,,,,,,,,,,,*******,,,,,,,,,,,,
      ,,,,,,,,,*****,,,,,,,,,.
         ,,,,,,,****,,,,,,
            .,,,***,,,,
                ,*,.


     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \\
  / /_/ / / /_/ /___/ /_/ / /_/ /
  \\____/_/\\__/_____/\\__,_/_.___/


Thank you for installing GitLab!

Now edit external_url in /etc/gitlab/gitlab.rb to set hostname. You can also configure other parameters. Replace gitlab.example.com with valid domain name.

$ sudo nano /etc/gitlab/gitlab.rb
external_url "<http://gitlab.example.com>"

OR, If you’re not going to use DNS. You can simply use your server’s IP address like in my case my local machine’s IP as:

external_url "<http://10.10.5.55>"

When done start GitLab by running the following command:

$ sudo gitlab-ctl reconfigure

After successful reconfiguration, check the status of GitLab:

$ sudo gitlab-ctl status

The output will be similar:

run: alertmanager: (pid 92581) 18s; run: log: (pid 92343) 80s
run: gitaly: (pid 92590) 18s; run: log: (pid 91561) 189s
run: gitlab-exporter: (pid 92551) 20s; run: log: (pid 92078) 98s
run: gitlab-kas: (pid 92520) 22s; run: log: (pid 91845) 175s
run: gitlab-workhorse: (pid 92531) 21s; run: log: (pid 91985) 117s
run: grafana: (pid 92610) 17s; run: log: (pid 92471) 38s
run: logrotate: (pid 91486) 202s; run: log: (pid 91494) 201s
run: nginx: (pid 91993) 114s; run: log: (pid 92013) 110s
run: node-exporter: (pid 92540) 21s; run: log: (pid 92049) 104s
run: postgres-exporter: (pid 92601) 18s; run: log: (pid 92367) 76s
run: postgresql: (pid 91693) 184s; run: log: (pid 91704) 183s
run: prometheus: (pid 92560) 20s; run: log: (pid 92297) 88s
run: puma: (pid 91904) 132s; run: log: (pid 91917) 129s
run: redis: (pid 91521) 196s; run: log: (pid 91538) 193s
run: redis-exporter: (pid 92553) 20s; run: log: (pid 92217) 94s
run: sidekiq: (pid 91922) 126s; run: log: (pid 91934) 122s

Step-5: GitLab Web Interface

Once the installation completes, open your URL set up as external_url gitlab.example.com on your browser. In my case, I have set it up locally using my server’s IP address. So, I can access it through http://10.10.5.55/

Now, first, you can log in as a root user using username as root and password from /etc/gitlab/intial_root_password. (The password for root user is randomly generated and stored for 24 hours only.)

To check the password run the following:

$ cat /etc/gitlab/initial_root_password

# WARNING: This value is valid only in the following conditions
#          1. If provided manually (either via `GITLAB_ROOT_PASSWORD` environment variable or via `gitlab_rails['initial_root_password']` setting in `gitlab.rb`, it was provided before database was seeded for the first time (usually, the first reconfigure run).
#          2. Password hasn't been changed manually, either via UI or via command line.
#
#          If the password shown here doesn't work, you must reset the admin password following <https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password>.

Password: dfdfkOtOjWp7v70OjkjtadsdfsafsadnSJAhcDbCNo9nTNGVC5UoSCyE=

# NOTE: This file will be automatically deleted in the first reconfigure run after 24 hours.
Copy the password and login:

After logging in, first reset the root user password. To do this, go to root user profile > Preferences > Password. Change.

You can start, stop or restart all the GitLab components by using the following commands:

$ sudo gitlab-ctl start
$ sudo gitlab-ctl restart
$ sudo gitlab-ctl stop

Also, you can start individual components also:

$ sudo gitlab-ctl restart prometheus

ok: run: prometheus: (pid 2673) 8s

This is all you need to set up GitLab Server on Ubuntu Server.

GitLab Runner

GitLab Runner can be installed:

In a container(Docker, K8s, or OpenShift)
By downloading binary manually
By using repository packages

In this, we’ll be installing Runner on the local machine as I have to deploy microservices on the same machine where GitLab and Runner will be running.

GitLab Runner – Installation

If you’re using a different server for a runner than that GitLab is configured, you need to first ssh into the server and follow the following steps. Otherwise, you can directly follow from step-1.

Step-1: Add Official Repository

First, add the official GitLab Runner Repository using the below command. You can find the Official Repository here.

$ curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

Step-2: Install Runner on Ubuntu (22.04 | 20.04 | 18.04)

Run the following command to install GitLab Runner:

$ sudo apt update
$ sudo apt install gitlab-runner

(Optional) To install a specific version:

$ sudo apt install gitlab-runner=10.0.0

After Installation, check the GitLab Runner version:

$ sudo gitlab-runner --version

Output:

Version:      15.0.0
Git revision: febb2a09
Git branch:   15-0-stable
GO version:   go1.17.7
Built:        2022-05-19T20:03:43+0000
OS/Arch:      linux/amd64

To check the status:

$ sudo gitlab-runner status

Output:

Runtime platform                                    arch=amd64 os=linux pid=29368 revision=febb2a09 version=15.0.0
gitlab-runner: Service is running!

You can Start, Stop, and Restart GitLab Runner by running the following:

$ sudo gitlab-runner start
$ sudo gitlab-runner stop
$ sudo gitlab-runner restart

Step-3: Grant Permission to GitLab Runner User

After successful installation, you’ll see a gitlab-runner user in /home directory. And you need to grant the sudo permission to the gitlab-runner user. For this, open visudo file:

$ sudo visudo

Add the following in sudoers group and set NOPASSWD also as shown below:

gitlab-runner ALL=(ALL:ALL) ALL

gitlab-runner ALL=(ALL) NOPASSWD: ALL

Output:

GNU nano 4.8                                /etc/sudoers.tmp
#
# This file MUST be edited with the 'visudo' command as root.
#
# Please consider adding local content in /etc/sudoers.d/ instead of
# directly modifying this file.
#
# See the man page for details on how to write a sudoers file.
#
Defaults        env_reset
Defaults        mail_badpass
Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"
# Host alias specification
# User alias specification
# Cmnd alias specification
# User privilege specification
root    ALL=(ALL:ALL) ALL
gitlab-runner ALL=(ALL:ALL) ALL
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) ALL
# See sudoers(5) for more information on "#include" directives:
#includedir /etc/sudoers.d
gitlab-runner ALL=(ALL) NOPASSWD: ALL

Step-4: Register GitLab Runner

Now, it’s time to register GitLab Runner to GitLab Server which was set up before.

Log in to GitLab Server with username and password(create a user if you haven’t already and create a project).
Navigate to Settings and click on CI/CD and click on Expand of Runners section.

Copy the URL and registration token. And register the runner.

$ gitlab-runner register --name project-name-runner --url http://10.10.5.55/ --registration-token  GR1348941Ah6PxTBi5S5asJ7fzYSa

OR,

$ sudo gitlab-runner register

Note: During registration, you’ll be asked some questions such as tags, name, executor etc., Select executor as shell (If you’re using docker you can select docker or docker+machine as an executor). Since I’m deploying on the same machine I will select executor as shell. Or you can select whatever as per your requirements.

Now you have successfully performed GitLab Runner Registration. Check on the GitLab CI/CD > Runners section, you’ll see the newly registered runner.

Error: This job is stuck because the project doesn’t have any runners online assigned to it. Go to Runners page.

It’s due to tags you’ve added during configuration, but not added to your CI/CD job. To solve this, you can add tags to your CICD jobs Or do the following:

Go to Settings > CI/CD > Runners (Click on Expand)
Click on the edit icon of the newly created runner. And check Run untagged jobs box.

And here it is, you have successfully installed and registered gitlab-runner for your projects.

Uninstalling GitLab Runner

To completely remove gitlab-runner run the following commands:

$ sudo apt purge --autoremove -y gitlab-runner
$ sudo apt-key del 51312f3f
$ sudo rm -rf /etc/apt/sources.list.d/runner_gitlab-runner.list
$ sudo deluser --remove-home gitlab-runner
$ sudo rm -rf /etc/gitlab-runner

CI/CD for Containerized Applications

My project structure:

.
├── Dockerfile
├── .env
├── .git
├── .gitignore
├── .gitlab-ci.yml
├── package.json
├── README.md
├── .sample.env
└── src

Step-1: Configure .gitlab-ci.yml File

Now it’s time to configure CI/CD for containerized applications to deploy on the same machine where the GitLab Runner is set up. For this, let’s configure .gitlab-ci.yml file as:

stages:   
  - build
  - deploy

before_script:
    - IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"

dev-build-job:
  stage: build
  script:
    # - export DOCKER_HOST=tcp://0.0.0.0:2375/
    - cp $ENV_FILE .env
    - docker build -t $APP_NAME:latest .
  only:
    - dev
  tags:
    - cloudyfox

dev-deploy-job:
  stage: deploy
  script:
    - docker container rm -f $APP_NAME || true
    - docker run -d -p $PORT:$PORT --name $APP_NAME $APP_NAME:latest
  only:
    - dev
  tags:
    - cloudyfox

Step-2: Add Required Variables and Deploy

Here, the pipeline has two stages: build and deploy. IMAGE_TAG variable is used to tag docker images later.

In the build stage, $ENV_FILE variable is copied from the GitLab variables to the project root directory as it is in the .gitignore file. So, create ENV_FILE File variable with the value of .env file. To do so, Go to GitLab Server and navigate to Settings > CI/CD > Variables (Click on Expand). And add a new File variable.

Also, I have added APP_NAME and PORT variables as they are required for me.

In the deploy stage, the docker built in the previous build stage is deployed by deleting a container of the same name if running which is deployed then deploys our docker container.

In this way, you can deploy all the containerized microservices to the machine. Whenever you push the changes to the remote repository, a new docker version will be built and run on the machine(where all GitLab, and Runner are configured.).

Configuring Nginx As a Load Balancer

Let’s say you have successfully run all the micro applications using different ports on the server. Then you have to set up the Nginx load balancer to handle the requests. So that you can handle it in accordance with the users’ requests.

Since my GitLab server is running in the same local host, I need to use a different port to serve microservices. So, let’s start by configuring NGINX on the server.

Step-1: Install NGINX

$ sudo apt update
$ sudo -H apt install nginx-common nginx-full

Start Nginx:

$ sudo nginx

And start Nginx then check the status:

$ sudo systemctl start nginx
$ sudo systemctl status nginx

Output:

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-06-15 09:05:01 +0545; 5h 20min ago
       Docs: man:nginx(8)
    Process: 875 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, stat>
    Process: 1065 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUC>
   Main PID: 1066 (nginx)
      Tasks: 5 (limit: 9408)
     Memory: 4.7M
     CGroup: /system.slice/nginx.service
             ├─1066 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             ├─1067 nginx: worker process
             ├─1068 nginx: worker process
             ├─1069 nginx: worker process
             └─1070 nginx: worker process

You can Start, Stop, and Restart Nginx as:

$ sudo systemctl start nginx
$ sudo systemctl stop nginx
$ sudo systemctl restart nginx

Step-2: Configure Nginx Load Balancer for Dockerized Apps

To configure Nginx as Loadbalancer, first, remove default conf file, /etc/nginx/sites-enabled/default and create /etc/nginx/conf.d/lb.conf with following content(change as your requirements)

I’ll be routing all microservices which are running in different ports to http://10.10.5.55:9999. (as http://10.10.5.55 (port 80) is already used by the GitLab server)

The requests for different services will be load balanced as:

upstream user {
    server 127.0.0.1:5001;
}
upstream content {
    server 127.0.0.1:5002;
}
upstream discussion {
    server 127.0.0.1:5003;
}
upstream payment {
    server 127.0.0.1:5004;
}
upstream read {
    server 127.0.0.1:5007;
}
upstream subscription {
    server 127.0.0.1:5005;
}
upstream card {
    server 127.0.0.1:5006;
}
server {
    listen 9999;
    server_name localhost;
    location /api/user {
        proxy_pass http://user/api/user;
    }
    location /api/content {
        proxy_pass http://content/api/content;
    }
    location /api/discussion {
        proxy_pass http://discussion/api/discussion;
    }
    location /api/payment {
        proxy_pass http://payment/api/payment;
    }
    location /api/read {
        proxy_pass http://read/api/read;
    }
    location /api/subscription {
        proxy_pass http://subscription/api/subscription;
    }
    location /api/card {
        proxy_pass http://card/api/card;
    }
}

Here, every service has a different request URI, and all will be routed accordingly.

server_name is your domain name(if you have configured it previously), for me it’s localhost.

And test the syntax:

$ sudo nginx -t

Output:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Now, restart the Nginx service:

$ sudo service nginx restart

Go to the browser and run http://10.10.5.55:9999/. You’ll get successful output. (Change 10.10.5.55 with your server’s IP or domain name)

In my case:

And that’s all for Nginx Loadbalancer(reverse-proxy) configuration.

Conclusion

Here you go, you have successfully configured GitLab server, GitLab Runner, CI/CD for dockerized microservices, and NGINX as a load balancer(reverse-proxy) at once.

Explore more blogs of mine: https://scanskill.com/profile/sagar/

Thank you!

Sagar Budhathoki

Sagar Budhathoki

Setup On-premise GitLab Server, Runner, CI/CD, and Nginx Configuration

All in one Local Set Up

Table of contents

GitLab Server

Prerequisites

On-Premise GitLab Server

On-Premise Installation

Step-1: Update System and Install Required Dependencies

Step-2: Install Postfix for Email Notifications (Optional)

Step-3: Add GitLab Package Repository and GPG Key

Step-4: Install GitLab EE on Ubuntu (22.04 | 20.04 | 18.04)

Step-5: GitLab Web Interface

GitLab Runner

GitLab Runner – Installation

Step-1: Add Official Repository

Step-2: Install Runner on Ubuntu (22.04 | 20.04 | 18.04)

Step-3: Grant Permission to GitLab Runner User

Step-4: Register GitLab Runner

Error: This job is stuck because the project doesn’t have any runners online assigned to it. Go to Runners page.

Uninstalling GitLab Runner

CI/CD for Containerized Applications

Step-1: Configure .gitlab-ci.yml File

Step-2: Add Required Variables and Deploy

Configuring Nginx As a Load Balancer

Step-1: Install NGINX

Step-2: Configure Nginx Load Balancer for Dockerized Apps

Conclusion