Contents

Setup gitlab runner

gitlab CI/CD 1388 words 7 minutes

Setup gitlab-runner for CI/CD

Note: Steps in this article follows gitlab-runner’s installation on ubuntu OS. I will be using Ubuntu 20.04.

  1. Installation of gitlab-runner
  2. Register gitlab-runner
  3. .gitlab-ci.yml examples

You can also follow official documentation for installation of gitlab-runner here.

1

curl -LJO https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_<arch>.deb

Find architectures here.

gitlab-runner has dependency on git.

1
2

sudo apt-get update
sudo apt-get install git -y

1

dpkg -i gitlab-runner_<arch>.deb

Check installation by executing gitlab-runner status. A successful response should look like the following:

1
2

Runtime platform                                    arch=amd64 os=linux pid=12090 revision=8fa89735 version=13.6.0
gitlab-runner: Service is running!

You can register gitlab-runner by executing the following command.

1

gitlab-runner register

You will be required to enter 4 inputs when prompted:

  1. gitlab website: If you are hosting your own gitlab server, you need enter its address. Since I am using gitlab.com, I will enter https://gitlab.com/.
  2. registration token: You can your project’s registration token in project’s Settings -> CI/CD -> Runners -> Set up a specific Runner manually.
  3. tags: You can create a tag for this particular gitlab-runner, which you can later use in .gitlab-ci.yml to address/use this instance of gitlab-runner. In case you have more than one instance of gitlab-runner with different configurations and resources, tags will help you address the instance you want to use for a certain job.
  4. executer: Here you select one of the options (docker-ssh, ssh, docker+machine, docker-ssh+machine, kubernetes, custom, docker, parallels, shell, virtualbox) to execute your jobs. You can read more about these options here. For the purposes of testing, I will select shell as executor.

Once the registration of gitlab-runner completes. You will see the following section appear in Settings -> CI/CD -> Runners -> Set up a specific Runner manually.

/gitlab-runner/gitlab-runner-registered.png
after registering gitlab-runner

Following is the Bash sample yml file provided by gitlab. Lets see analyse it before modifying it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# This file is a template, and might need editing before it works on your project.
# see https://docs.gitlab.com/ee/ci/yaml/README.html for all available options

# you can delete this line if you're not using Docker
image: busybox:latest

before_script:
  - echo "Before script section"
  - echo "For example you might run an update here or install a build dependency"
  - echo "Or perhaps you might print out some debugging details"

after_script:
  - echo "After script section"
  - echo "For example you might do some cleanup here"

build1:
  stage: build
  script:
    - echo "Do your build here"

test1:
  stage: test
  script:
    - echo "Do a test here"
    - echo "For example run a test suite"

test2:
  stage: test
  script:
    - echo "Do another parallel test here"
    - echo "For example run a lint test"

deploy1:
  stage: deploy
  script:
    - echo "Do your deploy here"

Above yml file produces the following pipeline:

/gitlab-runner/pipeline-1.png
sample gitlab pipeline

build1 job has the following log:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

Preparing the "docker+machine" executor
00:13
Using Docker executor with image busybox:latest ...
Pulling docker image busybox:latest ...
Using docker image sha256:dc3bacd8b5ea796cea5d6070c8f145df9076f26a6bc1c8981fd5b176d37de843 for busybox:latest with digest busybox@sha256:9f1c79411e054199210b4d489ae600a061595967adb643cd923f8515ad8123d2 ...
Preparing environment
00:01
Running on runner-ed2dce3a-project-22692563-concurrent-0 via runner-ed2dce3a-srm-1606320958-f3e4e349...
Getting source from Git repository
00:01
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/khan990/experiment-gitlab-runner/.git/
Created fresh repository.
Checking out c8ad39b1 as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:01
$ echo "Before script section"
Before script section
$ echo "For example you might run an update here or install a build dependency"
For example you might run an update here or install a build dependency
$ echo "Or perhaps you might print out some debugging details"
Or perhaps you might print out some debugging details
$ echo "Do your build here"
Do your build here
Running after_script
00:01
Running after script...
$ echo "After script section"
After script section
$ echo "For example you might do some cleanup here"
For example you might do some cleanup here
Cleaning up file based variables
00:01
Job succeeded

Note, that the very first line shows that this build is not executed on our gitlab-runner, but it ran on one of the common Shared runners that you can see in Settings -> CI/CD -> Runners -> Set up a specific Runner manually.

To run it in our own runner, lets first disable shared runner.

To run jobs using our runner, we have two options.

  1. Open up runner options in Settings -> CI/CD -> Runners -> Set up a specific Runner manually. Check Run untagged jobs and save.
  2. Mention runner tag in .gitlab-ci.yml file like the following:
1
2
3
4

default:
    tags:
        - sample
#...

Follow one of the options and then restart the job to check if runner works. Which it does, but fails, because we choose a shell runner, but our yml requires docker machine. Remove image property from ci. Resulting .gitlab-ci.yml will look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

default:
  tags:
    - sample

before_script:
  - echo "Before script section"
  - echo "For example you might run an update here or install a build dependency"
  - echo "Or perhaps you might print out some debugging details"

after_script:
  - echo "After script section"
  - echo "For example you might do some cleanup here"

build1:
  stage: build
  script:
    - echo "Do your build here"

test1:
  stage: test
  script:
    - echo "Do a test here"
    - echo "For example run a test suite"

test2:
  stage: test
  script:
    - echo "Do another parallel test here"
    - echo "For example run a lint test"

deploy1:
  stage: deploy
  script:
    - echo "Do your deploy here"
Info
On your ubuntu(gitlab-runner host) machine, you can also execute gitlab-runner --debug run to verbose log of gitlab-runner.

Lets invent a scenario, where we would like to do the following:

  • We have following stages:
    • Build
    • Test
    • Push
    • Tag(docker)
    • Deploy
    • Release
  • And following jobs:
    • build
    • test:lint
    • test:unit
    • test:e2e
    • push(docker)
    • tag(docker)
    • release(git version)
    • deploy:develop
    • deploy:staging
    • deploy:live
Warning
This section is work in progress

WHEN:

  • Code is pushed to a feature branch, no pipeline runs.

  • Merge request is created from feature branch(feature/*) to develop(develop), pipeline runs with following jobs:

    • build
    • test:lint
    • test:unit
    • push(docker)
    • deploy:develop

  • Code is pushed to develop:

    • build
    • test:lint
    • test:unit
    • test:e2e
    • push(docker)
    • deploy:develop

  • Merge request is created from develop(develop) to master(master), pipeline is skipped.

    bug
    Above requirement is still not fulfilled.

  • Merge request of develop(develop) to master(master) is accepted.

    • build
    • test:lint
    • test:unit
    • test:e2e
    • release(git version)
    • push(docker)
    • tag(docker)

  • Master is tagged

    • deploy:live

  • Create release branch (release/vX.X.X)

    • deploy:staging 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135


stages:
  - Build
  - Test
  - Release
  - Push
  - Tag
  - Deploy

build:
  stage: Build
  script:
    - echo "This is build job"
  only:
    refs:
      - merge_requests
      - develop
      - master
  except:
    variables:
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /develop/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /master/)
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /master/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /develop/)


test:lint:
  stage: Test
  script:
    - echo "This is test:lint job"
  only:
    refs:
      - merge_requests
      - develop
      - master
  except:
    variables:
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /develop/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /master/)
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /master/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /develop/)
  dependencies:
    - build

test:unit:
  stage: Test
  script:
    - echo "This is test:unit job"
  only:
    refs:
      - merge_requests
      - develop
      - master
  except:
    variables:
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /develop/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /master/)
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /master/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /develop/)
  dependencies:
    - build

test:e2e:
  stage: Test
  script:
    - echo "This is test:e2e job"
  only:
    refs:
      - develop
      - master
  except:
    variables:
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /develop/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /master/)
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /master/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /develop/)
  dependencies:
    - build

release:
  stage: Release
  script:
    - echo "This is release(git version) job"
  only:
    refs:
      - master
  dependencies:
    - test:lint
    - test:e2e
    - test:unit


push:
  stage: Push
  script:
    - echo "This is push(docker) job"
  only:
    refs:
      - merge_requests
      - develop
      - master
  except:
    variables:
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /develop/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /master/)
      - ($CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == /master/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == /develop/)

tag:
  stage: Tag
  script:
    - echo "This is tag(docker) job"
  only:
    refs:
      - master
  dependencies:
    - push

deploy:develop:
  stage: Deploy
  script:
    - echo "This is deploy:develop job"
  only:
    refs:
      - merge_requests
      - develop
  dependencies:
    - push

deploy:staging:
  stage: Deploy
  script:
    - echo "This is deploy:staging job"
  only:
    refs:
      - /^release\/v[0-9]+\.[0-9]+\.[0-9]+/

deploy:live:
  stage: Deploy
  script:
    - echo "This is deploy:live job"
  only:
    refs:
      - tags

Комментарии