Problem Statement
Explain GitLab Runner architecture, executor types, registration, and configuration. Include Docker, Kubernetes, and autoscaling setups.
Explanation
GitLab Runner is an application executing jobs from GitLab CI/CD. Runner architecture consists of runner manager (coordinates job execution), executor (environment where job runs), and job executor (specific implementation for platform). Runner polls GitLab instance for jobs matching its tags, downloads job artifacts, executes job script, uploads results and artifacts.
Executor types determine how jobs run. Shell executor runs commands directly on runner machine, sharing environment with runner process. Simple but no isolation:
```toml
[[runners]]
name = "shell-runner"
executor = "shell"
```
Use for simple cases or when runner has required tools installed.
Docker executor runs each job in fresh Docker container providing isolation, clean environment per job, and easy tool management via Docker images:
```toml
[[runners]]
name = "docker-runner"
executor = "docker"
[runners.docker]
image = "alpine:latest"
privileged = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
```
Jobs specify image:
```yaml
job:
image: node:18
script:
- npm test
```
Docker executor with DinD (Docker-in-Docker) enables building Docker images:
```yaml
build_image:
image: docker:20.10
services:
- docker:20.10-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
script:
- docker build -t myapp .
```
Kubernetes executor runs jobs as Kubernetes pods, ideal for cloud-native environments:
```toml
[[runners]]
name = "k8s-runner"
executor = "kubernetes"
[runners.kubernetes]
namespace = "gitlab-runner"
image = "alpine:latest"
privileged = false
cpu_request = "100m"
memory_request = "128Mi"
service_cpu_request = "100m"
service_memory_request = "128Mi"
```
Runner registration connects runner to GitLab:
```bash
gitlab-runner register \
--non-interactive \
--url "https://gitlab.com/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
--executor "docker" \
--docker-image "alpine:latest" \
--description "docker-runner" \
--tag-list "docker,linux" \
--locked="false"
```
Registration types: shared runners (available to all projects in GitLab instance), group runners (available to all projects in group), specific runners (tied to specific projects). Runners match jobs based on tags.
Autoscaling with Docker Machine enables dynamic runner provisioning based on load:
```toml
[[runners]]
executor = "docker+machine"
[runners.machine]
IdleCount = 2
IdleTime = 600
MaxBuilds = 100
MachineDriver = "amazonec2"
MachineName = "gitlab-runner-%s"
MachineOptions = [
"amazonec2-access-key=...",
"amazonec2-secret-key=...",
"amazonec2-region=us-east-1",
"amazonec2-instance-type=t3.medium"
]
```
Creates machines on demand when jobs queued, destroys after idle period.
Kubernetes autoscaling uses Kubernetes cluster autoscaler. Deploy runner as Kubernetes deployment, cluster automatically scales nodes based on pod resource requests.
Runner configuration file (config.toml) contains all runner settings:
```toml
concurrent = 4 # Max concurrent jobs
check_interval = 0
[[runners]]
name = "production-runner"
url = "https://gitlab.com/"
token = "TOKEN"
executor = "docker"
[runners.cache]
Type = "s3"
Shared = true
[runners.cache.s3]
ServerAddress = "s3.amazonaws.com"
BucketName = "runner-cache"
```
Cache configuration enables sharing cache between jobs:
```yaml
variables:
CACHE_KEY: "${CI_COMMIT_REF_SLUG}"
cache:
key: ${CACHE_KEY}
paths:
- node_modules/
policy: pull-push
```
Security considerations: don't use privileged mode unless necessary, isolate runners (dedicated machines/namespaces), use private Docker registry for base images, implement network policies restricting runner access, regularly update runners, rotate tokens, limit runner scope (specific > group > shared). Understanding runner architecture and configuration enables building scalable, secure CI/CD infrastructure.