Problem Statement
What strategies optimize build performance in CI/CD? Discuss caching, parallelization, incremental builds, and distributed compilation.
Explanation
Caching strategies dramatically reduce build time by reusing artifacts from previous builds:
Dependency caching stores downloaded packages:
```yaml
# GitLab CI
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .m2/repository/
- node_modules/
- target/
```
GitHub Actions:
```yaml
- uses: actions/cache@v3
with:
path: |
~/.m2/repository
~/.gradle/caches
~/.npm
key: ${{ runner.os }}-build-${{ hashFiles('**/package-lock.json', '**/pom.xml') }}
restore-keys: |
${{ runner.os }}-build-
```
Shared cache across builds and branches significantly reduces dependency download time.
Parallelization runs independent tasks concurrently:
```yaml
# GitLab parallel jobs
test_unit:
stage: test
script: npm run test:unit
test_integration:
stage: test
script: npm run test:integration
test_e2e:
stage: test
script: npm run test:e2e
```
Three test suites run simultaneously if multiple runners available.
Matrix parallelization:
```yaml
# GitHub Actions
strategy:
matrix:
node: [14, 16, 18]
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- run: npm test
```
Creates 6 parallel jobs (3 Node versions × 2 OSes).
Incremental compilation compiles only changed files:
Gradle:
```groovy
tasks.withType(JavaCompile) {
options.incremental = true
options.fork = true
options.forkOptions.jvmArgs = ['-Xmx2g']
}
```
TypeScript:
```json
{
"compilerOptions": {
"incremental": true,
"tsBuildInfoFile": ".tsbuildinfo"
}
}
```
Cache .tsbuildinfo between builds.
Build caching for task outputs (Gradle):
```groovy
buildCache {
local {
enabled = true
}
remote(HttpBuildCache) {
url = 'https://build-cache.company.com/'
push = true
}
}
```
Distributed compilation spreads work across multiple machines:
Distcc (C/C++):
```bash
export DISTCC_HOSTS='localhost build1.company.com build2.company.com'
make -j8 CC=distcc
```
Icecc alternative supports heterogeneous environments.
Gradle distributed builds with build cache enable sharing task outputs across machines in team.
Docker layer caching:
```yaml
# GitLab
build:
image: docker:20.10
services:
- docker:20.10-dind
script:
- docker build --cache-from $CI_REGISTRY_IMAGE:latest -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
```
BuildKit cache mounts:
```dockerfile
RUN --mount=type=cache,target=/root/.m2 mvn package
RUN --mount=type=cache,target=/root/.npm npm ci
```
Resource optimization:
```yaml
# Allocate more resources for heavy builds
build:
tags:
- high-memory
- high-cpu
variables:
MAVEN_OPTS: "-Xmx4g"
GRADLE_OPTS: "-Xmx4g -XX:MaxMetaspaceSize=512m"
```
Skip unnecessary work:
```yaml
build:
script:
- mvn package -DskipTests # Tests run in separate stage
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
- changes:
- src/**/*
```
Warmup builds for cold start reduction:
```yaml
warmup:
stage: .pre
script:
- mvn dependency:go-offline
- docker pull $BASE_IMAGE
cache:
key: warmup
paths:
- .m2/repository/
```
Monitoring and profiling:
```bash
# Gradle build scan
./gradlew build --scan
# Maven with timing
mvn clean install -Dorg.slf4j.simpleLogger.showDateTime=true
# Profile Docker build
time docker build --progress=plain .
```
Optimization checklist:
1. Cache dependencies aggressively
2. Parallelize independent tasks
3. Enable incremental compilation
4. Use build caching (Gradle, Bazel)
5. Optimize Docker layers
6. Use distributed compilation for C/C++
7. Skip unnecessary stages
8. Allocate appropriate resources
9. Monitor build times
10. Profile to identify bottlenecks
Before optimization, measure baseline. After changes, measure improvements. Focus on slowest stages first for maximum impact. Understanding optimization techniques reduces build time from hours to minutes, improving developer productivity.