Problem Statement
Explain performance and load testing in CI/CD including tools like JMeter, K6, Gatling, and strategies for baseline performance, regression detection, and scalability testing.
Explanation
Performance testing validates application speed, scalability, and stability under load. Types: load testing (normal expected load), stress testing (beyond normal capacity), spike testing (sudden traffic increase), endurance testing (sustained load over time).
Apache JMeter (GUI and CLI):
```xml
<!-- Create test plan programmatically -->
<jmeterTestPlan version="1.2">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan">
<elementProp name="TestPlan.user_defined_variables">
<collectionProp name="Arguments.arguments">
<elementProp name="USERS" elementType="Argument">
<stringProp name="Argument.value">100</stringProp>
</elementProp>
</collectionProp>
</elementProp>
</TestPlan>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup">
<stringProp name="ThreadGroup.num_threads">${USERS}</stringProp>
<stringProp name="ThreadGroup.ramp_time">10</stringProp>
<stringProp name="ThreadGroup.duration">60</stringProp>
</ThreadGroup>
</hashTree>
</jmeterTestPlan>
```
Run in CI/CD:
```yaml
performance_test:
stage: test
script:
- jmeter -n -t test-plan.jmx -l results.jtl -j jmeter.log
- jmeter-plugins-cmd --generate-png report.png --input-jtl results.jtl
artifacts:
paths:
- results.jtl
- report.png
only:
- main
```
K6 (modern, JavaScript-based):
```javascript
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate } from 'k6/metrics';
const errorRate = new Rate('errors');
export const options = {
stages: [
{ duration: '1m', target: 50 }, // Ramp up
{ duration: '3m', target: 100 }, // Stay at 100
{ duration: '1m', target: 0 }, // Ramp down
],
thresholds: {
http_req_duration: ['p(95)<500'], // 95% requests < 500ms
http_req_failed: ['rate<0.01'], // <1% error rate
errors: ['rate<0.1'],
},
};
export default function () {
const res = http.get('https://api.example.com/products');
const success = check(res, {
'status is 200': (r) => r.status === 200,
'response time < 500ms': (r) => r.timings.duration < 500,
});
errorRate.add(!success);
sleep(1);
}
```
CI/CD integration:
```yaml
k6_test:
stage: performance
image: grafana/k6:latest
script:
- k6 run --out json=results.json test.js
- k6 run --out influxdb=http://influxdb:8086 test.js
artifacts:
paths:
- results.json
only:
- main
```
Gatling (Scala-based, for JVM apps):
```scala
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
class BasicSimulation extends Simulation {
val httpProtocol = http
.baseUrl("https://api.example.com")
.acceptHeader("application/json")
val scn = scenario("API Load Test")
.exec(
http("Get Products")
.get("/products")
.check(status.is(200))
.check(responseTimeInMillis.lte(500))
)
.pause(1)
setUp(
scn.inject(
rampUsers(100) during (60 seconds)
)
).protocols(httpProtocol)
.assertions(
global.responseTime.max.lt(1000),
global.successfulRequests.percent.gt(95)
)
}
```
Baseline performance establishes reference metrics:
```yaml
baseline_test:
stage: performance
script:
- k6 run --vus 10 --duration 5m baseline.js
- python compare_with_baseline.py results.json baseline.json
only:
- main
```
Performance regression detection:
```python
# compare_with_baseline.py
import json
import sys
with open('results.json') as f:
current = json.load(f)
with open('baseline.json') as f:
baseline = json.load(f)
current_p95 = current['metrics']['http_req_duration']['p(95)']
baseline_p95 = baseline['metrics']['http_req_duration']['p(95)']
regression_threshold = 0.1 # 10% slower
if current_p95 > baseline_p95 * (1 + regression_threshold):
print(f"Performance regression detected: {current_p95}ms vs {baseline_p95}ms")
sys.exit(1)
else:
print(f"Performance acceptable: {current_p95}ms vs {baseline_p95}ms")
```
Scalability testing:
```javascript
// K6 scalability test
export const options = {
scenarios: {
low_load: {
executor: 'constant-vus',
vus: 10,
duration: '5m',
startTime: '0s',
},
medium_load: {
executor: 'ramping-vus',
startVUs: 10,
stages: [
{ duration: '2m', target: 50 },
{ duration: '5m', target: 50 },
],
startTime: '5m',
},
high_load: {
executor: 'ramping-vus',
startVUs: 50,
stages: [
{ duration: '2m', target: 100 },
{ duration: '5m', target: 100 },
{ duration: '2m', target: 0 },
],
startTime: '12m',
},
},
};
```
Monitoring during tests:
```yaml
performance_test:
stage: performance
services:
- name: grafana/grafana:latest
- name: influxdb:latest
script:
- k6 run --out influxdb=http://influxdb:8086 test.js
- curl -X POST http://grafana:3000/api/annotations \
-d '{"text":"Performance test completed"}'
```
Best practices: establish baseline early, run performance tests regularly (nightly or weekly), test realistic scenarios, monitor system resources (CPU, memory, database), test against production-like environment, use proper test data, include think time (user pauses), gradually increase load, fail builds on significant regressions, track metrics over time. Understanding performance testing prevents scalability issues in production.