GitLab CI/CD Documentation

Comprehensive guides for efficient, cost-effective CI/CD pipelines across the Agent Platform.

Overview

This documentation covers GitLab CI/CD best practices with extreme focus on cost optimization and preventing wasted CI minutes. Essential reading for managing 70+ projects efficiently.

Documentation Structure

Critical Priority

Cost Optimization - READ THIS FIRST

• Preventing wasted CI minutes
• Interruptible jobs
• Draft MR optimization
• Conditional execution
• Fail-fast patterns
• Runner sizing
• Expected savings: 40-80% of CI costs

Why critical: User has been burning money on inefficient pipelines. This guide provides immediate cost-saving techniques.

High Priority

Caching Strategies

• Docker layer caching (70-90% build time reduction)
• Dependency caching (npm, pip, Go, Maven)
• Build artifact caching
• Cache configuration best practices
• Troubleshooting cache issues

Pipeline Efficiency

• needs vs dependencies
• DAG (Directed Acyclic Graph) pipelines
• Parallelization strategies
• Critical path optimization
• Performance patterns

Validation

• Local pipeline testing with gitlab-ci-local
• Pre-commit hooks for validation
• GitLab CI Lint API
• Schema validation
• Testing before push (save 75% of wasted minutes)

Components

• Building reusable CI/CD components
• Component inputs and versioning
• Publishing to CI/CD Catalog
• Managing components across 70+ projects
• Migration from templates

Multi-Project Management

• Parent-child pipelines (monorepo)
• Multi-project pipelines (dependencies)
• Component libraries at scale
• Centralized configuration
• Governance across 70+ projects

Monitoring

• CI/CD analytics and dashboards
• Cost monitoring and budget alerts
• Performance metrics
• DORA metrics
• Audit and compliance

Medium Priority

Patterns

• Common pipeline patterns
• Workflow patterns
• Deployment strategies
• Testing patterns
• Anti-patterns to avoid

Quick Start Guides

For New Projects

1. Start with cost optimization: Read cost-optimization.md
2. Enable caching: Follow caching.md for your language
3. Use components: Reference components.md for approved components
4. Validate locally: Set up gitlab-ci-local
5. Monitor costs: Track usage per monitoring.md

For Existing Projects

1. Audit current costs: monitoring.md Usage tracking
2. Apply quick wins: cost-optimization.md Best practices summary
3. Add caching: caching.md Language-specific guides
4. Optimize DAG: pipeline-efficiency.md needs keyword
5. Migrate to components: components.md Using components

For Platform Team

1. Create component library: components.md Creating components
2. Set up monitoring: monitoring.md Custom dashboards
3. Enforce compliance: multi-project.md Compliance pipelines
4. Track costs: monitoring.md Cost attribution
5. Establish governance: multi-project.md Governance

Cost Savings Summary

Immediate Actions (Day 1)

Implementation time: 2-4 hours per project

Total potential savings: 40-80% of current CI costs

Short-term Optimizations (Week 1)

Long-term Improvements (Month 1)

Key Concepts

CI Minute Cost Factors

Compute minutes = wall-clock time runner multiplier

Example:

• 10-minute job on small runner (1x) = 10 CI minutes
• 10-minute job on large runner (4x) = 40 CI minutes
• 10-minute job on x-large runner (8x) = 80 CI minutes

Cost optimization focus: Reduce both time and multiplier.

Pipeline Efficiency Factors

1. Caching: Avoid re-downloading dependencies
2. Parallelization: Run jobs concurrently
3. DAG with needs: Don't wait for unrelated jobs
4. Conditional execution: Skip unnecessary jobs
5. Fail fast: Exit early on critical failures

Component-Based Architecture

Traditional (70 projects):

• 70 pipeline configs to maintain
• Changes require 70 updates
• Inconsistent practices
• High maintenance overhead

Component-Based (70 projects):

• 1 component library
• Changes affect all projects via version bump
• Consistent practices enforced
• Low maintenance overhead

Common Workflows

Daily Development

# 1. Edit code
vim src/app.js

# 2. Validate pipeline locally
gitlab-ci-local

# 3. Commit (pre-commit hook validates)
git commit -m "feat: add feature"

# 4. Push
git push

# 5. Monitor in GitLab UI
# Pipeline runs efficiently with all optimizations

Updating Components

# Component library maintainer
cd ci-components
vim templates/node-build/template.yml

# Test changes
gitlab-ci-local

# Release new version
git tag v2.0.0
git push origin v2.0.0

# Gradual rollout
# Week 1: 5 pilot projects update to v2.0.0
# Week 2: 20 more projects
# Week 3: Remaining 45 projects

Cost Monitoring

# Weekly cost review
./check-ci-costs.sh

# Output:
# Total usage: 38,500 / 50,000 minutes (77%)
# Top consumers:
#   project-a: 8,500 minutes
#   project-b: 6,200 minutes
#   project-c: 4,800 minutes

# Investigate top consumers
./analyze-project.sh project-a

# Apply optimizations
# Expected reduction: 50%  4,250 minutes saved

Resources

GitLab Documentation

• CI/CD Pipelines
• CI/CD Components
• CI/CD Analytics
• Compute Minutes
• YAML Reference

External Resources

• gitlab-ci-local - Local pipeline testing
• pre-commit - Pre-commit hook framework
• DORA Metrics - DevOps performance metrics

Internal Resources

• GitLab Ultimate Features - Observability, security, AI features
• Issue-Driven Development
• Release Strategy

Support

Questions or Issues

1. Review documentation: Check relevant guide above
2. Search GitLab docs: docs.gitlab.com
3. Check examples: Review working pipelines in other projects
4. Test locally: Use gitlab-ci-local to iterate quickly

Contributing

Found an issue or have an improvement?

1. Create GitLab issue in technical-docs.wiki project
2. Follow issue-driven development workflow
3. Submit merge request with changes
4. Update this documentation

Changelog

2026-01-08

• Initial documentation creation
• All 8 guides completed
• Focus on cost optimization
• Comprehensive coverage of GitLab CI/CD features

---

Last Updated: 2026-01-08 Maintained by: Platform Team Priority: CRITICAL - Cost optimization essential