multi project strategy

Multi-Project Component Strategy

This guide covers managing CI/CD components across 70+ projects in the Agent Platform.

Strategy Overview

Centralized Component Repository

Pattern: Single Component Repository Many Consuming Projects

gitlab_components/                    # Central component repository
 templates/
    docker-build.yml
    security-scan.yml
    k8s-deploy.yml
    node-pipeline.yml
    go-pipeline.yml
 .gitlab-ci.yml                   # Release automation

agent-project-1/                      # Consuming project
 .gitlab-ci.yml                   # Uses components
 ...

agent-project-2/                      # Consuming project
 .gitlab-ci.yml                   # Uses components
 ...

... (68 more projects)

Key Principles

Single Source of Truth: One repository for all shared components
Version Independence: Projects update at their own pace
Gradual Rollout: Test with subset before full deployment
Backward Compatibility: Maintain older versions during migration
Automated Testing: Validate changes before release

Component Organization

By Technology Stack

templates/
 docker/
    build.yml
    scan.yml
    push.yml
 node/
    pipeline.yml
    lint.yml
    test.yml
 go/
    pipeline.yml
    build.yml
 python/
    pipeline.yml
    test.yml
 deployment/
     k8s.yml
     helm.yml
     terraform.yml

By Pipeline Stage

templates/
 build/
    docker.yml
    npm.yml
    go.yml
 test/
    unit.yml
    integration.yml
    e2e.yml
 security/
    sast.yml
    dast.yml
    dependency-scan.yml
 deploy/
     staging.yml
     production.yml

By Function

templates/
 ci-full-pipeline.yml          # Complete CI pipeline
 cd-deployment.yml             # Deployment workflows
 security-comprehensive.yml    # All security scans
 quality-gates.yml            # Code quality checks
 notification.yml             # Slack/email notifications

Recommendation: Organize by technology stack for 70+ projects with diverse tech.

Version Management Strategy

Version Tiers

Tier 1: Stable (Production)

Current production version across all projects

# Version: 2.0.0
# Status: Stable
# Projects using: 65/70
# Support: Full support + security patches

Tier 2: Current (Recommended)

Latest version with new features

# Version: 2.1.0
# Status: Current
# Projects using: 5/70 (early adopters)
# Support: Full support

Tier 3: Preview (Beta)

Next version in testing

# Version: 3.0.0-beta.1
# Status: Preview
# Projects using: 0/70 (test projects only)
# Support: Best effort

Tier 4: Deprecated

Old versions being phased out

# Version: 1.x.x
# Status: Deprecated (EOL: 2026-06-01)
# Projects using: 0/70
# Support: Security patches only

Version Support Matrix

Version	Status	Support Level	Migration Deadline	Projects
1.9.x	Deprecated	Security only	2026-03-01	0/70
2.0.x	Stable	Full support	No deadline	65/70
2.1.x	Current	Full support	-	5/70
3.0.0	Preview	Best effort	-	0/70

Rollout Process

Phase 1: Development and Testing

Week 1-2: Component Development

Create feature branch in component repository
Develop new component or update existing
Update component tests
Update README documentation

# In gitlab_components repository
git checkout -b feat/add-helm-deployment
# Make changes
git commit -m "feat: add Helm deployment component"
git push origin feat/add-helm-deployment

Week 2: Internal Testing

Test with dedicated test projects:

test-projects/
 test-node-app/          # Tests node-pipeline component
 test-go-service/        # Tests go-pipeline component
 test-k8s-deploy/        # Tests k8s-deploy component

Phase 2: Beta Release

Week 3: Create Beta Release

# Tag beta version
git tag 3.0.0-beta.1
git push origin 3.0.0-beta.1

# CI/CD automatically publishes to catalog

Week 3-4: Early Adopter Testing

Select 3-5 representative projects:

# Early adopter projects
- agent-mesh          # Complex microservices
- agent-runtime       # Core service
- documentation-site  # Simple static site

Update their pipelines:

# agent-mesh/.gitlab-ci.yml
include:
  - component: gitlab.com/my-org/gitlab_components/k8s-deploy@3.0.0-beta.1
    inputs:
      environment: staging  # Test in staging first

Feedback Collection

Create feedback issue in component repository:

# Feedback: Version 3.0.0-beta.1

## Test Projects
- [ ] agent-mesh - @alice
- [ ] agent-runtime - @bob
- [ ] documentation-site - @carol

## Issues Found
- [ ] #123: k8s-deploy fails with custom namespaces
- [ ] #124: helm component needs timeout input

## Approval
- [ ] QA Team approval
- [ ] Security Team approval
- [ ] Platform Team approval

Phase 3: Stable Release

Week 5: Release v3.0.0

After beta validation:

# Tag stable version
git tag 3.0.0
git push origin 3.0.0

# Announce in Slack/Email:
# "Component version 3.0.0 released - tested by 5 projects"

Phase 4: Gradual Rollout

Week 6-8: Tier 1 Projects (Priority)

Roll out to critical projects first:

# Tier 1: Critical production services (10 projects)
- agent-mesh
- agent-runtime
- agent-api-gateway
- agent-scheduler
- agent-storage
... (6 more)

Week 9-11: Tier 2 Projects (Standard)

Roll out to standard projects:

# Tier 2: Standard services (40 projects)
- Various agent implementations
- Integration services
- Support tools

Week 12-14: Tier 3 Projects (Low Priority)

Roll out to remaining projects:

# Tier 3: Documentation, examples, experimental (20 projects)
- Documentation sites
- Example projects
- Proof-of-concepts

Phase 5: Deprecation

Week 15+: Deprecate Old Version

# Mark 2.0.x as deprecated
# Set migration deadline: +90 days

Communicate deprecation:

# Component Version 2.0.x Deprecation Notice

**Deprecated Version**: 2.0.x
**Replacement Version**: 3.0.0
**Migration Deadline**: 2026-09-01
**Support Status**: Security patches only

## Migration Guide
See: docs/migration-2.0-to-3.0.md

## Affected Projects
15 projects still using 2.0.x (see below)

Automation Tools

Component Version Tracker

# version-tracker.yml
# Tracks component usage across all projects

projects:
  - name: agent-mesh
    component_version: 3.0.0
    last_updated: 2026-01-08
    status: current

  - name: agent-runtime
    component_version: 2.0.5
    last_updated: 2025-12-15
    status: stable

  - name: legacy-agent
    component_version: 1.9.2
    last_updated: 2025-06-01
    status: deprecated_warning

Automated Rollout Script

// rollout-component-version.ts
import { glab } from './gitlab-api';

interface RolloutConfig {
  componentVersion: string;
  projects: string[];
  dryRun: boolean;
}

async function rolloutVersion(config: RolloutConfig) {
  for (const project of config.projects) {
    console.log(`Updating ${project} to ${config.componentVersion}`);

    // 1. Create feature branch
    const branch = `component-update-${config.componentVersion}`;
    await glab.createBranch(project, branch);

    // 2. Update .gitlab-ci.yml
    const ciConfig = await glab.getFile(project, '.gitlab-ci.yml');
    const updated = updateComponentVersion(ciConfig, config.componentVersion);

    if (config.dryRun) {
      console.log('DRY RUN - Would update:', updated);
      continue;
    }

    await glab.updateFile(project, '.gitlab-ci.yml', updated, branch);

    // 3. Create MR
    await glab.createMR(project, {
      source_branch: branch,
      target_branch: 'main',
      title: `Update component to ${config.componentVersion}`,
      description: `Automated component version update`,
      labels: ['component-update', 'automation']
    });

    console.log(` MR created for ${project}`);
  }
}

// Usage:
rolloutVersion({
  componentVersion: '3.0.0',
  projects: [
    'agent-mesh',
    'agent-runtime',
    'agent-api-gateway'
  ],
  dryRun: false  // Set true for testing
});

Version Compliance Checker

# .gitlab-ci.yml in component repository

check-version-compliance:
  stage: validate
  image: node:20
  script:
    - npm install
    - npx tsx check-compliance.ts
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"  # Run daily

  artifacts:
    reports:
      - compliance-report.json

// check-compliance.ts
import { fetchAllProjects, fetchProjectPipeline } from './gitlab-api';

interface ComplianceReport {
  compliant: number;
  deprecated: number;
  outdated: number;
  projects: ProjectStatus[];
}

async function checkCompliance(): Promise<ComplianceReport> {
  const projects = await fetchAllProjects('agent-platform/*');
  const report: ComplianceReport = {
    compliant: 0,
    deprecated: 0,
    outdated: 0,
    projects: []
  };

  for (const project of projects) {
    const ciConfig = await fetchProjectPipeline(project.id);
    const version = extractComponentVersion(ciConfig);

    const status = classifyVersion(version);
    report[status]++;

    report.projects.push({
      name: project.name,
      version,
      status,
      last_updated: project.last_activity_at
    });
  }

  // Alert if too many deprecated
  if (report.deprecated > 5) {
    await sendSlackAlert(
      ` ${report.deprecated} projects using deprecated components`
    );
  }

  return report;
}

Communication Strategy

Announcement Template

# Component Update Available: v3.0.0

## Summary
New version of gitlab_components available with Helm deployment support.

## What's New
-  Helm deployment component
-  Enhanced security scanning
-  30% faster Docker builds
-  Bug fixes for k8s-deploy

## Breaking Changes
 **k8s-deploy**: `cluster_name` renamed to `cluster`

## Migration Guide
See: https://gitlab.com/my-org/gitlab_components/-/blob/main/docs/migration-2.0-to-3.0.md

## Rollout Plan
- Week 1-2: Beta testing (5 projects)
- Week 3-5: Tier 1 projects (10 projects)
- Week 6-8: Tier 2 projects (40 projects)
- Week 9+: Remaining projects (20 projects)

## Timeline
- Beta Release: 2026-01-15
- Stable Release: 2026-02-01
- Full Rollout: 2026-03-31
- Deprecate 2.0.x: 2026-06-01

## Support
- Questions: #gitlab-components Slack channel
- Issues: https://gitlab.com/my-org/gitlab_components/issues
- Migration help: @platform-team

Slack Notifications

// notify-component-release.ts
import { WebClient } from '@slack/web-api';

const slack = new WebClient(process.env.SLACK_TOKEN);

await slack.chat.postMessage({
  channel: '#ci-cd-updates',
  blocks: [
    {
      type: 'header',
      text: {
        type: 'plain_text',
        text: ' Component v3.0.0 Released'
      }
    },
    {
      type: 'section',
      text: {
        type: 'mrkdwn',
        text: '*New features:*\n Helm deployment\n Enhanced security\n Faster builds'
      }
    },
    {
      type: 'actions',
      elements: [
        {
          type: 'button',
          text: { type: 'plain_text', text: 'View Release Notes' },
          url: 'https://gitlab.com/my-org/gitlab_components/-/releases/3.0.0'
        },
        {
          type: 'button',
          text: { type: 'plain_text', text: 'Migration Guide' },
          url: 'https://gitlab.com/my-org/gitlab_components/-/blob/main/docs/migration.md'
        }
      ]
    }
  ]
});

Handling Breaking Changes

Pre-Release Communication

4 weeks before release:

# Upcoming Breaking Change: v3.0.0

## What's Changing
The `k8s-deploy` component will rename:
- `cluster_name`  `cluster`
- `namespace_name`  `namespace`

## Why
Consistency with Kubernetes terminology

## Impact
Affects 45 projects using k8s-deploy

## Migration Required
Update your .gitlab-ci.yml:

**Before:**
```yaml
component: .../k8s-deploy@2.0.0
inputs:
  cluster_name: production
  namespace_name: app

After:

component: .../k8s-deploy@3.0.0
inputs:
  cluster: production
  namespace: app

Timeline

Announcement: 2026-01-01
Beta: 2026-01-15
Release: 2026-02-01
Migration deadline: 2026-05-01


### Compatibility Shims

Provide temporary backward compatibility:

```yaml
# templates/k8s-deploy.yml (v3.0.0)
spec:
  inputs:
    # New names (preferred)
    cluster:
      type: string
    namespace:
      type: string

    # Old names (deprecated)
    cluster_name:
      type: string
    namespace_name:
      type: string

---

.resolve-inputs:
  script:
    # Support both old and new names
    - |
      if [ -n "$[[ inputs.cluster ]]" ]; then
        CLUSTER="${{ inputs.cluster }}"
      else
        CLUSTER="${{ inputs.cluster_name }}"
        echo " cluster_name is deprecated, use cluster instead"
      fi

deploy:
  extends: .resolve-inputs
  script:
    - kubectl --context=$CLUSTER apply -f k8s/

Deprecation Warnings

Add warnings to CI/CD output:

warn-deprecated-inputs:
  stage: .pre
  script:
    - |
      if grep -q "cluster_name" .gitlab-ci.yml; then
        echo " WARNING "
        echo "You are using deprecated input: cluster_name"
        echo "Please migrate to: cluster"
        echo "Support ends: 2026-05-01"
        echo "Migration guide: https://..."
      fi
  allow_failure: true

Testing Multi-Project Changes

Test Matrix

Create test projects representing different scenarios:

# test-matrix.yml
test_projects:
  - name: test-simple
    type: single-job
    language: node
    complexity: simple

  - name: test-multi-stage
    type: multi-stage
    language: go
    complexity: medium

  - name: test-microservices
    type: complex-pipeline
    language: mixed
    complexity: high

  - name: test-monorepo
    type: monorepo
    language: node
    complexity: high

Automated Testing Pipeline

# .gitlab-ci.yml in gitlab_components

test-all-scenarios:
  stage: test
  trigger:
    strategy: depend
  parallel:
    matrix:
      - TEST_PROJECT: [test-simple, test-multi-stage, test-microservices, test-monorepo]
  script:
    - trigger-project-pipeline ${TEST_PROJECT}
  rules:
    - if: $CI_COMMIT_TAG =~ /^.*-beta.*$/  # Run on beta releases

Monitoring and Metrics

Component Usage Dashboard

Track metrics:

Adoption rate: Projects using latest version
Version distribution: Count per version
Update velocity: Time to upgrade
Failure rate: Pipeline failures after upgrade
Rollback rate: Projects reverting to old version

// metrics.ts
interface ComponentMetrics {
  total_projects: number;
  by_version: {
    [version: string]: {
      count: number;
      percentage: number;
    };
  };
  adoption_rate: {
    latest: number;
    current: number;
    deprecated: number;
  };
  update_velocity: {
    avg_days_to_upgrade: number;
    fastest: number;
    slowest: number;
  };
}

GitLab CI/CD Analytics

Use GitLab's built-in analytics:

Value Stream Analytics: Track time from component release to project adoption
DORA Metrics: Monitor deployment frequency changes
Pipeline Analytics: Track failure rates after component updates

Rollback Strategy

Quick Rollback

If critical issue found:

# Project can immediately rollback
include:
  - component: gitlab.com/org/gitlab_components/deploy@2.0.5  # Revert to stable
    inputs:
      environment: production

Component Version Yanking

For severe security issues:

# Remove version from catalog (GitLab 18.0+)
glab release delete 3.0.0 --repo my-org/gitlab_components

# Notify all users
notify-all-projects "Version 3.0.0 yanked due to security issue. Use 2.0.5."

Success Metrics

Target Metrics

Adoption rate: >90% on current version within 12 weeks
Update velocity: Average 2 weeks from release to project update
Failure rate: <5% pipeline failures after component update
Rollback rate: <2% projects rolling back
Deprecation compliance: 100% off deprecated versions before EOL

Tracking Progress

# Weekly report
week_of: 2026-01-08
version: 3.0.0
projects_updated: 12/70 (17%)
failed_updates: 1 (8%)
rollbacks: 0 (0%)
blockers:
  - Project X: Waiting for security approval
  - Project Y: Custom pipeline needs refactoring

Best Practices

Test with real projects: Don't just unit test components
Gradual rollout: Never update all 70 projects at once
Maintain old versions: Keep 2-3 versions supported
Clear communication: Over-communicate changes and timelines
Automation: Automate compliance checking and notifications
Documentation: Keep migration guides updated
Backward compatibility: Use shims for breaking changes
Monitor closely: Watch metrics during rollout
Quick rollback: Make it easy to revert if needed
Learn and adapt: Retrospective after each major rollout

Next Steps

Versioning: Deep dive into semantic versioning strategy
Governance: Approval processes and change management
Testing: Comprehensive component testing strategies
Patterns: Common multi-project patterns