Multi-Project Pipeline Integration

This document describes the multi-project pipeline architecture used across the blueflyio ecosystem for agent-based CI/CD automation.

Overview

Multi-project pipelines allow upstream repositories to trigger specialized agent jobs in downstream repositories. This enables centralized agent execution while maintaining repository autonomy.

Architecture

   Upstream Projects     
  (openstandardagents,   
   agent-buildkit, etc.) 

             trigger
            

    Platform Agents      
 (blueflyio/platform-    
       agents)           

  ts:build:agent        
  native:build:agent    
  deploy:infra:agent    
  deploy:ml:agent       

Configuration

Upstream Repository Setup

Add trigger jobs to your .gitlab-ci.yml or .gitlab/ci/agents.yml:

variables:
  AGENT_PLATFORM_PROJECT: blueflyio/platform-agents

ts:build:agent:
  stage: build
  trigger:
    project: ${AGENT_PLATFORM_PROJECT}
    strategy: mirror  # Reflects downstream status
  variables:
    AGENT_NAME: ts-prod
    AGENT_TASK: build
    BUILD_TARGET: production
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: on_success
      allow_failure: true

Key Configuration Elements

Authentication

JOB-TOKEN vs PRIVATE-TOKEN

When using CI_JOB_TOKEN for API calls, use the correct header:

# CORRECT - use JOB-TOKEN header
curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \
  "https://gitlab.com/api/v4/projects/..."

# WRONG - PRIVATE-TOKEN is for personal/project tokens only
# curl --header "PRIVATE-TOKEN: ${CI_JOB_TOKEN}"  # 401/403 error

Token Scopes

Variable Passing

Variables flow from upstream to downstream through:

1. Explicit variables in the trigger job
2. Predefined variables via interpolation
3. dotenv artifacts (Premium/Ultimate)

trigger_job:
  trigger:
    project: blueflyio/platform-agents
  variables:
    # Explicit
    AGENT_NAME: my-agent
    AGENT_TASK: build
    # Interpolated predefined variables
    SOURCE_REF: $CI_COMMIT_REF_NAME
    SOURCE_PROJECT: $CI_PROJECT_PATH

Agent Jobs in openstandardagents

The openstandardagents repository triggers these agent jobs:

Local Execution Agents

Some agents run locally (not triggered downstream):

Troubleshooting

Common Issues

Pipeline not triggered:

• Verify AGENT_PLATFORM_PROJECT: blueflyio/platform-agents
• Check upstream project has trigger access
• Verify rule conditions match

401/403 authentication errors:

• Use JOB-TOKEN header with CI_JOB_TOKEN
• Never use PRIVATE-TOKEN with CI_JOB_TOKEN

Downstream fails but upstream passes:

• Add strategy: mirror to reflect downstream status
• Check downstream pipeline logs

Debugging

View downstream pipeline status:

# Get pipeline info
glab api projects/blueflyio%2Fplatform-agents/pipelines --per-page 5

Best Practices

1. Use strategy: mirror to reflect downstream results
2. Put allow_failure at rule level, not job level
3. Use meaningful AGENT_NAME and AGENT_TASK variables
4. Add when: on_success to rules for clarity
5. Document required variables for each agent

Related Documentation

• GitLab Downstream Pipelines
• Platform Agents Wiki
• OSSA Agent Manifests

---

Last Updated: 2025-12-21