BAT decision matrix

BAT Decision Matrix: Best Available Technology Selection

Version: 1.0.0 Status: Standard Epic Alignment: #55 (GitLab Ultimate Showcase), #46 (Infrastructure) Last Updated: 2026-01-12

Executive Summary

BAT (Best Available Technology) is a systematic framework for selecting optimal technologies across the agent platform. This document defines the decision criteria, scoring methodology, and matrices for all technology categories.

Core Principle: Select the best available technology for each component based on requirements, cost, performance, feasibility, and vendor independence.

BAT Selection Criteria

Primary Criteria (Weighted)

Criterion	Weight	Description
Performance	25%	Meets or exceeds requirements (latency, throughput, accuracy)
Cost	20%	Total Cost of Ownership (TCO) - licensing, compute, maintenance
Reliability	20%	Uptime, error rates, proven production track record
Maintainability	15%	Standards-based, documentation quality, community support
Scalability	10%	Ability to grow with platform demands
Vendor Independence	10%	Avoid lock-in, migration path, open standards

Scoring Scale

Score	Meaning	Criteria
5	Excellent	Best in class, exceeds all requirements
4	Good	Meets all requirements, minor limitations
3	Acceptable	Meets most requirements, known limitations
2	Marginal	Meets minimum requirements, significant limitations
1	Poor	Does not meet requirements, major concerns

BAT Score Calculation

BAT Score = (Performance  0.25) + (Cost  0.20) + (Reliability  0.20)
          + (Maintainability  0.15) + (Scalability  0.10) + (Vendor Independence  0.10)

Thresholds:

> 4.0: Strong recommendation
3.5 - 4.0: Acceptable, proceed with caution
< 3.5: Not recommended, seek alternatives

Technology Category Matrices

1. LLM Provider Selection

Context: Selecting LLM providers for agent reasoning, code review, and task execution.

Provider	Model	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
Anthropic	Claude 3.5 Sonnet	5	4	5	4	4	3	4.30
Anthropic	Claude 3 Opus	5	2	5	4	4	3	3.90
OpenAI	GPT-4o	4	3	4	4	5	3	3.80
Google	Gemini 2.0 Flash	3	5	4	3	5	3	3.75
Google	Gemini 2.0 Pro	4	3	4	3	5	3	3.65
Meta	Llama 3.3 70B	4	5	3	4	3	5	3.95
Mistral	Mixtral 8x22B	3	5	3	4	3	5	3.70

BAT Decision:

Primary (Reasoning): Claude 3.5 Sonnet (Score: 4.30) - Best reasoning capability
Secondary (Cost-Sensitive): Gemini 2.0 Flash (Score: 3.75) - High-volume, low-cost tasks
Multimodal: GPT-4o (Score: 3.80) - Image/vision tasks
Self-Hosted: Llama 3.3 70B (Score: 3.95) - Vendor independence, on-premises

LLM Selection by Agent Type

Agent Type	Recommended LLM	Rationale
Orchestrators (@default-orchestrator, @release-coordinator)	Claude 3 Opus	Critical decisions require highest capability
Code Review (@mr-reviewer, @code-reviewer)	Claude 3.5 Sonnet	Strong reasoning for code analysis
Pipeline Analysis (@ci-fixer-worker, @pipeline-remediation)	Claude 3.5 Sonnet	Complex log analysis
High-Volume (@cost-intelligence-monitor, @task-dispatcher)	Gemini 2.0 Flash	Cost efficiency for frequent operations
Multimodal (@documentation-aggregator)	GPT-4o	Image/diagram processing
No LLM (@vuln-scanner, @drupal-standards-checker)	N/A	Rule-based, no LLM needed

2. Agent Framework Selection

Context: Selecting frameworks for agent definition, orchestration, and execution.

Framework	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
OSSA v0.3.2	4	5	4	5	4	5	4.40
kAgent CRD	4	5	4	4	5	4	4.25
LangGraph	4	3	4	4	4	2	3.60
CrewAI	3	4	3	3	3	3	3.20
AutoGen	3	4	3	3	3	3	3.20
Custom	4	5	3	2	3	5	3.50

BAT Decision:

Primary: OSSA v0.3.2 (Score: 4.40) - Open standard, comprehensive, vendor-neutral
Kubernetes Deployment: kAgent CRD (Score: 4.25) - Native K8s, auto-registration
Alternatives Considered: LangGraph (vendor lock-in), CrewAI (less mature)

3. Infrastructure Selection

Context: Selecting compute and storage infrastructure for agent execution.

Compute Infrastructure

Infrastructure	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
Vast.ai RTX 4090	5	5	4	3	4	4	4.25
AWS p4d.24xlarge	5	1	5	4	5	2	3.45
GCP A100	5	1	5	4	5	2	3.45
Local GPU	4	5	2	3	1	5	3.15
RunPod	4	4	3	3	4	4	3.65

BAT Decision:

GPU Compute: Vast.ai RTX 4090 (Score: 4.25) - Cost-effective, on-demand
Reasoning: 10x cheaper than AWS/GCP, comparable performance

Storage Infrastructure

Infrastructure	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
Synology NAS	4	5	4	4	3	5	4.15
AWS S3	4	2	5	4	5	2	3.55
MinIO (Self-Hosted)	4	5	4	4	4	5	4.25
Cloudflare R2	4	4	5	3	5	3	3.95

BAT Decision:

Primary Storage: Synology NAS (Score: 4.15) - Always-on, cost-effective
Object Storage: MinIO on NAS (Score: 4.25) - S3-compatible, self-hosted
Edge Storage: Cloudflare R2 (Score: 3.95) - Edge caching, egress-free

4. Database Selection

Context: Selecting databases for agent state, vector storage, and caching.

Database	Use Case	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
PostgreSQL	Agent State	5	5	5	5	4	5	4.85
Qdrant	Vector Search	5	5	4	4	4	5	4.50
Redis	Cache/Queue	5	5	4	5	4	5	4.65
MongoDB	Documents	4	4	4	4	4	4	4.00
Pinecone	Vector Search	5	2	5	3	5	2	3.60
Weaviate	Vector Search	4	4	4	4	4	5	4.15

BAT Decision:

Relational: PostgreSQL (Score: 4.85) - Agent state, metadata
Vector: Qdrant (Score: 4.50) - Embeddings, semantic search
Cache: Redis (Score: 4.65) - Session cache, message queue
Alternative Vector: Weaviate (Score: 4.15) - If Qdrant unavailable

5. Observability Selection

Context: Selecting observability stack for agent monitoring, tracing, and alerting.

Tool	Purpose	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
GitLab Observability	APM/Tracing	4	5	5	5	4	4	4.50
OpenTelemetry	Instrumentation	5	5	5	5	5	5	5.00
Prometheus	Metrics	5	5	5	4	4	5	4.65
Grafana	Dashboards	5	5	5	5	4	5	4.85
DataDog	Full Stack	5	1	5	4	5	2	3.50
New Relic	Full Stack	5	1	5	4	5	2	3.50

BAT Decision:

Primary: GitLab Observability (Score: 4.50) - Native integration with GitLab Ultimate
Instrumentation: OpenTelemetry (Score: 5.00) - Standard, vendor-neutral
Metrics: Prometheus + Grafana (Score: 4.75 avg) - Self-hosted, comprehensive
Implementation: agent-tracer (Epic #52) using OpenTelemetry GitLab Observability

6. Networking Selection

Context: Selecting networking infrastructure for agent communication and public access.

Technology	Purpose	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
Tailscale	Mesh VPN	5	5	5	5	4	4	4.70
Cloudflare Tunnel	Public Access	5	5	5	4	5	3	4.55
WireGuard	VPN	5	5	4	3	3	5	4.20
ngrok	Tunneling	4	3	4	4	3	4	3.70
ZeroTier	Mesh VPN	4	5	4	4	4	4	4.20

BAT Decision:

Private Mesh: Tailscale (Score: 4.70) - Zero-config mesh, secure
Public Ingress: Cloudflare Tunnel (Score: 4.55) - DDoS protection, free tier
Configuration:
- Private: *.tailcf98b3.ts.net
- Public: *.blueflyagents.com (via Cloudflare)

7. CI/CD Selection

Context: Selecting CI/CD platform for agent deployment and pipeline management.

Platform	Performance	Cost	Reliability	Maintainability	Scalability	Vendor Ind.	BAT Score
GitLab CI	5	5	5	5	5	4	4.85
GitHub Actions	4	4	5	4	5	3	4.15
Jenkins	4	5	4	3	4	5	4.05
Tekton	4	5	4	3	5	5	4.20
ArgoCD	4	5	5	4	5	5	4.55

BAT Decision:

Primary: GitLab CI (Score: 4.85) - Native to GitLab, components support
GitOps: ArgoCD (Score: 4.55) - Kubernetes deployments, declarative
Implementation: GitLab CI Components (gitlab_components project)

Agent-Specific BAT Decisions

Summary Table

Agent	LLM (BAT)	Framework (BAT)	Infrastructure (BAT)	Rationale
@default-orchestrator	Claude 3 Opus	OSSA + kAgent	Vast.ai (GPU)	Critical coordination decisions
@release-coordinator	Claude 3 Opus	OSSA + kAgent	Vast.ai (GPU)	Critical release decisions
@mr-reviewer	Claude 3.5 Sonnet	OSSA + kAgent	Vast.ai (GPU)	Strong code reasoning
@ci-fixer-worker	Claude 3.5 Sonnet	OSSA + kAgent	Vast.ai (GPU)	Complex log analysis
@issue-worker	Claude 3.5 Sonnet	OSSA + kAgent	Vast.ai (GPU)	Implementation planning
@code-reviewer	Claude 3.5 Sonnet	OSSA + kAgent	Vast.ai (GPU)	Multi-language SAST
@pipeline-remediation	Claude 3.5 Sonnet	OSSA + kAgent	Vast.ai (GPU)	Failure analysis
@vuln-scanner	None (rule-based)	OSSA	NAS (Docker)	Security tools only
@drupal-standards-checker	None (rule-based)	OSSA	NAS (Docker)	PHPCS/PHPStan only
@ossa-validator	None (rule-based)	OSSA	NAS (Docker)	Schema validation
@cost-intelligence-monitor	Gemini 2.0 Flash	OSSA	NAS (Docker)	High-volume analytics
@task-dispatcher	Gemini 2.0 Flash	OSSA	NAS (Docker)	Frequent routing
@documentation-aggregator	GPT-4o	OSSA	NAS (Docker)	Multimodal docs
@module-generator	Claude 3.5 Sonnet	OSSA	NAS (Docker)	Code generation
@recipe-publisher	None (automation)	OSSA	NAS (Docker)	Packagist automation
@kagent-worker	Claude 3.5 Sonnet	OSSA + kAgent	Vast.ai (GPU)	K8s orchestration
@cluster-operator	None (automation)	OSSA	NAS (Docker)	Infrastructure ops
@mcp-builder-worker	Claude 3.5 Sonnet	OSSA	NAS (Docker)	MCP server creation

BAT Decision Process

Step 1: Define Requirements

requirements:
  functional:
    - description: "What must the technology do?"
    - constraints: "Any hard requirements?"
  non_functional:
    - performance: "Latency, throughput targets"
    - availability: "Uptime requirements"
    - cost: "Budget constraints"

Step 2: Identify Candidates

Research available technologies
Filter by functional requirements
Create candidate shortlist (3-5 options)

Step 3: Score Candidates

For each candidate:

Score each criterion (1-5)
Apply weights
Calculate BAT Score
Document rationale

Step 4: Document Decision

decision:
  technology: "Selected technology"
  score: "BAT Score"
  rationale: "Why this was selected"
  alternatives_considered:
    - name: "Alternative 1"
      score: "Score"
      rejection_reason: "Why not selected"
  review_date: "When to re-evaluate"

Step 5: Implement & Monitor

Implement selected technology
Track MOE (Measure of Effectiveness) metrics
Review decision quarterly
Update BAT matrix if context changes

Integration with Projects

Project Mapping

BAT Category	Primary Project	Epic
LLM Routing	agent-router	#49
Agent Framework	platform-agents	#47
Vector Storage	agent-brain	#51
Observability	agent-tracer	#52
Communication	agent-mesh	#48
CLI Tools	agent-buildkit	#50
Infrastructure	NAS/Vast.ai	#46

BAT Review Cadence

Quarterly: Full BAT matrix review
Monthly: MOE tracking against BAT decisions
Ad-hoc: When new technology emerges or requirements change

MOE Metrics - Effectiveness tracking
Agent Roster - Agent specifications
MASTER-PLAN - Architecture overview
Infrastructure Audit - System improvements
GitLab Workspace Agents - Agent registration

Changelog

Version	Date	Changes
1.0.0	2026-01-12	Initial BAT Decision Matrix

Owner: Platform Architecture Review Cycle: Quarterly Next Review: 2026-04-12

BAT decision matrix

BAT Decision Matrix: Best Available Technology Selection

Executive Summary

BAT Selection Criteria

Primary Criteria (Weighted)

Scoring Scale

BAT Score Calculation

Technology Category Matrices

1. LLM Provider Selection

LLM Selection by Agent Type

2. Agent Framework Selection

3. Infrastructure Selection

Compute Infrastructure

Storage Infrastructure

4. Database Selection

5. Observability Selection

6. Networking Selection

7. CI/CD Selection

Agent-Specific BAT Decisions

Summary Table

BAT Decision Process

Step 1: Define Requirements

Step 2: Identify Candidates

Step 3: Score Candidates

Step 4: Document Decision

Step 5: Implement & Monitor

Integration with Projects

Project Mapping

BAT Review Cadence

Related Documents

Changelog