Intelligent Orchestration in Drupal — Master Plan

Date: 2026-02-10 | Status: Planning | Owner: Thomas Scola Drupal: 11.3.3 | Site: drupal_testing-php8.2 | Modules: 235 installed, 164 enabled

---

Vision

Build 100% Drupal-native intelligent orchestration by rebuilding all 20 custom modules to properly extend the contrib ecosystem (drupal/orchestration, drupal/ai, drupal/ai_agents, drupal/eca, drupal/tool, drupal/mcp). Every module stays. Every module gets better. Anti-patterns get replaced with Drupal-native patterns while preserving and enhancing the unique functionality each module provides.

Current State (after 2026-02-10 work)

• 20 custom modules with ~35,000 LOC — all have unique value
• 164 contrib modules now enabled (up from 79) — stack unlocked
• tool_ai_connector enabled — 41+ Tool plugins now visible to agents
• ECA visual editor enabled — eca_ui + eca_modeller_bpmn + event sources
• ai_logging enabled — AI call/token/cost tracking active
• mcp_tools enabled — 222 ready-made MCP tools available
• charts_chartjs + dashboards_views enabled — visualization stack ready

Architecture: Outerloop / Innerloop

EXTERNAL OUTERLOOP (Activepieces / n8n / Zapier)
    | orchestration module (REST/webhook bridge)
    v
DRUPAL OUTERLOOP: Governance
    ai_agents_ossa   -> compliance gate
    ai_agents_kagent  -> deployment topology
    eca               -> workflow state machine
    "Should this run? Where? Under what constraints?"

DRUPAL OUTERLOOP: Orchestration
    mcp_registry       -> service discovery + health
    api_normalization   -> API gateway + routing
    orchestration       -> external platform bridge
    "How do services connect? What's healthy?"

INNERLOOP: Execution
    ai_agents    -> agent lifecycle (task->tools->solve)
    tool         -> typed tool execution
    code_executor -> sandboxed compute
    ai           -> provider-agnostic LLM calls
    "Run the task. Return the result. Nothing else."

PROTOCOL: Transport
    mcp / mcp_client      -> AI-native clients
    orchestration          -> automation platforms
    http_client_manager    -> external HTTP APIs
    jsonrpc                -> JSON-RPC 2.0
    "Move bytes. Authenticate. Serialize."

Separation of Duties

Key Insight: drupal/orchestration

Dries-backed (v1.0.0, 47 sites). Already enabled with all 5 submodules. Auto-exposes:

• ALL #[Tool] plugins to external platforms (orchestration_tool)
• ALL AI agents to external platforms (orchestration_ai_agents)
• ALL FunctionCall plugins (orchestration_ai_function)
• ALL ECA workflows (orchestration_eca)

Zero integration code needed. Ship a Tool plugin, it's automatically callable from Activepieces/n8n.

---

Workstream 1: Enable Critical Contrib — DONE

25+ contrib modules enabled on 2026-02-10. Stack is live.

---

Workstream 2: Rebuild Custom Modules to Extend Contrib (Weeks 2-8)

No modules get removed. Every module gets rebuilt to properly use Drupal plugin patterns and extend the contrib stack. The unique functionality stays — the implementation patterns change.

Anti-Patterns to Convert (inside existing modules)

Per-Module Rebuild Plans

All 20 modules — what changes, what stays:

ai_agents_ossa (KEEP + enhance):

• Stays as-is — core OSSA IP, well-architected
• Minor: replace GuzzleHttpClient wrapper with http_client_manager
• Enhanced: merge cursor tools/generators into this module

ai_agents_kagent (REBUILD internals):

• Keep: K8s deployment logic, FunctionCall plugins, Messenger handlers
• Rebuild: 1 EventSubscriber -> ECA, 6 blocks -> Views+Charts, 5 data entities -> ECK

ai_agents_orchestra (REBUILD internals):

• Keep: agent mesh, consensus orchestration, ECA plugins, MCP bridge
• Rebuild: workflow engine -> ECA+FlowDrop, vector memory -> ai_context, LLM gateway -> ai ProviderProxy

mcp_registry (REBUILD internals — largest module):

• Keep: registry management, fleet orchestration, bulk ops, security layer
• Rebuild: 5 EventSubscribers -> ECA, 12 blocks -> Views+Charts, 8 entities -> ECK, 3 annotations -> attributes, consolidate 20+ controllers

api_normalization (REBUILD internals):

• Keep: AI API transforms (Anthropic->OpenAI, Ollama->OpenAI), circuit breaker, multi-provider routing
• Rebuild: gateway -> extend http_client_manager, schema -> extend openapi, 8 entities -> ECK

gov_compliance (REBUILD internals):

• Keep: all compliance services, Cedar engine, ECA plugins, 10 Tool plugins
• Rebuild: 3 blocks -> Views+Charts, 5 entities -> ECK, consolidate 18 controllers

code_executor (REBUILD internals):

• Keep: Docker execution engine, ECA plugins, Tool plugins
• Rebuild: QueueWorker -> Messenger, controller -> tool/mcp endpoint

ai_agents_crewai (REBUILD to extend ai_agents):

• Keep: crew orchestration patterns, multi-agent coordination logic
• Rebuild: properly extend AiAgentWrapper sub-agent system, use ECA for workflow routing

ai_agents_communication (REBUILD to extend mcp):

• Keep: A2A protocol logic, agent registry, message passing
• Rebuild: use MCP protocol as transport, REST controllers -> Tool plugins

ai_agents_client (REBUILD to extend mcp_client):

• Keep: OSSA client discovery, protocol adapters
• Rebuild: extend mcp_client for transport, QueueWorker -> Messenger

ai_provider_langchain (REBUILD to extend ai + eca):

• Keep: chain/workflow concepts, RAG pipeline, cost tracking
• Rebuild: workflow engine -> ECA+FlowDrop, entities -> ECK, cost tracking -> ai_logging, delete 14 dead backup files

ai_agents_claude (REBUILD to extend ai_provider_anthropic):

• Keep: Claude-specific tool logic
• Rebuild: properly extend ai_provider_anthropic, generic tools as #[Tool] plugins

ai_agents_huggingface (REBUILD to extend ai_provider_huggingface):

• Keep: model management, pipeline execution
• Rebuild: entity -> ECK, MCP server -> #[Mcp] plugin

ai_agents_cursor (MERGE into ai_agents_ossa):

• Keep: 10 MCP tools, 5 generators, AiAgent plugin, ECA events
• Rebuild: tools become standalone #[Tool] plugins, AiAgent moves to ai_agents_ossa

ai_provider_apple (REBUILD internals):

• Keep: 5 AIProvider plugins, ECA events
• Rebuild: 3 REST resources -> Tool plugins, 2 entities -> ECK, delete 8 dead files, consolidate controllers

alternative_services (REBUILD internals):

• Keep: DDEV integration, service discovery, addon management
• Rebuild: 4 entities -> ECK, 3 QueueWorkers -> advancedqueue, 2 blocks -> Views+Charts

layout_system_converter (REBUILD internals):

• Keep: conversion engines (Paragraphs->SDC, Layout Builder->XB, Site Studio->XB)
• Rebuild: 6 annotations -> PHP 8 attributes, consolidate controllers, Views for browsing

external_migration (KEEP as-is):

• Well-integrated with contrib, uses proper migrate patterns
• No anti-patterns found

dita_ccms (REBUILD minor):

• Keep: DITA XML engine, ECA plugins, Feeds parser
• Minor: REST resource -> Tool plugin, QueueWorker -> Messenger

drupal_patch_framework (KEEP as-is):

• Clean architecture, proper separation of concerns
• No anti-patterns found

---

Workstream 3: Immutable Audit Trail (Weeks 6-10)

Build trace_id propagation across the full outerloop/innerloop stack.

Design

ECK Entity: trace_entry

• Fields: trace_id (UUID), span_id, parent_span_id, layer (governance/orchestration/execution/protocol), module, event_type, timestamp, duration_ms, metadata (JSON), status (success/failure)
• Access control: insert-only (no edit, no delete form)
• Views integration for dashboards

Trace Context Service:

• Generates trace_id at outerloop entry points (orchestration webhook, ECA event, MCP call)
• Propagates via request attributes or DI
• Each layer writes trace entries with its span_id

Trace Points (9 steps):

1. External trigger arrives -> orchestration logs source/platform
2. OSSA compliance check -> ai_agents_ossa logs pass/fail
3. ECA workflow fires -> eca logs event/conditions/actions
4. Agent selects tools -> ai_agents logs triage/selection
5. Tool executes -> tool logs ExecutableResult
6. LLM called -> ai_logging logs tokens/cost/provider
7. MCP server called -> mcp_registry logs tool_id/params
8. Result returns to agent -> ai_agents logs solve/answer
9. Result returns to external -> orchestration logs response

Dashboard:

• Views + Charts for trace visualization
• Filter by trace_id, time range, layer, status
• Aggregate metrics: calls/minute, avg latency, error rate, token costs

---

All 20 Custom Modules — Final Architecture

Architecture Principles (enforced going forward)

---

Timeline

---

Success Metrics

• All 20 custom modules rebuilt to extend contrib properly
• EventSubscribers: 6 -> 0 (all ECA)
• Custom dashboard blocks: 21+ -> 0 (all Views + Charts)
• Custom entities: 30+ data containers -> ECK (keep complex entities as PHP)
• Custom REST resources: 4 -> 0 (all Tool/MCP)
• Dead code files: 22 -> 0
• Enabled contrib modules: 164 (already achieved)
• All 41+ Tool plugins visible to agents via tool_ai_connector (already achieved)
• Full trace_id propagation across outerloop/innerloop
• Visual ECA editor accessible for no-code workflow building (already achieved)
• Every module publishable to drupal.org

---

Generated 2026-02-10 | Updated 2026-02-10 — rebuild approach, no modules removed