Skip to main content

agents webhook 502 troubleshooting

agents.blueflyagents.com 502 - GitLab webhook

Canonical config: All info for Oracle, tunnel, and webhook lives in /Volumes/AgentPlatform/config. Use it first.

Cause

  • GitLab group webhook calls https://agents.blueflyagents.com (or path on that host).
  • Cloudflare tunnel (config/cloudflare/config.yml) routes agents.blueflyagents.com -> http://localhost:3001 on Oracle.
  • Nothing is listening on port 3001 on Oracle, so origin returns bad gateway -> Cloudflare returns HTTP 502 to GitLab.

Intended service on Oracle:3001

  • Duo webhook (gitlab-ossa-agent): deploy/duo-webhook/k8s/ — API deployment must use hostPort: 3001 so cloudflared (systemd on Oracle) can reach localhost:3001. Deploy via GitLab CI deploy:oracle or on Oracle: kubectl apply -f deploy/duo-webhook/k8s/ (namespace agent-platform; secrets gitlab-duo-webhook-secret + gitlab-registry-creds required). Oracle only; no OrbStack.

Config references (all in /Volumes/AgentPlatform/config)

  • cloudflare/config.yml: hostname agents.blueflyagents.com, service http://localhost:3001
  • cf_tunnel_subdomains.tsv: agents.blueflyagents.com -> http://localhost:3001
  • config.json: infrastructure.cloudflare_tunnel.routes id 10 -> agents.blueflyagents.com, port 3001, service "agent-platform", project "platform-agents", purpose "Agent Management Registry"
  • workspace.json: infrastructure.services.ports.gitlab_webhook = 3001
  • coordination-state.json: cloudflareRoutes.removeFromCF includes "agents.blueflyagents.com (no code)"
  • agents.yaml: agents.blueflyagents.com -> blueflynas.tailcf98b3.ts.net:3001 (legacy NAS; tunnel now points to Oracle localhost:3001)

Fix options

  1. Run a webhook stub on Oracle port 3001 (stops 502; hook "succeeds" with 200)

    • Stub lives in workspace: ~/Sites/blueflyio/gitlab-webhook-stub/server.js
    • On Oracle: copy server.js, then run: node server.js (or LOG_WEBHOOKS=1 node server.js)
    • Keep running via systemd, Docker, or screen/tmux.
    • Tunnel and GitLab URL stay unchanged.

  2. Remove agents.blueflyagents.com from the tunnel (no more 502 from that host; hook will fail with another error until repointed)

    • In Cloudflare Zero Trust dashboard: edit tunnel f6da7bdf-d0f8-4796-a804-afb7984bbe11, delete the Public Hostname for agents.blueflyagents.com.
    • Then in GitLab: Group -> Settings -> Webhooks -> edit hook 71485910 -> set URL to a working endpoint (e.g. workflow.bluefly.internal if you have a handler, or another service that accepts GitLab webhooks).

Status

  • coordination-state.removeFromCF marks "agents.blueflyagents.com (no code)" - route was flagged for removal but is still present in cloudflare/config.yml and the live tunnel. Either deploy something on 3001 or remove the route and repoint the webhook.