2312 lines
58 KiB
Markdown
2312 lines
58 KiB
Markdown
# Autonomous CI Agents — Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Deploy an autonomous agent system that detects CI failures and @agent mentions, then runs a 5-phase fix loop (Diagnose → Plan → Implement → Verify → Deliver) using OpenHands with LLM routing through LiteLLM.
|
|
|
|
**Architecture:** Rust Axum dispatcher receives GitLab webhooks, creates/labels issues, and invokes OpenHands headless with phase-specific microagents. LiteLLM proxy routes to Opus 4.6 (diagnose/plan), Sonnet 4.6 (implement), and Scaleway devstral-2-123b (verify/deliver). GitLab issues are the durable bus — labels track state, comments carry handoff context.
|
|
|
|
**Tech Stack:** Rust (Axum, reqwest, tokio), LiteLLM, OpenHands, GitLab CE API, Kubernetes (Kapsule)
|
|
|
|
**Design Doc:** `docs/plans/2026-02-27-autonomous-agents-design.md`
|
|
|
|
---
|
|
|
|
## Phase 0: LiteLLM Proxy
|
|
|
|
Deploy LiteLLM on Kapsule ci pool, verify all 3 model routes work.
|
|
|
|
---
|
|
|
|
### Task 1: Create K8s namespace and secrets
|
|
|
|
**Files:**
|
|
- Create: `infra/k8s/agents/namespace.yaml`
|
|
- Create: `infra/k8s/agents/secrets.yaml`
|
|
|
|
**Step 1: Write namespace manifest**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/namespace.yaml
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: foxhunt-agents
|
|
labels:
|
|
app.kubernetes.io/part-of: foxhunt
|
|
app.kubernetes.io/component: agents
|
|
```
|
|
|
|
**Step 2: Write secrets manifest (template — real values via `kubectl create secret`)**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/secrets.yaml
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: agent-credentials
|
|
namespace: foxhunt-agents
|
|
type: Opaque
|
|
stringData:
|
|
ANTHROPIC_API_KEY: "REPLACE_VIA_KUBECTL"
|
|
SCW_GENERATIVE_API_KEY: "REPLACE_VIA_KUBECTL"
|
|
GITLAB_TOKEN: "REPLACE_VIA_KUBECTL"
|
|
```
|
|
|
|
**Step 3: Apply namespace**
|
|
|
|
Run: `kubectl apply -f infra/k8s/agents/namespace.yaml`
|
|
Expected: `namespace/foxhunt-agents created`
|
|
|
|
**Step 4: Create real secret (not from template)**
|
|
|
|
Run: `kubectl create secret generic agent-credentials -n foxhunt-agents --from-literal=ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY" --from-literal=SCW_GENERATIVE_API_KEY="$SCW_GENERATIVE_API_KEY" --from-literal=GITLAB_TOKEN="$GITLAB_TOKEN" --dry-run=client -o yaml | kubectl apply -f -`
|
|
Expected: `secret/agent-credentials created`
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add -f infra/k8s/agents/namespace.yaml infra/k8s/agents/secrets.yaml
|
|
git commit -m "infra(agents): add foxhunt-agents namespace and secrets template"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Deploy LiteLLM proxy
|
|
|
|
**Files:**
|
|
- Create: `infra/k8s/agents/litellm-config.yaml`
|
|
- Create: `infra/k8s/agents/litellm.yaml`
|
|
|
|
**Step 1: Write LiteLLM config**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/litellm-config.yaml
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: litellm-config
|
|
namespace: foxhunt-agents
|
|
data:
|
|
config.yaml: |
|
|
model_list:
|
|
- model_name: "opus"
|
|
litellm_params:
|
|
model: "claude-opus-4-6"
|
|
api_key: "os.environ/ANTHROPIC_API_KEY"
|
|
- model_name: "sonnet"
|
|
litellm_params:
|
|
model: "claude-sonnet-4-6"
|
|
api_key: "os.environ/ANTHROPIC_API_KEY"
|
|
- model_name: "devstral"
|
|
litellm_params:
|
|
model: "openai/devstral-2-123b-instruct-2512"
|
|
api_base: "https://api.scaleway.ai/v1"
|
|
api_key: "os.environ/SCW_GENERATIVE_API_KEY"
|
|
|
|
litellm_settings:
|
|
max_budget: 50.0
|
|
budget_duration: "1d"
|
|
```
|
|
|
|
**Step 2: Write LiteLLM deployment + service**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/litellm.yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: litellm
|
|
namespace: foxhunt-agents
|
|
labels:
|
|
app: litellm
|
|
spec:
|
|
replicas: 1
|
|
selector:
|
|
matchLabels:
|
|
app: litellm
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: litellm
|
|
spec:
|
|
nodeSelector:
|
|
pool: ci
|
|
containers:
|
|
- name: litellm
|
|
image: ghcr.io/berriai/litellm:main-latest
|
|
args: ["--config", "/etc/litellm/config.yaml"]
|
|
ports:
|
|
- containerPort: 4000
|
|
envFrom:
|
|
- secretRef:
|
|
name: agent-credentials
|
|
volumeMounts:
|
|
- name: config
|
|
mountPath: /etc/litellm
|
|
resources:
|
|
requests:
|
|
cpu: "100m"
|
|
memory: "256Mi"
|
|
limits:
|
|
cpu: "500m"
|
|
memory: "512Mi"
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 4000
|
|
initialDelaySeconds: 10
|
|
periodSeconds: 10
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 4000
|
|
initialDelaySeconds: 15
|
|
periodSeconds: 20
|
|
volumes:
|
|
- name: config
|
|
configMap:
|
|
name: litellm-config
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: litellm
|
|
namespace: foxhunt-agents
|
|
spec:
|
|
selector:
|
|
app: litellm
|
|
ports:
|
|
- port: 4000
|
|
targetPort: 4000
|
|
```
|
|
|
|
**Step 3: Apply**
|
|
|
|
Run: `kubectl apply -f infra/k8s/agents/litellm-config.yaml -f infra/k8s/agents/litellm.yaml`
|
|
Expected: `configmap/litellm-config created`, `deployment.apps/litellm created`, `service/litellm created`
|
|
|
|
**Step 4: Wait for ready**
|
|
|
|
Run: `kubectl rollout status deployment/litellm -n foxhunt-agents --timeout=120s`
|
|
Expected: `deployment "litellm" successfully rolled out`
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add -f infra/k8s/agents/litellm-config.yaml infra/k8s/agents/litellm.yaml
|
|
git commit -m "infra(agents): deploy LiteLLM proxy with 3-model routing"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Smoke-test LiteLLM model routing
|
|
|
|
**Step 1: Port-forward LiteLLM**
|
|
|
|
Run: `kubectl port-forward -n foxhunt-agents svc/litellm 4000:4000 &`
|
|
|
|
**Step 2: Test Opus route**
|
|
|
|
Run:
|
|
```bash
|
|
curl -s http://localhost:4000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"opus","messages":[{"role":"user","content":"Say OK"}],"max_tokens":5}' \
|
|
| jq '.choices[0].message.content'
|
|
```
|
|
Expected: A short response (confirms Anthropic API key works)
|
|
|
|
**Step 3: Test Sonnet route**
|
|
|
|
Run:
|
|
```bash
|
|
curl -s http://localhost:4000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"sonnet","messages":[{"role":"user","content":"Say OK"}],"max_tokens":5}' \
|
|
| jq '.choices[0].message.content'
|
|
```
|
|
Expected: A short response
|
|
|
|
**Step 4: Test Devstral route**
|
|
|
|
Run:
|
|
```bash
|
|
curl -s http://localhost:4000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"devstral","messages":[{"role":"user","content":"Say OK"}],"max_tokens":5}' \
|
|
| jq '.choices[0].message.content'
|
|
```
|
|
Expected: A short response (confirms SCW Generative API key works)
|
|
|
|
**Step 5: Kill port-forward**
|
|
|
|
Run: `kill %1`
|
|
|
|
No commit needed — this is a manual verification step.
|
|
|
|
---
|
|
|
|
## Phase 1: Rust Dispatcher — Webhook Receiver + Issue Creator
|
|
|
|
Scaffold the Axum service that receives GitLab webhooks and creates issues.
|
|
|
|
---
|
|
|
|
### Task 4: Scaffold dispatcher crate
|
|
|
|
**Files:**
|
|
- Create: `services/agent_dispatcher/Cargo.toml`
|
|
- Create: `services/agent_dispatcher/src/main.rs`
|
|
- Modify: `Cargo.toml` (root — add workspace member)
|
|
|
|
**Step 1: Create Cargo.toml**
|
|
|
|
```toml
|
|
# services/agent_dispatcher/Cargo.toml
|
|
[package]
|
|
name = "agent_dispatcher"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
rust-version.workspace = true
|
|
authors.workspace = true
|
|
license.workspace = true
|
|
|
|
[[bin]]
|
|
name = "agent-dispatcher"
|
|
path = "src/main.rs"
|
|
|
|
[dependencies]
|
|
axum.workspace = true
|
|
tokio.workspace = true
|
|
serde = { workspace = true, features = ["derive"] }
|
|
serde_json.workspace = true
|
|
reqwest = { workspace = true, features = ["json"] }
|
|
tracing.workspace = true
|
|
tracing-subscriber.workspace = true
|
|
tower-http = { version = "0.6", features = ["cors", "trace"] }
|
|
hmac = "0.12"
|
|
sha2 = "0.10"
|
|
hex = "0.4"
|
|
```
|
|
|
|
**Step 2: Add to workspace members in root Cargo.toml**
|
|
|
|
Add `"services/agent_dispatcher"` to the `members` list.
|
|
|
|
**Step 3: Write minimal main.rs with health endpoint**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/main.rs
|
|
use axum::{Router, routing::get};
|
|
use tracing_subscriber::EnvFilter;
|
|
|
|
#[tokio::main]
|
|
async fn main() {
|
|
tracing_subscriber::fmt()
|
|
.with_env_filter(EnvFilter::from_default_env())
|
|
.init();
|
|
|
|
let app = Router::new()
|
|
.route("/health", get(|| async { "ok" }));
|
|
|
|
let listener = tokio::net::TcpListener::bind("0.0.0.0:8080")
|
|
.await
|
|
.expect("failed to bind port 8080");
|
|
|
|
tracing::info!("dispatcher listening on :8080");
|
|
axum::serve(listener, app).await.expect("server error");
|
|
}
|
|
```
|
|
|
|
**Step 4: Verify it compiles**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo check -p agent_dispatcher`
|
|
Expected: zero errors
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/Cargo.toml services/agent_dispatcher/src/main.rs Cargo.toml
|
|
git commit -m "feat(agent-dispatcher): scaffold Axum service with health endpoint"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: GitLab webhook types and signature verification
|
|
|
|
**Files:**
|
|
- Create: `services/agent_dispatcher/src/gitlab.rs`
|
|
- Create: `services/agent_dispatcher/src/webhook.rs`
|
|
|
|
**Step 1: Write the failing test (webhook signature verification)**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/webhook.rs
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_verify_valid_signature() {
|
|
let secret = "test-secret";
|
|
let body = b"hello world";
|
|
// Pre-computed HMAC-SHA256 of "hello world" with key "test-secret"
|
|
let sig = compute_hmac(secret.as_bytes(), body);
|
|
assert!(verify_signature(secret, &sig, body));
|
|
}
|
|
|
|
#[test]
|
|
fn test_reject_invalid_signature() {
|
|
assert!(!verify_signature("secret", "bad-sig", b"body"));
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- webhook`
|
|
Expected: FAIL (functions not defined)
|
|
|
|
**Step 3: Implement webhook signature verification and GitLab types**
|
|
|
|
`webhook.rs` — HMAC-SHA256 verification for `X-Gitlab-Token` header:
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/webhook.rs
|
|
use hmac::{Hmac, Mac};
|
|
use sha2::Sha256;
|
|
|
|
type HmacSha256 = Hmac<Sha256>;
|
|
|
|
pub fn compute_hmac(key: &[u8], body: &[u8]) -> String {
|
|
let mut mac = HmacSha256::new_from_slice(key).expect("HMAC key");
|
|
mac.update(body);
|
|
hex::encode(mac.finalize().into_bytes())
|
|
}
|
|
|
|
pub fn verify_signature(secret: &str, signature: &str, body: &[u8]) -> bool {
|
|
let expected = compute_hmac(secret.as_bytes(), body);
|
|
// Constant-time comparison
|
|
expected.len() == signature.len()
|
|
&& expected
|
|
.bytes()
|
|
.zip(signature.bytes())
|
|
.fold(0u8, |acc, (a, b)| acc | (a ^ b))
|
|
== 0
|
|
}
|
|
```
|
|
|
|
`gitlab.rs` — webhook payload types:
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/gitlab.rs
|
|
use serde::Deserialize;
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct PipelineWebhook {
|
|
pub object_kind: String,
|
|
pub object_attributes: PipelineAttributes,
|
|
pub project: Project,
|
|
pub builds: Option<Vec<Build>>,
|
|
}
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct PipelineAttributes {
|
|
pub id: u64,
|
|
pub status: String,
|
|
#[serde(rename = "ref")]
|
|
pub ref_name: String,
|
|
pub sha: String,
|
|
}
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct NoteWebhook {
|
|
pub object_kind: String,
|
|
pub object_attributes: NoteAttributes,
|
|
pub project: Project,
|
|
pub issue: Option<IssueRef>,
|
|
}
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct NoteAttributes {
|
|
pub id: u64,
|
|
pub note: String,
|
|
pub noteable_type: String,
|
|
}
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct Project {
|
|
pub id: u64,
|
|
pub name: String,
|
|
pub web_url: String,
|
|
}
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct Build {
|
|
pub id: u64,
|
|
pub name: String,
|
|
pub status: String,
|
|
pub stage: String,
|
|
}
|
|
|
|
#[derive(Debug, Deserialize)]
|
|
pub struct IssueRef {
|
|
pub iid: u64,
|
|
pub title: String,
|
|
}
|
|
```
|
|
|
|
**Step 4: Run tests to verify they pass**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- webhook`
|
|
Expected: 2 tests PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/webhook.rs services/agent_dispatcher/src/gitlab.rs
|
|
git commit -m "feat(agent-dispatcher): webhook signature verification + GitLab types"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: GitLab API client
|
|
|
|
**Files:**
|
|
- Create: `services/agent_dispatcher/src/gitlab_client.rs`
|
|
|
|
**Step 1: Write the failing test**
|
|
|
|
```rust
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_client_builds_issue_body() {
|
|
let body = format_ci_failure_issue(
|
|
"check",
|
|
"cargo check --workspace",
|
|
"error[E0433]: failed to resolve",
|
|
"abc1234",
|
|
"main",
|
|
);
|
|
assert!(body.contains("## CI Failure"));
|
|
assert!(body.contains("error[E0433]"));
|
|
assert!(body.contains("abc1234"));
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- gitlab_client`
|
|
Expected: FAIL
|
|
|
|
**Step 3: Implement GitLab API client**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/gitlab_client.rs
|
|
use reqwest::Client;
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
#[derive(Clone)]
|
|
pub struct GitLabClient {
|
|
client: Client,
|
|
base_url: String,
|
|
token: String,
|
|
project_id: u64,
|
|
}
|
|
|
|
#[derive(Serialize)]
|
|
struct CreateIssueRequest {
|
|
title: String,
|
|
description: String,
|
|
labels: String,
|
|
}
|
|
|
|
#[derive(Serialize)]
|
|
struct CreateCommentRequest {
|
|
body: String,
|
|
}
|
|
|
|
#[derive(Serialize)]
|
|
struct UpdateLabelsRequest {
|
|
labels: String,
|
|
}
|
|
|
|
#[derive(Deserialize)]
|
|
pub struct Issue {
|
|
pub iid: u64,
|
|
pub title: String,
|
|
pub labels: Vec<String>,
|
|
}
|
|
|
|
impl GitLabClient {
|
|
pub fn new(base_url: String, token: String, project_id: u64) -> Self {
|
|
Self {
|
|
client: Client::new(),
|
|
base_url,
|
|
token,
|
|
project_id,
|
|
}
|
|
}
|
|
|
|
pub async fn fetch_job_log(&self, job_id: u64) -> Result<String, reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/jobs/{}/trace",
|
|
self.base_url, self.project_id, job_id
|
|
);
|
|
self.client
|
|
.get(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.send()
|
|
.await?
|
|
.text()
|
|
.await
|
|
}
|
|
|
|
pub async fn create_issue(
|
|
&self,
|
|
title: &str,
|
|
description: &str,
|
|
labels: &[&str],
|
|
) -> Result<Issue, reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/issues",
|
|
self.base_url, self.project_id
|
|
);
|
|
self.client
|
|
.post(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.json(&CreateIssueRequest {
|
|
title: title.to_string(),
|
|
description: description.to_string(),
|
|
labels: labels.join(","),
|
|
})
|
|
.send()
|
|
.await?
|
|
.json()
|
|
.await
|
|
}
|
|
|
|
pub async fn add_comment(
|
|
&self,
|
|
issue_iid: u64,
|
|
body: &str,
|
|
) -> Result<(), reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/issues/{}/notes",
|
|
self.base_url, self.project_id, issue_iid
|
|
);
|
|
self.client
|
|
.post(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.json(&CreateCommentRequest {
|
|
body: body.to_string(),
|
|
})
|
|
.send()
|
|
.await?
|
|
.error_for_status()?;
|
|
Ok(())
|
|
}
|
|
|
|
pub async fn update_labels(
|
|
&self,
|
|
issue_iid: u64,
|
|
labels: &[&str],
|
|
) -> Result<(), reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/issues/{}",
|
|
self.base_url, self.project_id, issue_iid
|
|
);
|
|
self.client
|
|
.put(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.json(&UpdateLabelsRequest {
|
|
labels: labels.join(","),
|
|
})
|
|
.send()
|
|
.await?
|
|
.error_for_status()?;
|
|
Ok(())
|
|
}
|
|
|
|
pub async fn get_issue_labels(&self, issue_iid: u64) -> Result<Vec<String>, reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/issues/{}",
|
|
self.base_url, self.project_id, issue_iid
|
|
);
|
|
let issue: Issue = self.client
|
|
.get(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.send()
|
|
.await?
|
|
.json()
|
|
.await?;
|
|
Ok(issue.labels)
|
|
}
|
|
}
|
|
|
|
pub fn format_ci_failure_issue(
|
|
stage: &str,
|
|
job_name: &str,
|
|
log_excerpt: &str,
|
|
sha: &str,
|
|
ref_name: &str,
|
|
) -> String {
|
|
// Truncate log to last 200 lines if needed
|
|
let lines: Vec<&str> = log_excerpt.lines().collect();
|
|
let truncated = if lines.len() > 200 {
|
|
lines[lines.len() - 200..].join("\n")
|
|
} else {
|
|
log_excerpt.to_string()
|
|
};
|
|
|
|
format!(
|
|
"## CI Failure: `{job_name}` in stage `{stage}`\n\n\
|
|
**Commit**: `{sha}`\n\
|
|
**Branch**: `{ref_name}`\n\n\
|
|
### Log Excerpt\n\n\
|
|
```\n{truncated}\n```\n\n\
|
|
---\n\
|
|
*Auto-created by agent dispatcher. Agent will diagnose shortly.*"
|
|
)
|
|
}
|
|
```
|
|
|
|
**Step 4: Run test to verify it passes**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- gitlab_client`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/gitlab_client.rs
|
|
git commit -m "feat(agent-dispatcher): GitLab API client (issues, comments, labels, job logs)"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: Webhook handler + pipeline failure flow
|
|
|
|
**Files:**
|
|
- Modify: `services/agent_dispatcher/src/main.rs`
|
|
- Create: `services/agent_dispatcher/src/config.rs`
|
|
- Create: `services/agent_dispatcher/src/handler.rs`
|
|
|
|
**Step 1: Write config module**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/config.rs
|
|
#[derive(Clone)]
|
|
pub struct AppConfig {
|
|
pub gitlab_base_url: String,
|
|
pub gitlab_token: String,
|
|
pub gitlab_project_id: u64,
|
|
pub webhook_secret: String,
|
|
pub max_concurrent_agents: usize,
|
|
pub max_retries: u8,
|
|
}
|
|
|
|
impl AppConfig {
|
|
pub fn from_env() -> Self {
|
|
Self {
|
|
gitlab_base_url: std::env::var("GITLAB_BASE_URL")
|
|
.unwrap_or_else(|_| "https://git.fxhnt.ai".to_string()),
|
|
gitlab_token: std::env::var("GITLAB_TOKEN")
|
|
.expect("GITLAB_TOKEN required"),
|
|
gitlab_project_id: std::env::var("GITLAB_PROJECT_ID")
|
|
.expect("GITLAB_PROJECT_ID required")
|
|
.parse()
|
|
.expect("GITLAB_PROJECT_ID must be u64"),
|
|
webhook_secret: std::env::var("WEBHOOK_SECRET")
|
|
.unwrap_or_default(),
|
|
max_concurrent_agents: std::env::var("MAX_CONCURRENT_AGENTS")
|
|
.unwrap_or_else(|_| "2".to_string())
|
|
.parse()
|
|
.unwrap_or(2),
|
|
max_retries: 3,
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Write the failing test (handler routes webhook to correct flow)**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/handler.rs
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_extract_failed_jobs() {
|
|
let builds = vec![
|
|
Build { id: 1, name: "check".to_string(), status: "success".to_string(), stage: "check".to_string() },
|
|
Build { id: 2, name: "test-ml".to_string(), status: "failed".to_string(), stage: "test".to_string() },
|
|
Build { id: 3, name: "test-risk".to_string(), status: "failed".to_string(), stage: "test".to_string() },
|
|
];
|
|
let failed = extract_failed_jobs(&builds);
|
|
assert_eq!(failed.len(), 2);
|
|
assert_eq!(failed[0].name, "test-ml");
|
|
}
|
|
|
|
#[test]
|
|
fn test_detect_agent_mention() {
|
|
assert!(contains_agent_mention("Please fix this @agent"));
|
|
assert!(contains_agent_mention("@agent can you look at this?"));
|
|
assert!(!contains_agent_mention("No mention here"));
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 3: Run tests to verify they fail**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- handler`
|
|
Expected: FAIL
|
|
|
|
**Step 4: Implement handler**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/handler.rs
|
|
use axum::{
|
|
body::Bytes,
|
|
extract::State,
|
|
http::{HeaderMap, StatusCode},
|
|
response::IntoResponse,
|
|
};
|
|
use std::sync::Arc;
|
|
use tokio::sync::Semaphore;
|
|
|
|
use crate::config::AppConfig;
|
|
use crate::gitlab::{Build, NoteWebhook, PipelineWebhook};
|
|
use crate::gitlab_client::{format_ci_failure_issue, GitLabClient};
|
|
use crate::webhook::verify_signature;
|
|
|
|
pub struct AppState {
|
|
pub config: AppConfig,
|
|
pub gitlab: GitLabClient,
|
|
pub concurrency: Arc<Semaphore>,
|
|
}
|
|
|
|
pub fn extract_failed_jobs(builds: &[Build]) -> Vec<&Build> {
|
|
builds.iter().filter(|b| b.status == "failed").collect()
|
|
}
|
|
|
|
pub fn contains_agent_mention(text: &str) -> bool {
|
|
text.contains("@agent")
|
|
}
|
|
|
|
pub async fn handle_webhook(
|
|
State(state): State<Arc<AppState>>,
|
|
headers: HeaderMap,
|
|
body: Bytes,
|
|
) -> impl IntoResponse {
|
|
// Verify signature if secret is configured
|
|
if !state.config.webhook_secret.is_empty() {
|
|
let sig = headers
|
|
.get("X-Gitlab-Token")
|
|
.and_then(|v| v.to_str().ok())
|
|
.unwrap_or("");
|
|
if sig != state.config.webhook_secret {
|
|
return StatusCode::UNAUTHORIZED;
|
|
}
|
|
}
|
|
|
|
let event = headers
|
|
.get("X-Gitlab-Event")
|
|
.and_then(|v| v.to_str().ok())
|
|
.unwrap_or("");
|
|
|
|
match event {
|
|
"Pipeline Hook" => {
|
|
if let Ok(payload) = serde_json::from_slice::<PipelineWebhook>(&body) {
|
|
if payload.object_attributes.status == "failed" {
|
|
let state = Arc::clone(&state);
|
|
tokio::spawn(async move {
|
|
if let Err(e) = handle_pipeline_failure(state, payload).await {
|
|
tracing::error!("pipeline failure handler error: {e}");
|
|
}
|
|
});
|
|
}
|
|
}
|
|
StatusCode::OK
|
|
}
|
|
"Note Hook" => {
|
|
if let Ok(payload) = serde_json::from_slice::<NoteWebhook>(&body) {
|
|
if contains_agent_mention(&payload.object_attributes.note) {
|
|
let state = Arc::clone(&state);
|
|
tokio::spawn(async move {
|
|
if let Err(e) = handle_agent_mention(state, payload).await {
|
|
tracing::error!("agent mention handler error: {e}");
|
|
}
|
|
});
|
|
}
|
|
}
|
|
StatusCode::OK
|
|
}
|
|
_ => StatusCode::OK,
|
|
}
|
|
}
|
|
|
|
async fn handle_pipeline_failure(
|
|
state: Arc<AppState>,
|
|
payload: PipelineWebhook,
|
|
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
|
let failed = payload.builds.as_deref().map(extract_failed_jobs).unwrap_or_default();
|
|
if failed.is_empty() {
|
|
return Ok(());
|
|
}
|
|
|
|
// Fetch log from first failed job
|
|
let job = failed[0];
|
|
let log = state.gitlab.fetch_job_log(job.id).await.unwrap_or_default();
|
|
|
|
let title = format!(
|
|
"[CI] {} failed on {}",
|
|
job.name, payload.object_attributes.ref_name
|
|
);
|
|
let body = format_ci_failure_issue(
|
|
&job.stage,
|
|
&job.name,
|
|
&log,
|
|
&payload.object_attributes.sha,
|
|
&payload.object_attributes.ref_name,
|
|
);
|
|
|
|
let labels = &["trigger/ci-failure", "phase/diagnose", "agent/active"];
|
|
let issue = state.gitlab.create_issue(&title, &body, labels).await?;
|
|
|
|
tracing::info!(
|
|
issue_iid = issue.iid,
|
|
job_name = job.name,
|
|
"created agent issue for CI failure"
|
|
);
|
|
|
|
// TODO Phase 2: invoke OpenHands diagnose agent here
|
|
Ok(())
|
|
}
|
|
|
|
async fn handle_agent_mention(
|
|
state: Arc<AppState>,
|
|
payload: NoteWebhook,
|
|
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
|
let issue = match payload.issue {
|
|
Some(issue) => issue,
|
|
None => return Ok(()), // Only handle issue comments
|
|
};
|
|
|
|
let labels = &["trigger/feature", "phase/diagnose", "agent/active"];
|
|
state.gitlab.update_labels(issue.iid, labels).await?;
|
|
|
|
tracing::info!(
|
|
issue_iid = issue.iid,
|
|
"activated agent for @agent mention"
|
|
);
|
|
|
|
// TODO Phase 2: invoke OpenHands diagnose agent here
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
**Step 5: Wire up main.rs**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/main.rs
|
|
mod config;
|
|
mod gitlab;
|
|
mod gitlab_client;
|
|
mod handler;
|
|
mod webhook;
|
|
|
|
use std::sync::Arc;
|
|
|
|
use axum::{Router, routing::{get, post}};
|
|
use tokio::sync::Semaphore;
|
|
use tracing_subscriber::EnvFilter;
|
|
|
|
use config::AppConfig;
|
|
use gitlab_client::GitLabClient;
|
|
use handler::{AppState, handle_webhook};
|
|
|
|
#[tokio::main]
|
|
async fn main() {
|
|
tracing_subscriber::fmt()
|
|
.with_env_filter(EnvFilter::from_default_env())
|
|
.init();
|
|
|
|
let config = AppConfig::from_env();
|
|
let gitlab = GitLabClient::new(
|
|
config.gitlab_base_url.clone(),
|
|
config.gitlab_token.clone(),
|
|
config.gitlab_project_id,
|
|
);
|
|
let concurrency = Arc::new(Semaphore::new(config.max_concurrent_agents));
|
|
|
|
let state = Arc::new(AppState {
|
|
config,
|
|
gitlab,
|
|
concurrency,
|
|
});
|
|
|
|
let app = Router::new()
|
|
.route("/health", get(|| async { "ok" }))
|
|
.route("/webhook", post(handle_webhook))
|
|
.with_state(state);
|
|
|
|
let listener = tokio::net::TcpListener::bind("0.0.0.0:8080")
|
|
.await
|
|
.expect("failed to bind port 8080");
|
|
|
|
tracing::info!("dispatcher listening on :8080");
|
|
axum::serve(listener, app).await.expect("server error");
|
|
}
|
|
```
|
|
|
|
**Step 6: Run tests**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib`
|
|
Expected: all tests PASS
|
|
|
|
**Step 7: Run check**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo check -p agent_dispatcher`
|
|
Expected: zero errors
|
|
|
|
**Step 8: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/
|
|
git commit -m "feat(agent-dispatcher): webhook handler with pipeline failure → issue creation"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: Deploy dispatcher to Kapsule
|
|
|
|
**Files:**
|
|
- Create: `infra/k8s/agents/dispatcher.yaml`
|
|
|
|
**Step 1: Write deployment manifest**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/dispatcher.yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: dispatcher
|
|
namespace: foxhunt-agents
|
|
labels:
|
|
app: dispatcher
|
|
spec:
|
|
replicas: 1
|
|
selector:
|
|
matchLabels:
|
|
app: dispatcher
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: dispatcher
|
|
spec:
|
|
nodeSelector:
|
|
pool: ci
|
|
containers:
|
|
- name: dispatcher
|
|
image: rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:latest
|
|
ports:
|
|
- containerPort: 8080
|
|
env:
|
|
- name: GITLAB_BASE_URL
|
|
value: "https://git.fxhnt.ai"
|
|
- name: GITLAB_PROJECT_ID
|
|
value: "1" # Adjust to actual project ID
|
|
- name: WEBHOOK_SECRET
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: agent-credentials
|
|
key: GITLAB_TOKEN
|
|
- name: MAX_CONCURRENT_AGENTS
|
|
value: "2"
|
|
- name: RUST_LOG
|
|
value: "info"
|
|
envFrom:
|
|
- secretRef:
|
|
name: agent-credentials
|
|
resources:
|
|
requests:
|
|
cpu: "50m"
|
|
memory: "64Mi"
|
|
limits:
|
|
cpu: "200m"
|
|
memory: "128Mi"
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 8080
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 10
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 8080
|
|
initialDelaySeconds: 10
|
|
periodSeconds: 20
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: dispatcher
|
|
namespace: foxhunt-agents
|
|
spec:
|
|
selector:
|
|
app: dispatcher
|
|
ports:
|
|
- port: 8080
|
|
targetPort: 8080
|
|
```
|
|
|
|
**Step 2: Create Dockerfile**
|
|
|
|
```dockerfile
|
|
# infra/docker/Dockerfile.agent-dispatcher
|
|
FROM rust:1.89-bookworm AS builder
|
|
RUN apt-get update && apt-get install -y mold clang
|
|
WORKDIR /build
|
|
COPY . .
|
|
RUN SQLX_OFFLINE=true cargo build --release -p agent_dispatcher
|
|
RUN strip target/release/agent-dispatcher
|
|
|
|
FROM debian:bookworm-slim
|
|
RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*
|
|
COPY --from=builder /build/target/release/agent-dispatcher /usr/local/bin/
|
|
ENTRYPOINT ["agent-dispatcher"]
|
|
```
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add -f infra/k8s/agents/dispatcher.yaml infra/docker/Dockerfile.agent-dispatcher
|
|
git commit -m "infra(agents): dispatcher deployment manifest + Dockerfile"
|
|
```
|
|
|
|
**Step 4: Build and push (manual)**
|
|
|
|
Run: `docker build -f infra/docker/Dockerfile.agent-dispatcher -t rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:latest . && docker push rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:latest`
|
|
|
|
**Step 5: Deploy**
|
|
|
|
Run: `kubectl apply -f infra/k8s/agents/dispatcher.yaml`
|
|
|
|
---
|
|
|
|
### Task 9: Configure GitLab webhook
|
|
|
|
**Step 1: Create labels via GitLab API**
|
|
|
|
Run:
|
|
```bash
|
|
for label in "agent/active:#28a745" "agent/blocked:#dc3545" "agent/done:#6c757d" \
|
|
"phase/diagnose:#007bff" "phase/plan:#6f42c1" "phase/implement:#fd7e14" \
|
|
"phase/verify:#ffc107" "phase/deliver:#28a745" \
|
|
"trigger/ci-failure:#dc3545" "trigger/feature:#17a2b8" \
|
|
"retry/1:#6c757d" "retry/2:#6c757d" "retry/3:#6c757d"; do
|
|
name="${label%%:*}"
|
|
color="${label#*:}"
|
|
curl -s --request POST "https://git.fxhnt.ai/api/v4/projects/1/labels" \
|
|
--header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
|
|
--data "name=$name&color=$color"
|
|
done
|
|
```
|
|
|
|
**Step 2: Register webhook in GitLab**
|
|
|
|
Run:
|
|
```bash
|
|
curl -s --request POST "https://git.fxhnt.ai/api/v4/projects/1/hooks" \
|
|
--header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
|
|
--data "url=http://dispatcher.foxhunt-agents.svc.cluster.local:8080/webhook" \
|
|
--data "pipeline_events=true" \
|
|
--data "note_events=true" \
|
|
--data "token=$WEBHOOK_SECRET"
|
|
```
|
|
|
|
Expected: 201 Created with hook ID
|
|
|
|
No commit — this is GitLab configuration.
|
|
|
|
---
|
|
|
|
## Phase 2: OpenHands + Diagnose Agent
|
|
|
|
Deploy OpenHands with K8s runtime and implement the first microagent.
|
|
|
|
---
|
|
|
|
### Task 10: OpenHands RBAC and config
|
|
|
|
**Files:**
|
|
- Create: `infra/k8s/agents/openhands-rbac.yaml`
|
|
- Create: `infra/k8s/agents/openhands-config.yaml`
|
|
|
|
**Step 1: Write RBAC for OpenHands pod scheduling**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/openhands-rbac.yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: openhands-agent
|
|
namespace: foxhunt-agents
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: Role
|
|
metadata:
|
|
name: openhands-pod-manager
|
|
namespace: foxhunt-agents
|
|
rules:
|
|
- apiGroups: [""]
|
|
resources: ["pods", "pods/log", "pods/exec"]
|
|
verbs: ["get", "list", "watch", "create", "delete"]
|
|
- apiGroups: [""]
|
|
resources: ["persistentvolumeclaims"]
|
|
verbs: ["get", "list"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: RoleBinding
|
|
metadata:
|
|
name: openhands-pod-manager
|
|
namespace: foxhunt-agents
|
|
roleRef:
|
|
apiGroup: rbac.authorization.k8s.io
|
|
kind: Role
|
|
name: openhands-pod-manager
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: openhands-agent
|
|
namespace: foxhunt-agents
|
|
```
|
|
|
|
**Step 2: Write OpenHands config**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/openhands-config.yaml
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: openhands-config
|
|
namespace: foxhunt-agents
|
|
data:
|
|
config.toml: |
|
|
[core]
|
|
workspace_base = "/tmp/openhands-workspace"
|
|
max_iterations = 50
|
|
|
|
[llm]
|
|
model = "opus"
|
|
base_url = "http://litellm.foxhunt-agents.svc.cluster.local:4000/v1"
|
|
api_key = "via-litellm"
|
|
|
|
[sandbox]
|
|
timeout = 300
|
|
enable_gpu = false
|
|
|
|
[kubernetes]
|
|
namespace = "foxhunt-agents"
|
|
---
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: agent-workspace
|
|
namespace: foxhunt-agents
|
|
spec:
|
|
accessModes: ["ReadWriteOnce"]
|
|
storageClassName: scw-bssd
|
|
resources:
|
|
requests:
|
|
storage: 10Gi
|
|
```
|
|
|
|
**Step 3: Apply**
|
|
|
|
Run: `kubectl apply -f infra/k8s/agents/openhands-rbac.yaml -f infra/k8s/agents/openhands-config.yaml`
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add -f infra/k8s/agents/openhands-rbac.yaml infra/k8s/agents/openhands-config.yaml
|
|
git commit -m "infra(agents): OpenHands RBAC, config, and workspace PVC"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 11: Deploy OpenHands server
|
|
|
|
**Files:**
|
|
- Create: `infra/k8s/agents/openhands.yaml`
|
|
|
|
**Step 1: Write OpenHands deployment**
|
|
|
|
```yaml
|
|
# infra/k8s/agents/openhands.yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: openhands
|
|
namespace: foxhunt-agents
|
|
labels:
|
|
app: openhands
|
|
spec:
|
|
replicas: 1
|
|
selector:
|
|
matchLabels:
|
|
app: openhands
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: openhands
|
|
spec:
|
|
serviceAccountName: openhands-agent
|
|
nodeSelector:
|
|
pool: ci
|
|
containers:
|
|
- name: openhands
|
|
image: ghcr.io/openhands/openhands:latest
|
|
ports:
|
|
- containerPort: 3000
|
|
env:
|
|
- name: LLM_BASE_URL
|
|
value: "http://litellm.foxhunt-agents.svc.cluster.local:4000/v1"
|
|
- name: LLM_API_KEY
|
|
value: "via-litellm"
|
|
- name: LLM_MODEL
|
|
value: "opus"
|
|
- name: SANDBOX_RUNTIME_CONTAINER_IMAGE
|
|
value: "ghcr.io/openhands/runtime:latest"
|
|
envFrom:
|
|
- secretRef:
|
|
name: agent-credentials
|
|
volumeMounts:
|
|
- name: config
|
|
mountPath: /app/config.toml
|
|
subPath: config.toml
|
|
- name: workspace
|
|
mountPath: /tmp/openhands-workspace
|
|
resources:
|
|
requests:
|
|
cpu: "500m"
|
|
memory: "1Gi"
|
|
limits:
|
|
cpu: "2"
|
|
memory: "4Gi"
|
|
volumes:
|
|
- name: config
|
|
configMap:
|
|
name: openhands-config
|
|
- name: workspace
|
|
persistentVolumeClaim:
|
|
claimName: agent-workspace
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: openhands
|
|
namespace: foxhunt-agents
|
|
spec:
|
|
selector:
|
|
app: openhands
|
|
ports:
|
|
- port: 3000
|
|
targetPort: 3000
|
|
```
|
|
|
|
**Step 2: Apply**
|
|
|
|
Run: `kubectl apply -f infra/k8s/agents/openhands.yaml`
|
|
|
|
**Step 3: Wait for ready**
|
|
|
|
Run: `kubectl rollout status deployment/openhands -n foxhunt-agents --timeout=180s`
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add -f infra/k8s/agents/openhands.yaml
|
|
git commit -m "infra(agents): deploy OpenHands server with K8s runtime"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 12: Write diagnose microagent
|
|
|
|
**Files:**
|
|
- Create: `.openhands/microagents/diagnose.md`
|
|
|
|
**Step 1: Write microagent definition**
|
|
|
|
```markdown
|
|
<!-- .openhands/microagents/diagnose.md -->
|
|
---
|
|
name: diagnose
|
|
agent: CodeActAgent
|
|
---
|
|
|
|
# Diagnose Agent
|
|
|
|
You are a Rust CI diagnostician for the foxhunt HFT trading system.
|
|
|
|
## Your Task
|
|
|
|
Analyze the CI failure described in this GitLab issue and produce a root cause analysis.
|
|
|
|
## Context
|
|
|
|
- This is a Rust workspace with 30+ crates, 4500+ tests
|
|
- Build uses `SQLX_OFFLINE=true cargo check --workspace`
|
|
- Tests use `SQLX_OFFLINE=true cargo test -p <crate> --lib`
|
|
- Clippy: `SQLX_OFFLINE=true cargo clippy --workspace`
|
|
- Crate deny rules: `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::indexing_slicing)]`
|
|
- Linker: clang+mold
|
|
|
|
## Process
|
|
|
|
1. Read the issue description to understand the failure
|
|
2. Clone the repo if not already available: `git clone ssh://git@100.90.76.85:2222/root/foxhunt.git`
|
|
3. Identify the failing stage and job from the CI log excerpt
|
|
4. Reproduce the error locally: run the exact command that failed
|
|
5. Trace the error to its root cause (file, line, reason)
|
|
6. Check recent commits with `git log --oneline -10` for likely culprits
|
|
|
|
## Output Format
|
|
|
|
Post a GitLab comment with this exact structure:
|
|
|
|
```
|
|
## Phase 1: Diagnose Complete
|
|
|
|
**Agent**: diagnose
|
|
**LLM**: opus
|
|
**Duration**: {elapsed}s
|
|
**Tokens**: {input_tokens} in / {output_tokens} out
|
|
|
|
### Root Cause
|
|
{1-2 sentence summary}
|
|
|
|
### Details
|
|
- **Failing stage**: {stage name}
|
|
- **Failing job**: {job name}
|
|
- **Error type**: {compilation|test|clippy|link}
|
|
- **File(s)**: {file paths with line numbers}
|
|
- **Cause**: {detailed explanation}
|
|
- **Likely culprit commit**: {short sha + message}
|
|
|
|
### Handoff
|
|
**Next phase**: plan
|
|
**Context for next agent**: {key facts the plan agent needs — affected files, error type, whether it's a simple typo or architectural issue}
|
|
```
|
|
|
|
## Constraints
|
|
|
|
- Do NOT attempt to fix the code — only diagnose
|
|
- Focus on precision: identify the exact file and line
|
|
- If multiple errors, identify the root one (others may cascade)
|
|
- If the issue is a feature request (not CI failure), summarize the requirements instead
|
|
```
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add -f .openhands/microagents/diagnose.md
|
|
git commit -m "feat(agents): add diagnose microagent definition"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 13: Add OpenHands invocation to dispatcher
|
|
|
|
**Files:**
|
|
- Create: `services/agent_dispatcher/src/openhands.rs`
|
|
- Modify: `services/agent_dispatcher/src/handler.rs`
|
|
|
|
**Step 1: Write the failing test**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/openhands.rs
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_build_headless_task() {
|
|
let task = build_task(
|
|
Phase::Diagnose,
|
|
42,
|
|
"CI failure in test-ml",
|
|
);
|
|
assert!(task.contains("issue #42"));
|
|
assert!(task.contains("diagnose"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_phase_to_model() {
|
|
assert_eq!(phase_model(&Phase::Diagnose), "opus");
|
|
assert_eq!(phase_model(&Phase::Plan), "opus");
|
|
assert_eq!(phase_model(&Phase::Implement), "sonnet");
|
|
assert_eq!(phase_model(&Phase::Verify), "devstral");
|
|
assert_eq!(phase_model(&Phase::Deliver), "devstral");
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- openhands`
|
|
Expected: FAIL
|
|
|
|
**Step 3: Implement OpenHands invocation module**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/openhands.rs
|
|
use reqwest::Client;
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum Phase {
|
|
Diagnose,
|
|
Plan,
|
|
Implement,
|
|
Verify,
|
|
Deliver,
|
|
}
|
|
|
|
impl Phase {
|
|
pub fn label(&self) -> &'static str {
|
|
match self {
|
|
Phase::Diagnose => "phase/diagnose",
|
|
Phase::Plan => "phase/plan",
|
|
Phase::Implement => "phase/implement",
|
|
Phase::Verify => "phase/verify",
|
|
Phase::Deliver => "phase/deliver",
|
|
}
|
|
}
|
|
|
|
pub fn next(&self) -> Option<Phase> {
|
|
match self {
|
|
Phase::Diagnose => Some(Phase::Plan),
|
|
Phase::Plan => Some(Phase::Implement),
|
|
Phase::Implement => Some(Phase::Verify),
|
|
Phase::Verify => Some(Phase::Deliver),
|
|
Phase::Deliver => None,
|
|
}
|
|
}
|
|
}
|
|
|
|
pub fn phase_model(phase: &Phase) -> &'static str {
|
|
match phase {
|
|
Phase::Diagnose | Phase::Plan => "opus",
|
|
Phase::Implement => "sonnet",
|
|
Phase::Verify | Phase::Deliver => "devstral",
|
|
}
|
|
}
|
|
|
|
pub fn build_task(phase: Phase, issue_iid: u64, context: &str) -> String {
|
|
let phase_name = match phase {
|
|
Phase::Diagnose => "diagnose",
|
|
Phase::Plan => "plan",
|
|
Phase::Implement => "implement",
|
|
Phase::Verify => "verify",
|
|
Phase::Deliver => "deliver",
|
|
};
|
|
format!(
|
|
"You are the {phase_name} agent for foxhunt issue #{issue_iid}.\n\n\
|
|
Follow the instructions in .openhands/microagents/{phase_name}.md\n\n\
|
|
## Issue Context\n\n{context}"
|
|
)
|
|
}
|
|
|
|
#[derive(Serialize)]
|
|
struct HeadlessRequest {
|
|
task: String,
|
|
model: String,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
max_iterations: Option<u32>,
|
|
}
|
|
|
|
#[derive(Deserialize)]
|
|
pub struct HeadlessResponse {
|
|
pub status: String,
|
|
pub output: Option<String>,
|
|
}
|
|
|
|
pub struct OpenHandsClient {
|
|
client: Client,
|
|
base_url: String,
|
|
}
|
|
|
|
impl OpenHandsClient {
|
|
pub fn new(base_url: String) -> Self {
|
|
Self {
|
|
client: Client::new(),
|
|
base_url,
|
|
}
|
|
}
|
|
|
|
pub async fn invoke(
|
|
&self,
|
|
phase: Phase,
|
|
issue_iid: u64,
|
|
context: &str,
|
|
) -> Result<HeadlessResponse, reqwest::Error> {
|
|
let task = build_task(phase, issue_iid, context);
|
|
let model = phase_model(&phase);
|
|
|
|
self.client
|
|
.post(format!("{}/api/submit-task", self.base_url))
|
|
.json(&HeadlessRequest {
|
|
task,
|
|
model: model.to_string(),
|
|
max_iterations: Some(50),
|
|
})
|
|
.send()
|
|
.await?
|
|
.json()
|
|
.await
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Run tests**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- openhands`
|
|
Expected: PASS
|
|
|
|
**Step 5: Wire invocation into handler (replace TODO comments)**
|
|
|
|
In `handler.rs`, add the OpenHands invocation call at the `// TODO Phase 2` comment sites.
|
|
|
|
**Step 6: Run full crate tests**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib`
|
|
Expected: all PASS
|
|
|
|
**Step 7: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/openhands.rs services/agent_dispatcher/src/handler.rs
|
|
git commit -m "feat(agent-dispatcher): OpenHands invocation with phase-based model routing"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 3: Plan + Implement Agents
|
|
|
|
Add the plan and implement microagents so the loop can produce code.
|
|
|
|
---
|
|
|
|
### Task 14: Write plan microagent
|
|
|
|
**Files:**
|
|
- Create: `.openhands/microagents/plan.md`
|
|
|
|
**Step 1: Write microagent**
|
|
|
|
```markdown
|
|
<!-- .openhands/microagents/plan.md -->
|
|
---
|
|
name: plan
|
|
agent: CodeActAgent
|
|
---
|
|
|
|
# Plan Agent
|
|
|
|
You are a Rust architecture planner for the foxhunt HFT trading system.
|
|
|
|
## Your Task
|
|
|
|
Read the diagnosis from Phase 1 and create a fix strategy.
|
|
|
|
## Process
|
|
|
|
1. Read the most recent "Phase 1: Diagnose Complete" comment on this issue
|
|
2. Examine the identified file(s) in the codebase
|
|
3. Assess scope: is this a 1-file fix or multi-file change?
|
|
4. Check for related tests that need updating
|
|
5. Identify risks (breaking changes, cascading effects)
|
|
|
|
## Output Format
|
|
|
|
Post a GitLab comment:
|
|
|
|
```
|
|
## Phase 2: Plan Complete
|
|
|
|
**Agent**: plan
|
|
**LLM**: opus
|
|
**Duration**: {elapsed}s
|
|
**Tokens**: {input_tokens} in / {output_tokens} out
|
|
|
|
### Fix Strategy
|
|
{1-2 sentence summary}
|
|
|
|
### Changes
|
|
| File | Action | Description |
|
|
|------|--------|-------------|
|
|
| `path/to/file.rs:42` | Modify | {what to change} |
|
|
| `path/to/test.rs` | Modify | {update test expectations} |
|
|
|
|
### Approach
|
|
{step-by-step implementation guide, 3-7 steps}
|
|
|
|
### Risks
|
|
- {risk 1 and mitigation}
|
|
|
|
### Handoff
|
|
**Next phase**: implement
|
|
**Context for next agent**: {branch name to create, exact changes to make, test commands to run}
|
|
```
|
|
|
|
## Constraints
|
|
|
|
- Do NOT write code — only plan
|
|
- Be specific: exact file paths and line numbers
|
|
- Consider the `#![deny(clippy::unwrap_used, ...)]` rules
|
|
- Verify your proposed changes won't break other crates
|
|
```
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add -f .openhands/microagents/plan.md
|
|
git commit -m "feat(agents): add plan microagent definition"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 15: Write implement microagent
|
|
|
|
**Files:**
|
|
- Create: `.openhands/microagents/implement.md`
|
|
|
|
**Step 1: Write microagent**
|
|
|
|
```markdown
|
|
<!-- .openhands/microagents/implement.md -->
|
|
---
|
|
name: implement
|
|
agent: CodeActAgent
|
|
---
|
|
|
|
# Implement Agent
|
|
|
|
You are a Rust implementation agent for the foxhunt HFT trading system.
|
|
|
|
## Your Task
|
|
|
|
Read the plan from Phase 2 and execute the code changes.
|
|
|
|
## Process
|
|
|
|
1. Read the most recent "Phase 2: Plan Complete" comment on this issue
|
|
2. Create branch: `git checkout -b agent/fix-{issue_iid}`
|
|
3. Make the planned changes, following the exact file/line references
|
|
4. Run `SQLX_OFFLINE=true cargo check --workspace` — fix any errors
|
|
5. Run `SQLX_OFFLINE=true cargo clippy --workspace` — fix any warnings
|
|
6. Run relevant tests: `SQLX_OFFLINE=true cargo test -p {crate} --lib`
|
|
7. Commit with message: `fix: {description} (agent #{issue_iid})`
|
|
8. Push: `git push origin agent/fix-{issue_iid}`
|
|
|
|
## Rules
|
|
|
|
- NEVER use `.unwrap()`, `.expect()`, `.panic!()`, or `[index]` — use `.get()`, `?`, `.ok_or()`
|
|
- NEVER push to `main` — only `agent/fix-*` branches
|
|
- Keep changes minimal — fix the issue, nothing more
|
|
- If `cargo check` fails after your changes, fix it before pushing
|
|
- If the plan is unclear, post a comment asking for clarification and stop
|
|
|
|
## Output Format
|
|
|
|
Post a GitLab comment:
|
|
|
|
```
|
|
## Phase 3: Implement Complete
|
|
|
|
**Agent**: implement
|
|
**LLM**: sonnet
|
|
**Duration**: {elapsed}s
|
|
**Tokens**: {input_tokens} in / {output_tokens} out
|
|
|
|
### Changes Made
|
|
| File | Lines Changed | Description |
|
|
|------|--------------|-------------|
|
|
| `path/to/file.rs` | +5/-2 | {what changed} |
|
|
|
|
### Verification
|
|
- `cargo check`: ✅ PASS
|
|
- `cargo clippy`: ✅ PASS
|
|
- `cargo test -p {crate}`: ✅ PASS ({N} tests)
|
|
|
|
### Branch
|
|
`agent/fix-{issue_iid}` pushed to origin
|
|
|
|
### Handoff
|
|
**Next phase**: verify
|
|
**Context for next agent**: Branch `agent/fix-{issue_iid}` is ready. Run full CI pipeline to verify.
|
|
```
|
|
```
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add -f .openhands/microagents/implement.md
|
|
git commit -m "feat(agents): add implement microagent definition"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 16: Add phase transition logic to dispatcher
|
|
|
|
**Files:**
|
|
- Create: `services/agent_dispatcher/src/phase_machine.rs`
|
|
- Modify: `services/agent_dispatcher/src/handler.rs`
|
|
|
|
**Step 1: Write the failing test**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/phase_machine.rs
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_next_labels_diagnose_to_plan() {
|
|
let current = vec!["trigger/ci-failure", "phase/diagnose", "agent/active"];
|
|
let next = transition_labels(¤t, Phase::Diagnose, Phase::Plan);
|
|
assert!(next.contains(&"phase/plan".to_string()));
|
|
assert!(!next.contains(&"phase/diagnose".to_string()));
|
|
assert!(next.contains(&"agent/active".to_string()));
|
|
}
|
|
|
|
#[test]
|
|
fn test_retry_labels() {
|
|
let current = vec!["phase/verify", "agent/active", "retry/1"];
|
|
let next = add_retry_label(¤t, 2);
|
|
assert!(next.contains(&"retry/2".to_string()));
|
|
assert!(!next.contains(&"retry/1".to_string()));
|
|
}
|
|
|
|
#[test]
|
|
fn test_blocked_on_max_retry() {
|
|
assert!(should_block(3, 3));
|
|
assert!(!should_block(2, 3));
|
|
}
|
|
|
|
#[test]
|
|
fn test_parse_retry_count() {
|
|
let labels = vec!["phase/verify".to_string(), "retry/2".to_string(), "agent/active".to_string()];
|
|
assert_eq!(current_retry_count(&labels), 2);
|
|
}
|
|
|
|
#[test]
|
|
fn test_parse_retry_count_none() {
|
|
let labels = vec!["phase/verify".to_string(), "agent/active".to_string()];
|
|
assert_eq!(current_retry_count(&labels), 0);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- phase_machine`
|
|
Expected: FAIL
|
|
|
|
**Step 3: Implement phase machine**
|
|
|
|
```rust
|
|
// services/agent_dispatcher/src/phase_machine.rs
|
|
use crate::openhands::Phase;
|
|
|
|
pub fn transition_labels(
|
|
current: &[impl AsRef<str>],
|
|
from: Phase,
|
|
to: Phase,
|
|
) -> Vec<String> {
|
|
let from_label = from.label();
|
|
let to_label = to.label();
|
|
let mut result: Vec<String> = current
|
|
.iter()
|
|
.filter(|l| l.as_ref() != from_label)
|
|
.map(|l| l.as_ref().to_string())
|
|
.collect();
|
|
if !result.iter().any(|l| l == to_label) {
|
|
result.push(to_label.to_string());
|
|
}
|
|
result
|
|
}
|
|
|
|
pub fn current_retry_count(labels: &[String]) -> u8 {
|
|
labels
|
|
.iter()
|
|
.filter_map(|l| l.strip_prefix("retry/"))
|
|
.filter_map(|n| n.parse::<u8>().ok())
|
|
.max()
|
|
.unwrap_or(0)
|
|
}
|
|
|
|
pub fn add_retry_label(current: &[impl AsRef<str>], retry_num: u8) -> Vec<String> {
|
|
let mut result: Vec<String> = current
|
|
.iter()
|
|
.filter(|l| !l.as_ref().starts_with("retry/"))
|
|
.map(|l| l.as_ref().to_string())
|
|
.collect();
|
|
result.push(format!("retry/{retry_num}"));
|
|
result
|
|
}
|
|
|
|
pub fn should_block(retry_count: u8, max_retries: u8) -> bool {
|
|
retry_count >= max_retries
|
|
}
|
|
|
|
pub fn blocked_labels(current: &[impl AsRef<str>]) -> Vec<String> {
|
|
let mut result: Vec<String> = current
|
|
.iter()
|
|
.filter(|l| l.as_ref() != "agent/active")
|
|
.map(|l| l.as_ref().to_string())
|
|
.collect();
|
|
if !result.iter().any(|l| l == "agent/blocked") {
|
|
result.push("agent/blocked".to_string());
|
|
}
|
|
result
|
|
}
|
|
|
|
pub fn done_labels(current: &[impl AsRef<str>]) -> Vec<String> {
|
|
let mut result: Vec<String> = current
|
|
.iter()
|
|
.filter(|l| {
|
|
let s = l.as_ref();
|
|
s != "agent/active" && !s.starts_with("phase/")
|
|
})
|
|
.map(|l| l.as_ref().to_string())
|
|
.collect();
|
|
if !result.iter().any(|l| l == "agent/done") {
|
|
result.push("agent/done".to_string());
|
|
}
|
|
result
|
|
}
|
|
```
|
|
|
|
**Step 4: Run tests**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- phase_machine`
|
|
Expected: all PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/phase_machine.rs services/agent_dispatcher/src/handler.rs
|
|
git commit -m "feat(agent-dispatcher): phase transition state machine with retry logic"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 4: Verify + Deliver Agents
|
|
|
|
Close the loop with CI verification and MR creation.
|
|
|
|
---
|
|
|
|
### Task 17: Write verify microagent
|
|
|
|
**Files:**
|
|
- Create: `.openhands/microagents/verify.md`
|
|
|
|
**Step 1: Write microagent**
|
|
|
|
```markdown
|
|
<!-- .openhands/microagents/verify.md -->
|
|
---
|
|
name: verify
|
|
agent: CodeActAgent
|
|
---
|
|
|
|
# Verify Agent
|
|
|
|
You verify that the implementation from Phase 3 passes CI.
|
|
|
|
## Your Task
|
|
|
|
1. Read the "Phase 3: Implement Complete" comment to get the branch name
|
|
2. Trigger CI pipeline on the branch (push is sufficient — GitLab auto-triggers)
|
|
3. Poll pipeline status: `glab ci status --branch agent/fix-{issue_iid}`
|
|
4. Wait up to 15 minutes for completion
|
|
|
|
## Output Format
|
|
|
|
### On GREEN pipeline:
|
|
|
|
```
|
|
## Phase 4: Verify Complete
|
|
|
|
**Agent**: verify
|
|
**LLM**: devstral
|
|
**Duration**: {elapsed}s
|
|
|
|
### Result: ✅ PASS
|
|
Pipeline #{pipeline_id} passed on `agent/fix-{issue_iid}`
|
|
|
|
### Handoff
|
|
**Next phase**: deliver
|
|
**Context for next agent**: Pipeline green. Create MR from `agent/fix-{issue_iid}` to `main`.
|
|
```
|
|
|
|
### On RED pipeline:
|
|
|
|
```
|
|
## Phase 4: Verify Failed
|
|
|
|
**Agent**: verify
|
|
**LLM**: devstral
|
|
|
|
### Result: ❌ FAIL
|
|
Pipeline #{pipeline_id} failed on `agent/fix-{issue_iid}`
|
|
|
|
### Failed Jobs
|
|
| Job | Stage | Error |
|
|
|-----|-------|-------|
|
|
| {name} | {stage} | {first error line} |
|
|
|
|
### Handoff
|
|
**Next phase**: diagnose (retry)
|
|
**Context for next agent**: {error details for re-diagnosis}
|
|
```
|
|
|
|
## Constraints
|
|
|
|
- Do NOT modify code — only check pipeline status
|
|
- If pipeline takes >15 minutes, report timeout and let dispatcher handle retry
|
|
```
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add -f .openhands/microagents/verify.md
|
|
git commit -m "feat(agents): add verify microagent definition"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 18: Write deliver microagent
|
|
|
|
**Files:**
|
|
- Create: `.openhands/microagents/deliver.md`
|
|
|
|
**Step 1: Write microagent**
|
|
|
|
```markdown
|
|
<!-- .openhands/microagents/deliver.md -->
|
|
---
|
|
name: deliver
|
|
agent: CodeActAgent
|
|
---
|
|
|
|
# Deliver Agent
|
|
|
|
You create merge requests for verified fixes.
|
|
|
|
## Your Task
|
|
|
|
1. Read the "Phase 4: Verify Complete" comment for confirmation
|
|
2. Create MR using `glab`:
|
|
```bash
|
|
glab mr create \
|
|
--source-branch agent/fix-{issue_iid} \
|
|
--target-branch main \
|
|
--title "fix: {short description} (agent #{issue_iid})" \
|
|
--description "$(cat <<'EOF'
|
|
## Summary
|
|
{1-2 bullet points from the plan}
|
|
|
|
## Agent Trail
|
|
- Issue: #{issue_iid}
|
|
- Diagnose: Phase 1 comment
|
|
- Plan: Phase 2 comment
|
|
- Implement: Phase 3 comment
|
|
- Verify: Phase 4 comment (pipeline #{pipeline_id})
|
|
|
|
---
|
|
*Auto-generated by foxhunt agent system*
|
|
EOF
|
|
)"
|
|
```
|
|
|
|
## Auto-Merge Rules
|
|
|
|
- If issue has `trigger/ci-failure` label: add `--auto-merge` flag (merge when MR pipeline passes)
|
|
- If issue has `trigger/feature` label: do NOT auto-merge (requires human review)
|
|
|
|
## Output Format
|
|
|
|
```
|
|
## Phase 5: Deliver Complete
|
|
|
|
**Agent**: deliver
|
|
**LLM**: devstral
|
|
**Duration**: {elapsed}s
|
|
|
|
### Merge Request
|
|
MR !{mr_iid}: {title}
|
|
URL: {mr_url}
|
|
Auto-merge: {yes/no}
|
|
|
|
### Handoff
|
|
Agent work complete. Removing `agent/active`, adding `agent/done`.
|
|
```
|
|
|
|
## Constraints
|
|
|
|
- NEVER force-merge or skip CI on the MR
|
|
- NEVER target any branch other than `main`
|
|
```
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add -f .openhands/microagents/deliver.md
|
|
git commit -m "feat(agents): add deliver microagent definition"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 19: Add auto-merge support to dispatcher
|
|
|
|
**Files:**
|
|
- Modify: `services/agent_dispatcher/src/gitlab_client.rs`
|
|
|
|
**Step 1: Write the failing test**
|
|
|
|
```rust
|
|
#[test]
|
|
fn test_is_ci_failure_trigger() {
|
|
let labels = vec!["trigger/ci-failure".to_string(), "agent/active".to_string()];
|
|
assert!(is_ci_failure(&labels));
|
|
|
|
let labels = vec!["trigger/feature".to_string(), "agent/active".to_string()];
|
|
assert!(!is_ci_failure(&labels));
|
|
}
|
|
```
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- gitlab_client::tests::test_is_ci_failure`
|
|
Expected: FAIL
|
|
|
|
**Step 3: Add auto-merge API call to GitLabClient**
|
|
|
|
```rust
|
|
// Add to gitlab_client.rs
|
|
|
|
pub fn is_ci_failure(labels: &[String]) -> bool {
|
|
labels.iter().any(|l| l == "trigger/ci-failure")
|
|
}
|
|
|
|
impl GitLabClient {
|
|
pub async fn set_auto_merge(&self, mr_iid: u64) -> Result<(), reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/merge_requests/{}/merge",
|
|
self.base_url, self.project_id, mr_iid
|
|
);
|
|
self.client
|
|
.put(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.json(&serde_json::json!({
|
|
"merge_when_pipeline_succeeds": true
|
|
}))
|
|
.send()
|
|
.await?
|
|
.error_for_status()?;
|
|
Ok(())
|
|
}
|
|
|
|
pub async fn close_issue(&self, issue_iid: u64) -> Result<(), reqwest::Error> {
|
|
let url = format!(
|
|
"{}/api/v4/projects/{}/issues/{}",
|
|
self.base_url, self.project_id, issue_iid
|
|
);
|
|
self.client
|
|
.put(&url)
|
|
.header("PRIVATE-TOKEN", &self.token)
|
|
.json(&serde_json::json!({ "state_event": "close" }))
|
|
.send()
|
|
.await?
|
|
.error_for_status()?;
|
|
Ok(())
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Run test**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib -- gitlab_client`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/gitlab_client.rs
|
|
git commit -m "feat(agent-dispatcher): auto-merge for CI-failure MRs + issue close"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 20: Wire full phase loop in dispatcher handler
|
|
|
|
**Files:**
|
|
- Modify: `services/agent_dispatcher/src/handler.rs`
|
|
|
|
**Step 1: Implement the full agent loop**
|
|
|
|
Add a `run_agent_loop` function that:
|
|
1. Acquires semaphore permit (concurrency control)
|
|
2. Invokes OpenHands for the current phase
|
|
3. Parses agent comment for handoff signals
|
|
4. Transitions labels
|
|
5. If verify fails → check retry count → loop or block
|
|
6. If deliver completes → set auto-merge (if ci-failure) → close issue → add `agent/done`
|
|
|
|
**Step 2: Run all tests**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib`
|
|
Expected: all PASS
|
|
|
|
**Step 3: Run cargo check**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo check -p agent_dispatcher`
|
|
Expected: zero errors
|
|
|
|
**Step 4: Run clippy**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo clippy -p agent_dispatcher`
|
|
Expected: zero warnings
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add services/agent_dispatcher/src/
|
|
git commit -m "feat(agent-dispatcher): full 5-phase agent loop with retry and auto-merge"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 5: @agent Mention Trigger
|
|
|
|
This was already wired in Task 7 (`handle_agent_mention`). Remaining work is end-to-end testing.
|
|
|
|
---
|
|
|
|
### Task 21: End-to-end smoke test
|
|
|
|
**Step 1: Create a test issue with @agent mention**
|
|
|
|
Run:
|
|
```bash
|
|
curl -s --request POST "https://git.fxhnt.ai/api/v4/projects/1/issues" \
|
|
--header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
|
|
--data-urlencode "title=[Test] Agent smoke test" \
|
|
--data-urlencode "description=@agent Please add a comment to this file: README.md" \
|
|
| jq '.iid'
|
|
```
|
|
Expected: returns issue IID
|
|
|
|
**Step 2: Verify dispatcher picks up the webhook**
|
|
|
|
Run: `kubectl logs -n foxhunt-agents deployment/dispatcher --tail=50 | grep "activated agent"`
|
|
Expected: log line showing agent activation for the test issue
|
|
|
|
**Step 3: Verify labels applied**
|
|
|
|
Run: `curl -s "https://git.fxhnt.ai/api/v4/projects/1/issues/{IID}" --header "PRIVATE-TOKEN: $GITLAB_TOKEN" | jq '.labels'`
|
|
Expected: `["trigger/feature", "phase/diagnose", "agent/active"]`
|
|
|
|
**Step 4: Close test issue**
|
|
|
|
Run: `curl -s --request PUT "https://git.fxhnt.ai/api/v4/projects/1/issues/{IID}" --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --data "state_event=close"`
|
|
|
|
---
|
|
|
|
### Task 22: Add CI job to build dispatcher image
|
|
|
|
**Files:**
|
|
- Modify: `.gitlab-ci.yml`
|
|
|
|
**Step 1: Add build stage for dispatcher**
|
|
|
|
Add a `build-dispatcher` job in the `build` stage:
|
|
|
|
```yaml
|
|
build-dispatcher:
|
|
stage: build
|
|
image: rg.fr-par.scw.cloud/foxhunt-ci/ci-builder:latest
|
|
script:
|
|
- docker build -f infra/docker/Dockerfile.agent-dispatcher
|
|
-t rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:$CI_COMMIT_SHORT_SHA
|
|
-t rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:latest .
|
|
- docker push rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:$CI_COMMIT_SHORT_SHA
|
|
- docker push rg.fr-par.scw.cloud/foxhunt-ci/agent-dispatcher:latest
|
|
rules:
|
|
- changes:
|
|
- services/agent_dispatcher/**/*
|
|
- infra/docker/Dockerfile.agent-dispatcher
|
|
```
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add .gitlab-ci.yml
|
|
git commit -m "ci: add build job for agent-dispatcher image"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 23: Final workspace verification
|
|
|
|
**Step 1: Cargo check entire workspace**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo check --workspace -j12`
|
|
Expected: zero errors
|
|
|
|
**Step 2: Cargo clippy entire workspace**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo clippy --workspace --examples -j12`
|
|
Expected: zero warnings
|
|
|
|
**Step 3: Run dispatcher tests**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p agent_dispatcher --lib`
|
|
Expected: all PASS
|
|
|
|
**Step 4: Run ML tests (regression check)**
|
|
|
|
Run: `SQLX_OFFLINE=true cargo test -p ml --lib -- confidence`
|
|
Expected: 13 tests PASS (ported confidence aggregator still works)
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
| Phase | Tasks | What Ships |
|
|
|-------|-------|-----------|
|
|
| 0 | 1-3 | K8s namespace, secrets, LiteLLM proxy with 3 model routes |
|
|
| 1 | 4-9 | Rust dispatcher (webhook → issue), GitLab labels + webhook |
|
|
| 2 | 10-13 | OpenHands deployment, RBAC, diagnose microagent, invocation wiring |
|
|
| 3 | 14-16 | Plan + implement microagents, phase transition state machine |
|
|
| 4 | 17-20 | Verify + deliver microagents, auto-merge, full loop closure |
|
|
| 5 | 21-23 | @agent trigger e2e test, CI build job, workspace verification |
|
|
|
|
**Total**: 23 tasks across 6 rollout phases.
|
|
|
|
**New files**: ~15 (K8s manifests, Dockerfile, Rust service modules, microagent definitions)
|
|
|
|
**Estimated LLM cost per agent run**: ~$2-5 for CI fix (mostly Sonnet), ~$10-20 for feature work (Opus heavy)
|