← Back to home

Flagship case study

Agent Mesh

Go workers, Redis coordination, RSS-driven backpressure, WebSockets, and a React command center designed to stay readable under high-frequency event flow.

Go · Redis · React · WebSockets 2025 GitHub repo ↗

SUMMARY

Distributed task orchestration system simulating a software-engineering squad at 750+ synthetic events/sec, with explicit browser-side backpressure.

Problem

A high-frequency orchestration UI cannot re-render for every event without turning observability into the bottleneck it was meant to diagnose.

What I built

I built a simulated agentic engineering squad with worker roles, task graph orchestration, event streaming, RSS telemetry, and a command center that buffers and flushes updates intentionally.

Architecture / system design

The system separates worker/queue orchestration from UI state. WebSocket events land in a non-reactive buffer and flush to React state every 100 ms, so the browser paints at a human-readable cadence.

Failure modes / what broke

The browser became the bottleneck before the backend did. The fix was treating UI updates as a backpressure problem instead of pretending React should process every event as app state.

Proof / metrics / tests

The current page documents 750+ synthetic events/sec and a max UI event rate bounded by the browser paint cadence.

Lessons learned

Observability surfaces are systems too. If they have no backpressure model, they can fail exactly when the underlying system gets interesting.

field notes

Expanded field notes

WHERE THE HARD PROBLEM ACTUALLY LIVES

An orchestration experiment: task state, worker coordination, event flow, and UI visibility for a multi-agent workflow. The frontend is here to make the system legible - the interesting questions are on the worker / queue / backpressure side. 750+ events/sec is the synthetic load the project pushes itself to, not a production traffic claim.

Agent Mesh is an event-driven orchestration system that simulates a software-engineering squad — architect, developer, QA — running as Go worker nodes against a Redis-backed task graph, with a React command center that visualizes the whole thing in real time. The hard part isn't the agents; it's keeping the UI honest when 750 events per second hit the browser.

The naive WebSocket-to-state loop tries to repaint every 1.33 ms; a 60 Hz display can only honor a repaint every 16.67 ms. The result is a "rendering avalanche" — the main thread saturates, layouts thrash, cards teleport across the screen faster than the eye can track. Agent Mesh fixes this with a throttled batching engine: a non-reactive ref buffers incoming updates, then flushes them to React state on a fixed 100 ms cadence.

DESIGN NOTE

Trading 100 ms of perceived latency for a 10x drop in render frequency is the whole game. The UI feels real-time, the main thread sits 85% idle, and the system can scale ingest without scaling pixels.

On the backend, workers report RSS (resident set size) via gopsutil — not heap, because heap is noisy. When RSS crosses a soft limit, the worker raises backpressure on the producer until GC reclaims. Task transitions go through atomic Redis BLMOVE and ACID PostgreSQL writes; graceful shutdown rides Go's sync.WaitGroup + context cancellation so no task gets half-persisted on SIGTERM.

THROUGHPUT · WITH vs WITHOUT BATCHING

metric                  no batching        agent mesh
─────────────────────────────────────────────────────
max ui events/sec       ~60 (60Hz cap)     750+
main thread idle %      <5% (laggy)       ~85% (smooth)
state consistency       fragile            guaranteed

fig. 2 — producer → redis → worker pool with RSS-driven backpressure feeding back into ingest.

hover or tab through any node to trace the flow

contact

Open to backend systems, AI infrastructure, and product engineering roles.

yehiasaidgewily@gmail.com GitHub LinkedIn