rhobound

Know what a change costs your latency before it ships.

Every change that reaches production, whether a security patch, a rebuilt image or a pull request written by an agent, can slow services nobody looked at. Rhobound is the performance gate that tells you before rollout: which services slow down, which SLOs break, and how much headroom you lose.

Delay against utilization ρ. The curve stays flat at low load, then climbs steeply toward a vertical line at ρ equal to 1, the bound where a service saturates.0delayρutilization1bound
ρ
How busy a service is. Delay grows like 1/(1−ρ)², so the last few percent cost the most.
bound
The traffic at which a change pushes a service past saturation. The one hard guarantee.

rhobound

Drag the traffic slider. The change stays the same; only the load changes.

55 req/s
Added latency at the entry point
+13.8 ms
Direct cost of the patch
+6.25 ms
Latency objectives broken
1 of 8
Busiest service, ρ
0.48

Queueing already turns the direct cost into 2.2× as much added latency.

Added latency at the entry point against traffic. The same patch adds about 14 ms at 55 requests per second and rises steeply toward saturation near 114 requests per second.05010015020040 req/s118 req/sSaturates at 114 req/s (was 139)
Added mean latency and objective status per service
ServiceAdded latencyAdded msWithin objective
ingress+13.8within objective
auth+7.2objective broken
catalog+6.6within objective
shard-a+6.8within objective
shard-b+3.2within objective
payments+6.2within objective
db-proxy+3.2within objective
checkout+6.2within objective
Bold names run the patched base image. The others slow down only because they wait on services that do. Model of the 12-service reference system, mean latency; checked against a discrete-event simulation.

Software change is accelerating. Nobody checks what it costs.

Change is accelerating

Vulnerabilities are found, and fixed images built, faster every year. One rebuilt base image reaches every service at once.

48,185
CVEs published in 2025, up 20.6% on 2024. That is about 130 a day. Jerry Gamblin, 2025 CVE Data Review
5 days
Average time from disclosure to exploitation in 2023, down from 63 days in 2018–19. Google Cloud / Mandiant

Agents write the change

Coding agents now open pull requests on their own, and the tasks they finish keep getting longer.

1M+
Pull requests created by GitHub’s Copilot coding agent in its first five months, May to September 2025. GitHub Octoverse 2025
~7 months
Doubling time, over six years, of the length of task AI agents can complete on their own at 50% reliability. METR

Nobody checks what it costs

Tests say a change is correct. Nothing says what it does to latency at production load, so teams guess, and pad.

$300k+
Cost of one hour of downtime for over 90% of mid-size and large enterprises. ITIC 2024 Hourly Cost of Downtime
83%
Share of container costs that go to idle resources: over-provisioned clusters and oversized requests. Datadog, State of Cloud Costs 2024

A small slowdown doesn’t stay small

Services queue. A slowdown at a busy service is amplified roughly by 1/(1−ρ)², where ρ is how busy it is, and it spreads to services the change never touched.

  1. ×4at 50% busy
  2. ×11at 70% busy
  3. ×25at 80% busy
  4. ×100at 90% busy

When code is written at machine speed, testing and performance limits have to be checked at machine speed too. DORA’s 2025 research already found that AI adoption is associated with increasing delivery instability.

The gate every change passes through

In an agentic workflow the agent writes the change, opens the pull request and waits for checks. Tests catch wrong code. Rhobound catches slow code, at your peak traffic, before it merges.

  • Scale. Agents open changes around the clock. A human performance review per change does not scale.
  • Correct is not fast. Tests go green on code that slows a busy service.
  • A limit, not an opinion. A verdict with an error band, and a certified breach when a change pushes a service past saturation. That last one is arithmetic, not a forecast.
Send a change through:
  1. Issue or CVEScanner or ticket
  2. Agent writes the fixCoding agent
  3. Pull requestOpened by the agent
  4. CIBuild and tests
  5. Rhobound gateLatency, SLOs, bound
  6. Merge and canaryProgressive delivery
  7. ProductionReal traffic
  1. scannerVulnerability reported in libssl, in the shared base image
  2. agentFix #1 drafted. Benchmarked direct cost: +6.25 ms per request
  3. gitPull request opened
  4. ciBuild ✓ Tests ✓ (correct, but is it fast enough?)
  5. rhoboundFAIL +19.5 ms at the entry point, 4 of 8 SLOs break at 70 req/s
  6. agentBudget returned to the agent: direct cost ≤ 3.87 ms per request
  7. agentFix #2 drafted within budget: +3.40 ms per request
  8. ciBuild ✓ Tests ✓
  9. rhoboundPASS +9.2 ms at the entry point, all 8 SLOs hold at 70 req/s
  10. deployMerged. Canary confirms the measured cost per call
  11. prodRolled out. Telemetry refreshes the model for the next change
Gate verdict
Pass
Added latency at the entry point
+9.2 ms
SLOs holding at 70 req/s
8 of 8
Bound: saturation traffic
139 → 124 req/s
Budget for this change
≤ 3.87 ms direct

Available Every verdict above is computed live by Rhobound’s queueing model on a 12-service reference system at a 70 req/s peak. Roadmap The latency budget returned to the agent, and running Rhobound as a CI check and an MCP tool that agents call.

The performance gate, from traces to a verdict

It sits between “the change builds” and “the change is rolled out”, needs a few minutes of OpenTelemetry traces, and asks for no changes to your code.

  1. Capture traces

    A few minutes of OpenTelemetry traces from your cluster: every request, or a head-sampled share, which Rhobound reads from the trace IDs. No code changes and no switches describing your system.

  2. Infer the architecture

    Calls and their order, parallel fan-outs, replicas and routing, and per pod the worker count and own time, rebuilt from the spans.

  3. Profile the change in your cluster

    The new image runs against the old one among live traffic, replaying your own arrivals with the pod’s own CPU request and placement, so CPU limits and contention show up in the profile.

  4. Predict

    Latency change at every service, an SLO verdict, p99 before and after, lost headroom, and a certified breach when a service would pass 100% busy.

  5. Verify

    After the deploy, the prediction is judged against the new traces, criterion by criterion. Each verified deploy calibrates the next prediction and sets its error range.

What every change gets

  • One impact level

    Available

    None, low, moderate, high or critical, with one line per entry point.

  • Per-service impact

    Available

    Mean latency before and after for every service, including ones the change never touched, with an input-noise band.

  • Percentiles

    Available

    p50 to p99 before and after, and a planning bound: the modelled p99 after the change × 1.10.

  • Capacity

    Available

    The load at which the busiest service saturates, before and after, and a sweep from 0.5× to 1.5× today’s load. After a CPU-bound change, a range that allows for bursts.

  • Error ranges

    Preview

    A range drawn from this cluster’s own past deploys, and a running check of whether earlier ranges held. It needs a few verified deploys first.

  • Risks

    Available

    A certified breach, thin headroom, and own-time bursts large enough to saturate a service after the change.

  • Verification

    Available

    After the deploy, the report against what the new traces show.

  • Latency budget

    Roadmap

    The most slowdown a change may add per call before an objective breaks, handed back to the fixer, human or agent.

Profile the new image, predict the deploy, verify it afterwards. Runs in your environment
rhobound profile CAPTURE registry/auth:1.1
rhobound predict CAPTURE CAPTURE/profiles/*
rhobound verify  CAPTURE/rhobound/report.json AFTER
Excerpt of a real report, condensed. A 10-service Kubernetes benchmark
IMPACT    MODERATE
  checkout    93.5 -> 110.1 ms  (+17.8%)  p99 207 -> 239
  storefront  71.8 ->  78.3 ms   (+9.0%)  p99 206 -> 214
  capacity    180 -> 156 rps (-13%), bottleneck pricing -> auth

PER SERVICE      util          mean ms         change
  payments *     44% -> 55%    23.1 -> 33.3    +10.2 ms
  auth *         48% -> 64%    11.9 -> 18.3     +6.4 ms
  catalog        40% -> 40%    27.1 -> 27.1     +0.0 ms
  * changed image

Headroom over today's load: 81% -> 57%
Inferred from 686,104 spans: 10 services, 11 pods,
2 entry points. Nothing else was given.

Where it fits in your pipeline

  1. ScannersTrivy, Grype, OSV
  2. Patch or rebuildCopacetic, Renovate, agents
  3. CI build and testsGitHub Actions, GitLab
  4. RhoboundPredict impact before rollout
  5. Progressive deliveryArgo Rollouts, Flagger
  6. ObservabilityPrometheus, OpenTelemetry

Built to be audited

Criteria are declared before each run, tested on fresh cases, and the failures are published next to the successes. Every number says what it was measured on.

See every result, by what it was measured on
Headline results
Measured onWhat was testedResult
Kubernetes clusterEntry latency after a real change, 10-service benchmark, traces onlyStorefront 78.28 ms predicted, 78.46 observed (−0.2%). Checkout 110.07 vs 111.32 (−1.1%). Every service within 6.6%
Kubernetes clusterProfile of the changed image, taken in the clusterWithin 1.6% of the deployed change (+3.06 ms profiled, +3.01 ms deployed). The p99 planning bound held
Kubernetes clusterArchitecture inferred from traces aloneReplicas, routing, calls, parallel fan-out and conditional share all matched. Worker counts right on 70 of 70 service-captures
Kubernetes clusterOnline Boutique, a real open-source benchmark in five languages, under CPU limitsEndpoints within 0.3% to 6.4% before the change. Replaying production’s arrivals put the profiled change within 0.1% of production’s and the product and cart pages within 1.1%
Kubernetes clusterSampled traces: 10% of requests on kshop, 25% on Online BoutiqueThe kept share found from the trace IDs. 9 of 9 criteria: entries within 4.4%, and within 0.6 to 3.3 ms of the full capture
Kubernetes clusterCalibration by the cluster’s own past deploys, and error rangesEach calibrated deploy within 10%, median 3.2% (uncalibrated 10.0%). Early: 89% and 94% of ranges held on fresh deploys, against 80% and 90% stated
Kubernetes clusterA change that waits (a lock), and a service with two replicas sharing the loadLock: every criterion 3 of 3 on fresh deploys, the catalog within 1.3%. Two replicas: browse mean within 1.9%, composed p95 within 2.8%
Kubernetes clusterComposed p95 on Online Boutique pages, fresh deploysWithin 20% on 18 of 18, running 5% to 7% high. Real-cluster p99 is not claimed: the truth’s own noise floor is 46%
Public data66 real fault-injection cases on Online BoutiqueFront-end latency change within 9.1% median, at light load
Public dataA real call-center queue at high loadMeasured wait matched within 0.94 to 1.06× from 70% utilization to saturation
SimulationReference 12-service system, 48% to 93% utilizationEntry impact within ±1.8% at every load
Simulation155 fresh random architecturesEntry error median 5.1%, worst 48%
Simulationp99 on 114 fresh cases, and the planning boundEntry p99 mean error 4.3%. The ×1.10 bound held on 95% to 100% of fresh rows
SimulationRetry-driven meltdown onset, 26 fresh casesWithin 5% everywhere
SimulationNaive approach: add up the direct costsUnder-estimates by a median 60% to 94%, every time
Simulation: the same change at different loads. Added latency at the entry point, ms
req/sBusiest ρNaiveRhoboundSimulated
5548%6.2513.813.5
7566%6.2522.322.8
9079%6.2538.037.6
10088%6.2567.064.2
10693%6.25120.7127.5

396 automated tests lock the validated results. Accuracy figures are measured on benchmark applications on a Kubernetes test cluster, on public data from real systems and against independent simulation, and each is labelled. Validation on a customer’s production system is the design-partner milestone.

Open source: rhobound-cli

Our open-source command-line tool. It reads OpenTelemetry traces, infers your service graph and predicts mean latency before and after a change, using the queueing-theory approach behind Rhobound. One static Go binary, Apache-2.0.

rhobound-cli, from the public repository
go install github.com/Sharvil2704/rhobound-cli/cmd/rhobound@latest

rhobound predict ./capture --patch checkout=+3ms

A missing gate in every delivery pipeline

Tests and security scans are standard gates in the pipeline. Performance is the missing one, and agents writing code at machine speed make it urgent. Rhobound is built to be that gate.

  • Platform engineering and SRE

    Own the latency objectives and the capacity bill. Rhobound tells them what a change does to both before rollout.

  • Security and vulnerability management

    Own the patch rollout. A verdict per service lets safe fixes ship now instead of waiting for a performance review.

  • Engineering leadership

    Carry the trade-off between security speed and reliability. They get an evidence-based answer per change.

82%
Of container users run Kubernetes in production. Rhobound’s customers already run the platform it reads. CNCF Annual Cloud Native Survey 2025
41%
Of observability teams use OpenTelemetry in production, extensively or exclusively. Its traces are all Rhobound needs. Grafana Observability Survey 2025

Why today’s tools don’t close it

CategoryWhat it answersWhat Rhobound adds
Vulnerability scannersWhich images contain the packageWhich services execute it, and what the fix costs
CI: build and testsDoes it build and passDoes it keep the latency objectives at real load
Load testingWhat happens under synthetic loadA small targeted benchmark, extended to the whole graph at real traffic
Canary analysisDoes the new version look fine on a sliceWhat a full rollout does, including the amplification a canary cannot see
Observability and APMWhat happened after we shippedWhat will happen, from the same telemetry
Black-box ML predictorsA number, with no reasonDeterministic and explainable, and it refuses outside its validated range

How we grow

  1. Design partners

    We backtest your last patch rollouts and incidents and show what Rhobound would have predicted. Each partner adds a measured record of predicted against deployed.

  2. The pipeline

    A check in CI and a pre-rollout gate for progressive delivery, so every change gets a verdict without anyone asking for one.

  3. The agents’ check

    A tool that coding agents call inside their loop. Clear results let cheaper models keep iterating; borderline ones escalate.

What compounds

Where this goes

  1. Now

    MVP with design partners

    • The rhobound command: profile, predict, verify
    • Backtests on partners’ real rollouts, with predicted against deployed quoted
    • A better first prediction on a new cluster, and the answer-size case: measuring what callers spend decoding a bigger response
    • Onboarding without custom hooks
  2. Next

    The pipeline

    • CI check for GitHub and GitLab, and a rollout gate for Argo Rollouts and Flagger
    • Latency budget per call and per service
    • Validated change of p99, and burstiness measured from real traffic
    • Event-driven consumer lag and autoscaling
  3. Then

    The platform

    • MCP server so coding agents call Rhobound inside their loop
    • Versioned architecture store with every change and its measured impact
    • Signed performance attestations next to image provenance
    • Admission policy: no rollout without a passing verdict
Integrations and their status

What you give it

  • OpenTelemetry tracesThe main input: OTLP/JSON from the Collector’s file exporter
    Available
  • kubectlDeployments, services and pods, for profiling and placement
    Available
  • Zipkin and B3 tracesRead as calls, alongside OTLP
    Available
  • Per-pod CPU countersOptional: rhobound cpu-sample reads the kubelet’s counters into the capture
    Available
  • Kubernetes and DockerWhere the new image is profiled: in the cluster, or on one machine
    Available
  • Istio metricsTraffic, call counts and per-edge latency
    Available
  • CycloneDX SBOMsBase-image lineage; some tools need a small adapter
    Preview
  • Google Benchmark, bpftrace, OpenSLOCost per call, per-request service time, objectives
    Available
  • CloudFront logsCDN edge traffic
    Available
  • Linkerd, Envoy, Parca, PyroscopeMore meshes and continuous profiles
    Roadmap

Upstream: find and fix

  • Trivy, Grype, OSVWhich CVE, which package, which function
    Roadmap
  • EPSS, CISA KEVExploit likelihood for ship-or-defer decisions
    Roadmap
  • CopaceticPatched images to check
    Roadmap
  • Renovate, DependabotEvery update PR gets a verdict
    Roadmap
  • OpenVEXExport reachability as VEX statements
    Vision

Downstream: ship

  • GitHub Actions, GitLab CIThe verdict as a check or PR comment
    Roadmap
  • Argo Rollouts, FlaggerPre-rollout gate on the verdict
    Roadmap
  • k6, wrk2, GoReplayReplay and service-level A/B
    Roadmap
  • Model Context ProtocolTools coding agents call: verify a change, fetch a latency budget
    Roadmap
  • Sigstore, OPA, KyvernoSigned verdicts and admission policy
    Vision

Status refers to Rhobound’s integration, not to the project itself.

Team

Questions

Is it validated?

On Kubernetes benchmark applications, where entry latency after a real change was predicted from traces alone within 1.1% on a 10-service deployment and the product and cart pages of Online Boutique within 1.1%; on public data from real systems; and against independent simulation. A customer’s production system is the design-partner milestone, and we say so wherever a number appears.

Does it predict p99?

Yes. It predicts p50 to p99 before and after each change, and gives a planning bound: the modelled p99 after the change × 1.10, which held on 95% to 100% of fresh simulated cases. On real clusters we claim p95, within 20% on 18 of 18 fresh deploys. Real-cluster p99 is not claimed because the truth’s own noise floor is 46%. The change of p99 itself is not validated, so plan against the bound.

What do we have to change?

Nothing in your code. Rhobound reads standard OpenTelemetry or Zipkin traces, every request or a head-sampled share, plus kubectl output to profile the changed image. Optionally it takes per-pod CPU counters and a sample request body for routes that read one.

Where does our data go?

Rhobound is a command-line tool that runs in your environment on your traces and your cluster. Nothing has to leave it.

Is it machine learning?

No. The core is closed-form queueing mathematics. It is deterministic and explainable, every number traces to an input, and it refuses to answer outside the range it has been validated for.

Does it replace load testing and canaries?

It replaces most full-scale load testing for a change, and complements canaries. A canary measures a change’s direct cost well but misses the amplification that appears only at full rollout. Rhobound predicts that part.

What does it support?

Synchronous request-response microservices: parallel fan-outs, shared databases and caches, replicas with different routing policies, conditional calls, CDN edges, network hops, third-party APIs, timeouts and retries. Also serverless functions. Event-driven consumer lag is on the roadmap.

Become a design partner

We’ll backtest your last few patch rollouts and incidents and show what Rhobound would have predicted. We’re looking for teams running containerized microservices on Kubernetes, with shared base images and OpenTelemetry or a service mesh.