Know what a change costs your latency before it ships.
Every change that reaches production, whether a security patch, a rebuilt image or a pull request written by an agent, can slow services nobody looked at. Rhobound is the performance gate that tells you before rollout: which services slow down, which SLOs break, and how much headroom you lose.
- ρ
- How busy a service is. Delay grows like 1/(1−ρ)², so the last few percent cost the most.
- bound
- The traffic at which a change pushes a service past saturation. The one hard guarantee.
rhobound
Drag the traffic slider. The change stays the same; only the load changes.
- Added latency at the entry point
- +13.8 ms
- Direct cost of the patch
- +6.25 ms
- Latency objectives broken
- 1 of 8
- Busiest service, ρ
- 0.48
Queueing already turns the direct cost into 2.2× as much added latency.
- Kubernetes cluster1.1%Largest entry-point error after a real change on a 10-service Kubernetes benchmark, predicted from traces alone. On Online Boutique the truth itself moves up to 13.6% between windows.
- Simulation4.3%Mean error on the predicted entry-point p99, across 114 fresh architectures it had never seen.
- Record115 roundsOf experiments with criteria declared in advance, 396 automated tests, and the failures published next to the successes.
Software change is accelerating. Nobody checks what it costs.
Change is accelerating
Vulnerabilities are found, and fixed images built, faster every year. One rebuilt base image reaches every service at once.
- 48,185
- CVEs published in 2025, up 20.6% on 2024. That is about 130 a day. Jerry Gamblin, 2025 CVE Data Review
- 5 days
- Average time from disclosure to exploitation in 2023, down from 63 days in 2018–19. Google Cloud / Mandiant
Agents write the change
Coding agents now open pull requests on their own, and the tasks they finish keep getting longer.
- 1M+
- Pull requests created by GitHub’s Copilot coding agent in its first five months, May to September 2025. GitHub Octoverse 2025
- ~7 months
- Doubling time, over six years, of the length of task AI agents can complete on their own at 50% reliability. METR
Nobody checks what it costs
Tests say a change is correct. Nothing says what it does to latency at production load, so teams guess, and pad.
- $300k+
- Cost of one hour of downtime for over 90% of mid-size and large enterprises. ITIC 2024 Hourly Cost of Downtime
- 83%
- Share of container costs that go to idle resources: over-provisioned clusters and oversized requests. Datadog, State of Cloud Costs 2024
A small slowdown doesn’t stay small
Services queue. A slowdown at a busy service is amplified roughly by 1/(1−ρ)², where ρ is how busy it is, and it spreads to services the change never touched.
- ×4at 50% busy
- ×11at 70% busy
- ×25at 80% busy
- ×100at 90% busy
When code is written at machine speed, testing and performance limits have to be checked at machine speed too. DORA’s 2025 research already found that AI adoption is associated with increasing delivery instability.
The gate every change passes through
In an agentic workflow the agent writes the change, opens the pull request and waits for checks. Tests catch wrong code. Rhobound catches slow code, at your peak traffic, before it merges.
- Scale. Agents open changes around the clock. A human performance review per change does not scale.
- Correct is not fast. Tests go green on code that slows a busy service.
- A limit, not an opinion. A verdict with an error band, and a certified breach when a change pushes a service past saturation. That last one is arithmetic, not a forecast.
- Issue or CVEScanner or ticket
- Agent writes the fixCoding agent
- Pull requestOpened by the agent
- CIBuild and tests
- Rhobound gateLatency, SLOs, bound
- Merge and canaryProgressive delivery
- ProductionReal traffic
- scannerVulnerability reported in libssl, in the shared base image
- agentFix #1 drafted. Benchmarked direct cost: +6.25 ms per request
- gitPull request opened
- ciBuild ✓ Tests ✓ (correct, but is it fast enough?)
- rhoboundFAIL +19.5 ms at the entry point, 4 of 8 SLOs break at 70 req/s
- agentBudget returned to the agent: direct cost ≤ 3.87 ms per request
- agentFix #2 drafted within budget: +3.40 ms per request
- ciBuild ✓ Tests ✓
- rhoboundPASS +9.2 ms at the entry point, all 8 SLOs hold at 70 req/s
- deployMerged. Canary confirms the measured cost per call
- prodRolled out. Telemetry refreshes the model for the next change
- Gate verdict
- Pass
- Added latency at the entry point
- +9.2 ms
- SLOs holding at 70 req/s
- 8 of 8
- Bound: saturation traffic
- 139 → 124 req/s
- Budget for this change
- ≤ 3.87 ms direct
Available Every verdict above is computed live by Rhobound’s queueing model on a 12-service reference system at a 70 req/s peak. Roadmap The latency budget returned to the agent, and running Rhobound as a CI check and an MCP tool that agents call.
The performance gate, from traces to a verdict
It sits between “the change builds” and “the change is rolled out”, needs a few minutes of OpenTelemetry traces, and asks for no changes to your code.
Capture traces
A few minutes of OpenTelemetry traces from your cluster: every request, or a head-sampled share, which Rhobound reads from the trace IDs. No code changes and no switches describing your system.
Infer the architecture
Calls and their order, parallel fan-outs, replicas and routing, and per pod the worker count and own time, rebuilt from the spans.
Profile the change in your cluster
The new image runs against the old one among live traffic, replaying your own arrivals with the pod’s own CPU request and placement, so CPU limits and contention show up in the profile.
Predict
Latency change at every service, an SLO verdict, p99 before and after, lost headroom, and a certified breach when a service would pass 100% busy.
Verify
After the deploy, the prediction is judged against the new traces, criterion by criterion. Each verified deploy calibrates the next prediction and sets its error range.
What every change gets
One impact level
AvailableNone, low, moderate, high or critical, with one line per entry point.
Per-service impact
AvailableMean latency before and after for every service, including ones the change never touched, with an input-noise band.
Percentiles
Availablep50 to p99 before and after, and a planning bound: the modelled p99 after the change × 1.10.
Capacity
AvailableThe load at which the busiest service saturates, before and after, and a sweep from 0.5× to 1.5× today’s load. After a CPU-bound change, a range that allows for bursts.
Error ranges
PreviewA range drawn from this cluster’s own past deploys, and a running check of whether earlier ranges held. It needs a few verified deploys first.
Risks
AvailableA certified breach, thin headroom, and own-time bursts large enough to saturate a service after the change.
Verification
AvailableAfter the deploy, the report against what the new traces show.
Latency budget
RoadmapThe most slowdown a change may add per call before an objective breaks, handed back to the fixer, human or agent.
rhobound profile CAPTURE registry/auth:1.1
rhobound predict CAPTURE CAPTURE/profiles/*
rhobound verify CAPTURE/rhobound/report.json AFTERIMPACT MODERATE
checkout 93.5 -> 110.1 ms (+17.8%) p99 207 -> 239
storefront 71.8 -> 78.3 ms (+9.0%) p99 206 -> 214
capacity 180 -> 156 rps (-13%), bottleneck pricing -> auth
PER SERVICE util mean ms change
payments * 44% -> 55% 23.1 -> 33.3 +10.2 ms
auth * 48% -> 64% 11.9 -> 18.3 +6.4 ms
catalog 40% -> 40% 27.1 -> 27.1 +0.0 ms
* changed image
Headroom over today's load: 81% -> 57%
Inferred from 686,104 spans: 10 services, 11 pods,
2 entry points. Nothing else was given.- Deterministic and explainable. Closed-form queueing mathematics, not a black box. Same inputs, same answer, and every number traces to an input.
- Refuses instead of guessing. Contradictory data or inputs outside the validated range return a reason, not a number.
- Runs in your environment. A command-line tool that works on your traces and your cluster. Nothing has to leave it.
- Milliseconds for means. Mean predictions take milliseconds. Percentiles take seconds to minutes at high load.
Where it fits in your pipeline
- ScannersTrivy, Grype, OSV
- Patch or rebuildCopacetic, Renovate, agents
- CI build and testsGitHub Actions, GitLab
- RhoboundPredict impact before rollout
- Progressive deliveryArgo Rollouts, Flagger
- ObservabilityPrometheus, OpenTelemetry
Built to be audited
Criteria are declared before each run, tested on fresh cases, and the failures are published next to the successes. Every number says what it was measured on.
- Kubernetes cluster0.1%Gap between the profiled change and production’s on Online Boutique, after replaying production’s own traffic in the profile.
- Kubernetes cluster9 of 9Criteria passed with only 10% to 25% of requests traced. The sampled share is found from the trace IDs.
- Kubernetes cluster18 of 18Fresh Online Boutique deploys with composed p95 within 20% of what happened.
- Kubernetes cluster70 of 70Worker-pool sizes inferred correctly from traces alone, across every service capture.
See every result, by what it was measured on
| Measured on | What was tested | Result |
|---|---|---|
| Kubernetes cluster | Entry latency after a real change, 10-service benchmark, traces only | Storefront 78.28 ms predicted, 78.46 observed (−0.2%). Checkout 110.07 vs 111.32 (−1.1%). Every service within 6.6% |
| Kubernetes cluster | Profile of the changed image, taken in the cluster | Within 1.6% of the deployed change (+3.06 ms profiled, +3.01 ms deployed). The p99 planning bound held |
| Kubernetes cluster | Architecture inferred from traces alone | Replicas, routing, calls, parallel fan-out and conditional share all matched. Worker counts right on 70 of 70 service-captures |
| Kubernetes cluster | Online Boutique, a real open-source benchmark in five languages, under CPU limits | Endpoints within 0.3% to 6.4% before the change. Replaying production’s arrivals put the profiled change within 0.1% of production’s and the product and cart pages within 1.1% |
| Kubernetes cluster | Sampled traces: 10% of requests on kshop, 25% on Online Boutique | The kept share found from the trace IDs. 9 of 9 criteria: entries within 4.4%, and within 0.6 to 3.3 ms of the full capture |
| Kubernetes cluster | Calibration by the cluster’s own past deploys, and error ranges | Each calibrated deploy within 10%, median 3.2% (uncalibrated 10.0%). Early: 89% and 94% of ranges held on fresh deploys, against 80% and 90% stated |
| Kubernetes cluster | A change that waits (a lock), and a service with two replicas sharing the load | Lock: every criterion 3 of 3 on fresh deploys, the catalog within 1.3%. Two replicas: browse mean within 1.9%, composed p95 within 2.8% |
| Kubernetes cluster | Composed p95 on Online Boutique pages, fresh deploys | Within 20% on 18 of 18, running 5% to 7% high. Real-cluster p99 is not claimed: the truth’s own noise floor is 46% |
| Public data | 66 real fault-injection cases on Online Boutique | Front-end latency change within 9.1% median, at light load |
| Public data | A real call-center queue at high load | Measured wait matched within 0.94 to 1.06× from 70% utilization to saturation |
| Simulation | Reference 12-service system, 48% to 93% utilization | Entry impact within ±1.8% at every load |
| Simulation | 155 fresh random architectures | Entry error median 5.1%, worst 48% |
| Simulation | p99 on 114 fresh cases, and the planning bound | Entry p99 mean error 4.3%. The ×1.10 bound held on 95% to 100% of fresh rows |
| Simulation | Retry-driven meltdown onset, 26 fresh cases | Within 5% everywhere |
| Simulation | Naive approach: add up the direct costs | Under-estimates by a median 60% to 94%, every time |
| req/s | Busiest ρ | Naive | Rhobound | Simulated |
|---|---|---|---|---|
| 55 | 48% | 6.25 | 13.8 | 13.5 |
| 75 | 66% | 6.25 | 22.3 | 22.8 |
| 90 | 79% | 6.25 | 38.0 | 37.6 |
| 100 | 88% | 6.25 | 67.0 | 64.2 |
| 106 | 93% | 6.25 | 120.7 | 127.5 |
396 automated tests lock the validated results. Accuracy figures are measured on benchmark applications on a Kubernetes test cluster, on public data from real systems and against independent simulation, and each is labelled. Validation on a customer’s production system is the design-partner milestone.
Open source: rhobound-cli
Our open-source command-line tool. It reads OpenTelemetry traces, infers your service graph and predicts mean latency before and after a change, using the queueing-theory approach behind Rhobound. One static Go binary, Apache-2.0.
go install github.com/Sharvil2704/rhobound-cli/cmd/rhobound@latest
rhobound predict ./capture --patch checkout=+3msA missing gate in every delivery pipeline
Tests and security scans are standard gates in the pipeline. Performance is the missing one, and agents writing code at machine speed make it urgent. Rhobound is built to be that gate.
Platform engineering and SRE
Own the latency objectives and the capacity bill. Rhobound tells them what a change does to both before rollout.
Security and vulnerability management
Own the patch rollout. A verdict per service lets safe fixes ship now instead of waiting for a performance review.
Engineering leadership
Carry the trade-off between security speed and reliability. They get an evidence-based answer per change.
- 82%
- Of container users run Kubernetes in production. Rhobound’s customers already run the platform it reads. CNCF Annual Cloud Native Survey 2025
- 41%
- Of observability teams use OpenTelemetry in production, extensively or exclusively. Its traces are all Rhobound needs. Grafana Observability Survey 2025
Why today’s tools don’t close it
| Category | What it answers | What Rhobound adds |
|---|---|---|
| Vulnerability scanners | Which images contain the package | Which services execute it, and what the fix costs |
| CI: build and tests | Does it build and pass | Does it keep the latency objectives at real load |
| Load testing | What happens under synthetic load | A small targeted benchmark, extended to the whole graph at real traffic |
| Canary analysis | Does the new version look fine on a slice | What a full rollout does, including the amplification a canary cannot see |
| Observability and APM | What happened after we shipped | What will happen, from the same telemetry |
| Black-box ML predictors | A number, with no reason | Deterministic and explainable, and it refuses outside its validated range |
How we grow
Design partners
We backtest your last patch rollouts and incidents and show what Rhobound would have predicted. Each partner adds a measured record of predicted against deployed.
The pipeline
A check in CI and a pre-rollout gate for progressive delivery, so every change gets a verdict without anyone asking for one.
The agents’ check
A tool that coding agents call inside their loop. Clear results let cheaper models keep iterating; borderline ones escalate.
What compounds
- A validated, auditable model with a published error record for each kind of architecture.
- An inference engine that rebuilds an architecture from traces alone.
- In-cluster profiling that replays your own traffic, which sees the contention and CPU limits an idle benchmark misses.
- A per-cluster history: every verified deploy calibrates the next prediction and sets its error range, and every design partner adds to the record.
Where this goes
Now
MVP with design partners
- The rhobound command: profile, predict, verify
- Backtests on partners’ real rollouts, with predicted against deployed quoted
- A better first prediction on a new cluster, and the answer-size case: measuring what callers spend decoding a bigger response
- Onboarding without custom hooks
Next
The pipeline
- CI check for GitHub and GitLab, and a rollout gate for Argo Rollouts and Flagger
- Latency budget per call and per service
- Validated change of p99, and burstiness measured from real traffic
- Event-driven consumer lag and autoscaling
Then
The platform
- MCP server so coding agents call Rhobound inside their loop
- Versioned architecture store with every change and its measured impact
- Signed performance attestations next to image provenance
- Admission policy: no rollout without a passing verdict
Integrations and their status
What you give it
- OpenTelemetry tracesThe main input: OTLP/JSON from the Collector’s file exporterAvailable
- kubectlDeployments, services and pods, for profiling and placementAvailable
- Zipkin and B3 tracesRead as calls, alongside OTLPAvailable
- Per-pod CPU countersOptional: rhobound cpu-sample reads the kubelet’s counters into the captureAvailable
- Kubernetes and DockerWhere the new image is profiled: in the cluster, or on one machineAvailable
- Istio metricsTraffic, call counts and per-edge latencyAvailable
- CycloneDX SBOMsBase-image lineage; some tools need a small adapterPreview
- Google Benchmark, bpftrace, OpenSLOCost per call, per-request service time, objectivesAvailable
- CloudFront logsCDN edge trafficAvailable
- Linkerd, Envoy, Parca, PyroscopeMore meshes and continuous profilesRoadmap
Upstream: find and fix
- Trivy, Grype, OSVWhich CVE, which package, which functionRoadmap
- EPSS, CISA KEVExploit likelihood for ship-or-defer decisionsRoadmap
- CopaceticPatched images to checkRoadmap
- Renovate, DependabotEvery update PR gets a verdictRoadmap
- OpenVEXExport reachability as VEX statementsVision
Downstream: ship
- GitHub Actions, GitLab CIThe verdict as a check or PR commentRoadmap
- Argo Rollouts, FlaggerPre-rollout gate on the verdictRoadmap
- k6, wrk2, GoReplayReplay and service-level A/BRoadmap
- Model Context ProtocolTools coding agents call: verify a change, fetch a latency budgetRoadmap
- Sigstore, OPA, KyvernoSigned verdicts and admission policyVision
Status refers to Rhobound’s integration, not to the project itself.
Team
Sharvil Bhatt
Founder
Researches software supply chain security, secure systems architecture, and the trade-off between security and performance in real systems. First author in IEEE Open Journal of Vehicular Technology, reviewer for IEEE Transactions on Vehicular Technology, and speaker at KubeCon India and OpenSSF Korea.
Profile
Questions
Is it validated?
On Kubernetes benchmark applications, where entry latency after a real change was predicted from traces alone within 1.1% on a 10-service deployment and the product and cart pages of Online Boutique within 1.1%; on public data from real systems; and against independent simulation. A customer’s production system is the design-partner milestone, and we say so wherever a number appears.
Does it predict p99?
Yes. It predicts p50 to p99 before and after each change, and gives a planning bound: the modelled p99 after the change × 1.10, which held on 95% to 100% of fresh simulated cases. On real clusters we claim p95, within 20% on 18 of 18 fresh deploys. Real-cluster p99 is not claimed because the truth’s own noise floor is 46%. The change of p99 itself is not validated, so plan against the bound.
What do we have to change?
Nothing in your code. Rhobound reads standard OpenTelemetry or Zipkin traces, every request or a head-sampled share, plus kubectl output to profile the changed image. Optionally it takes per-pod CPU counters and a sample request body for routes that read one.
Where does our data go?
Rhobound is a command-line tool that runs in your environment on your traces and your cluster. Nothing has to leave it.
Is it machine learning?
No. The core is closed-form queueing mathematics. It is deterministic and explainable, every number traces to an input, and it refuses to answer outside the range it has been validated for.
Does it replace load testing and canaries?
It replaces most full-scale load testing for a change, and complements canaries. A canary measures a change’s direct cost well but misses the amplification that appears only at full rollout. Rhobound predicts that part.
What does it support?
Synchronous request-response microservices: parallel fan-outs, shared databases and caches, replicas with different routing policies, conditional calls, CDN edges, network hops, third-party APIs, timeouts and retries. Also serverless functions. Event-driven consumer lag is on the roadmap.
Become a design partner
We’ll backtest your last few patch rollouts and incidents and show what Rhobound would have predicted. We’re looking for teams running containerized microservices on Kubernetes, with shared base images and OpenTelemetry or a service mesh.