A production-oriented traffic-control platform built with Java 21 and Spring Boot, using Redis-backed leases, atomic coordination, and bounded capacity allocation to enforce shared global limits across multiple application instances.
Overview · Architecture · Coordination · Quick Start · Performance · Observability · AWS · Production Readiness
Distributed traffic control is straightforward on one process and much harder once the same capacity must be shared by several independent application instances.
A naive per-node limiter can accidentally multiply a global limit:
Global Capacity = 100
┌──────────────┐
│ Gateway 1 │ ──► 100
├──────────────┤
│ Gateway 2 │ ──► 100
├──────────────┤
│ Gateway 3 │ ──► 100
└──────────────┘
Total = 300 ❌
This project separates the problem into two layers:
- Local request enforcement — fast, in-process quota checks on each gateway.
- Global capacity coordination — Redis-backed shared capacity managed through bounded leases.
The result is a distributed traffic-control engine where multiple gateways can make local decisions while still respecting a single global capacity boundary.
The interesting engineering problem is not implementing a basic rate limiter. It is handling the distributed-systems concerns around it:
- shared global capacity
- concurrent allocation
- atomic multi-key state changes
- bounded reservations
- lease expiration
- unused-capacity reclamation
- gateway failure and recovery
- low-cardinality observability
- measurable latency and throughput
- Distributed multi-gateway traffic control
- Redis-backed global capacity coordination
- Lease-based capacity allocation
- Atomic Redis Lua scripts for distributed mutations
- Bounded lease size to prevent one node from reserving the entire pool
- Lease acquisition, consumption, renewal, release, expiration, and reclamation
- Protection against concurrent over-allocation
- Token Bucket, Fixed Window, and Sliding Window local algorithms
- Prometheus metrics and Grafana dashboards
- JUnit 5 + Testcontainers integration validation
- Automated GitHub Actions CI with Redis service support
- Docker Compose local deployment
- AWS deployment using ECR, EC2, ALB, ElastiCache Redis, and SSM
- k6 single-node and distributed performance validation
- Controlled gateway failure and recovery validation
flowchart TB
R[Traffic Request] --> E[Traffic Decision Engine]
E --> P[Policy Resolver]
E --> Q[Local Quota Enforcement]
Q -->|Quota available| A[ALLOWED]
Q -->|Quota exhausted| C[Capacity Allocator]
C --> L[Redis Lease Coordinator]
L --> G[(Redis Global Capacity)]
L --> H[(Lease Registry)]
G --> L
H --> L
L --> C
C -->|Lease acquired| A
C -->|No global capacity| D[REJECTED]
Request
│
▼
Resolve Policy
│
├── missing / inactive ───────────────► REJECT
│
▼
Build Quota Key
│
▼
Local Quota Check
│
├── capacity available ───────────────► ALLOW
│
▼
Capacity Allocator
│
▼
Acquire Redis-backed Lease
│
├── acquisition fails ────────────────► REJECT
│
▼
Consume Lease Capacity
│
▼
ALLOW
| Concern | Identity | Store | Purpose |
|---|---|---|---|
| Local quota | policy + subject + resource |
In-memory | Fast request-path enforcement |
| Global capacity | policy + resource |
Redis | Shared distributed budget |
| Lease state | leaseId |
Redis | Capacity ownership and reclamation |
| Lease registry | policy + resource |
Redis sorted set | Expiration tracking |
This separation keeps Redis off the normal local-capacity path while preserving a single globally coordinated capacity boundary when nodes need more quota.
A TrafficSubject identifies the entity generating traffic.
Supported subject types:
USER
API_KEY
IP_ADDRESS
TENANT
A TrafficPolicy defines how traffic is controlled.
Supported policy types:
TOKEN_BUCKET
FIXED_WINDOW
SLIDING_WINDOW
Policies also carry lifecycle state and capacity/refill configuration.
The local quota identity is:
policy + subject + resource
The gateway can therefore make many decisions without contacting Redis.
The distributed capacity identity is:
policy + resource
All gateway instances coordinate against the same shared pool.
A QuotaLease temporarily assigns a bounded portion of the global pool to a gateway.
Global capacity = 100
Gateway 1 → lease 10
Gateway 2 → lease 10
Gateway 3 → lease 10
...
The default deployment caps an individual lease at 10 units.
Redis state is conceptually organized as:
traffic-control:quota:<policy-id>:<resource>
traffic-control:quota:<policy-id>:<resource>:leases
traffic-control:lease:<lease-id>
Several distributed operations update multiple pieces of Redis state together. Performing those mutations as independent commands would introduce race windows.
The coordinator therefore uses Lua scripts for operations such as:
- lease acquisition
- expired-lease reclamation
- lease consumption
- lease renewal
- lease release
This makes the state transition atomic from Redis's point of view.
Conceptually:
Request lease
│
▼
┌─────────────────────┐
│ Redis atomic script │
└──────────┬──────────┘
│
┌──────────┴──────────┐
│ │
▼ ▼
Reclaim expired Check global
leases capacity
│ │
└──────────┬──────────┘
▼
Reserve capacity
│
▼
Create lease
│
▼
Register expiry
Redis key expiration alone is not sufficient when unused lease capacity must be returned to the global pool.
The implementation keeps an explicit lease registry:
Expired lease
│
▼
Find expired lease
│
▼
Read remaining capacity
│
▼
Return unused capacity
│
▼
Delete lease state
│
▼
Remove registry entry
This prevents abandoned reservations from permanently reducing the shared capacity pool.
┌───────────┐
│ Acquire │
└─────┬─────┘
│
▼
┌───────────┐
│ Active │
└─────┬─────┘
│
┌──────────┼──────────┐
│ │ │
▼ ▼ ▼
Consume Renew Release
│ │ │
│ ▼ ▼
│ Active Return unused
│
▼
Remaining = 0
│
▼
Release
Expired lease
│
▼
Read remaining capacity
│
▼
Return to global pool
│
▼
Delete lease + registry entry
A lease is therefore a stateful distributed resource with explicit ownership and lifecycle semantics, not just an integer counter.
| Area | Technology |
|---|---|
| Language | Java 21 |
| Framework | Spring Boot 4.1.1 |
| Build | Maven |
| Runtime | Eclipse Temurin 21 |
| Distributed coordination | Redis 7 |
| Atomic coordination | Redis Lua |
| Containerization | Docker / Docker Compose |
| Metrics | Micrometer |
| Monitoring | Prometheus |
| Dashboards | Grafana |
| Integration testing | JUnit 5 + Testcontainers |
| Load testing | k6 |
| HTTP server | Tomcat 11 |
| AWS runtime | ECR + EC2 + ALB + ElastiCache Redis + SSM |
POST /api/v1/traffic/evaluate
Content-Type: application/jsonExample request:
{
"requestId": "req-001",
"subject": {
"subjectId": "user-123",
"type": "USER"
},
"resource": "/api/orders",
"requestedAt": "2026-09-07T15:20:00Z"
}Example response:
{
"status": "ALLOWED",
"reason": "Request allowed",
"remainingCapacity": 99
}Possible decision status values include:
ALLOWED
REJECTED
Traffic admission is intentionally separate from HTTP transport semantics: a request can receive a valid HTTP response while the traffic-control decision itself is REJECTED.
Install:
- Java 21
- Docker Desktop
- Git
git clone https://github.com/yashdotdev13/distributed-traffic-control.git
cd distributed-traffic-controlLinux/macOS:
./mvnw -B verifyWindows PowerShell:
.\mvnw.cmd -B verifydocker compose build
docker compose up -dCheck the services:
docker compose pshttp://localhost:8081/actuator/health
http://localhost:8082/actuator/health
http://localhost:8083/actuator/health
Expected health state:
{
"status": "UP"
}docker compose downThe application uses environment-driven configuration.
SERVER_PORT=8081
REDIS_HOST=localhost
REDIS_PORT=6379
NODE_ID=local-node
LEASE_DURATION=PT30S
LEASE_CAPACITY=10For distributed deployment, each gateway receives a unique NODE_ID while all gateways share the same Redis-backed global capacity pool.
Prometheus metrics are exposed at:
/actuator/prometheus
Key application metrics:
traffic_control_decision_duration_seconds
traffic_control_decisions_total
traffic_control_lease_allocations_total
traffic_control_quota_exhausted_total
Metrics intentionally avoid high-cardinality identifiers such as request IDs and subject IDs.
The metric model is designed to make the following behaviors observable:
- decision throughput
- allowed vs. rejected traffic
- decision latency
- quota exhaustion
- lease allocation attempts
- successful lease allocation
- failed lease allocation
Prometheus scrapes the gateway instances and Grafana provides the dashboard layer.
The application uses Spring Boot's Micrometer Observation/OpenTelemetry integration to export distributed traces over OTLP, with Jaeger providing the local trace visualization interface.
The instrumentation covers the traffic-decision workflow and Redis-backed lease operations, helping distinguish local quota evaluation from distributed coordination latency.
Key observations include:
traffic.decision— overall traffic-decision operationtraffic.policy.lookup— traffic-policy lookuptraffic.quota.evaluate— local quota evaluationtraffic.lease.allocate— lease allocationtraffic.lease.consume— consumption against an allocated leaseredis.lease.acquire— Redis lease acquisitionredis.lease.consume— Redis lease consumptionredis.lease.renew— Redis lease renewalredis.lease.release— Redis lease release
The local Docker Compose environment includes Jaeger for inspecting traces. Span emission depends on the operation and code path being executed.
See the OpenTelemetry Tracing Guide for configuration, local setup, test requests, and troubleshooting.
The automated suite covers:
- traffic decision evaluation
- policy resolution
- Token Bucket behavior
- Fixed Window behavior
- Sliding Window behavior
- capacity allocation
- lease acquisition
- lease consumption
- lease renewal
- lease release
- lease expiration
- expired-capacity reclamation
- global capacity protection
- concurrent allocation safety
Automated tests: 106
Failures: 0
Errors: 0
Skipped: 0
BUILD SUCCESS
The Redis integration tests run against a real Redis 7 instance supplied by Testcontainers.
Integration tests: 8
Failures: 0
Errors: 0
Skipped: 0
The system has been benchmarked with k6 in both single-node and distributed configurations.
Requests: 372,864
Throughput: 12,399 req/s
Average latency: 0.737 ms
p90 latency: 1.02 ms
p95 latency: 1.41 ms
Maximum latency: 73.85 ms
HTTP failures: 0.00%
Checks passed: 100.00%
Requests: 128,487
Throughput: 4,273 req/s
Average latency: 2.20 ms
p90 latency: 3.66 ms
p95 latency: 5.10 ms
Maximum latency: 257.87 ms
HTTP failures: 0.00%
Checks passed: 100.00%
| Scenario | Throughput | Avg | p90 | p95 | HTTP Failures |
|---|---|---|---|---|---|
| Single gateway | 12,399 req/s | 0.737 ms | 1.02 ms | 1.41 ms | 0.00% |
| 3 gateways | 4,273 req/s | 2.20 ms | 3.66 ms | 5.10 ms | 0.00% |
These are environment-specific benchmark observations, not universal capacity guarantees.
During distributed validation:
Lease allocation attempts
Gateway 1 → 426,788
Gateway 2 → 36,578
Gateway 3 → 36,405
Successful leases → 10
Lease size → 10 units
Total allocated → 100 units
This demonstrates that requests exercised the distributed lease path and contended for the shared global capacity pool.
The current distributed k6 scenario does not guarantee perfectly even request distribution across gateways. Per-gateway request counts should therefore not be treated as a load-balancer fairness benchmark.
A controlled multi-gateway failure test was executed against the deployed topology.
One gateway was stopped while Redis and the other gateways remained available.
gateway-1 → STOPPED
gateway-2 → UP
gateway-3 → UP
Redis → UP
Traffic continued successfully through the surviving gateways.
The failed gateway was restarted:
gateway-1 → UP
gateway-2 → UP
gateway-3 → UP
The recovered gateway returned a healthy state and successfully resumed traffic evaluation.
- gateway-level failure does not require global coordination state to disappear
- surviving nodes can continue serving traffic
- the failed node can rejoin the deployment
- Redis-backed global state remains shared across nodes
The project has also been deployed and validated on AWS using a multi-node topology:
Internet
│
▼
┌─────────────────┐
│ Application ALB │
│ :80 │
└────────┬────────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ EC2 #1 │ │ EC2 #2 │ │ EC2 #3 │
│ Gateway │ │ Gateway │ │ Gateway │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└────────────┼────────────┘
│
▼
┌─────────────────┐
│ ElastiCache │
│ Redis OSS 7.1 │
└─────────────────┘
AWS components used:
- Amazon ECR for the container image
- EC2 for the three gateway nodes
- Application Load Balancer for ingress
- ElastiCache for Redis OSS
- AWS Systems Manager for instance administration
- Security groups for network isolation
The deployment was validated for:
- ALB health checks
- public traffic routing
- shared Redis-backed capacity coordination
- stopping a gateway while the other nodes continue serving requests
- restarting the failed gateway and returning it to service
See the complete deployment procedure in docs/aws-deployment.md.
Important: the AWS environment is a validated deployment baseline, not a claim that every production-hardening control is already enabled.
This repository is intentionally production-oriented in architecture and engineering practices, while the current AWS environment remains a controlled reference deployment.
- explicit separation of local and distributed state
- atomic distributed mutations
- bounded capacity leasing
- explicit lease ownership
- expiration and reclamation
- deterministic integration testing against Redis
- CI validation on every push / pull request
- low-cardinality operational metrics
- containerized deployment
- multi-node runtime validation
- failure and recovery testing
- environment-driven configuration
Before using this exact AWS baseline for a security-sensitive production workload, add and verify:
- HTTPS/TLS at the public ingress
- authenticated and authorized administrative APIs
- secrets management instead of plain sensitive environment values
- Redis TLS/authentication where required
- highly available Redis configuration with automatic failover
- CloudWatch alarms and durable operational alerting
- centralized log retention and auditability
- WAF / edge protection as appropriate
- infrastructure-as-code for repeatable environments
- autoscaling and capacity planning
- disaster-recovery and backup/restore procedures
- security review, dependency scanning, and image vulnerability scanning
This distinction is deliberate: the architecture is designed around production concerns, but a production claim should only be made after the surrounding security, reliability, and operations controls are also in place.
distributed-traffic-control/
│
├── src/
│ ├── main/java/
│ │ └── com/...
│ └── test/java/
│ └── com/...
│
├── load-tests/
│ ├── baseline.js
│ └── distributed.js
│
├── docs/
│ ├── phase-1-product-requirements/
│ ├── phase-2-system-design/
│ ├── aws-deployment.md
│ └── benchmark-and-validation.md
│
├── scripts/
│ └── aws/
│ └── ec2-gateway-user-data.sh
│
├── .github/
│ └── workflows/
│ └── ci.yml
│
├── Dockerfile
├── docker-compose.yml
├── prometheus.yml
├── pom.xml
└── README.md
Local quota enforcement handles the hot path, while Redis coordinates the shared global budget.
Operations that update multiple Redis structures are coordinated through Lua scripts.
Nodes acquire bounded chunks of capacity instead of reserving the entire global pool.
Released and expired leases return unused capacity to the shared pool.
Metrics remain low-cardinality so request-specific identifiers do not become monitoring labels.
Distributed behavior is validated using integration tests, multi-node runtime tests, load tests, and controlled failure scenarios.
Detailed project documentation lives under docs/:
These documents cover requirements, architecture, distributed coordination, failure modes, deployment topology, and measured validation results.
- Traffic subject modeling
- Traffic policy modeling
- Token Bucket
- Fixed Window
- Sliding Window
- Local quota enforcement
- Global capacity management
- Redis-backed lease coordination
- Atomic Redis Lua operations
- Lease renewal
- Lease release
- Lease expiration
- Expired-capacity reclamation
- Concurrent allocation protection
- REST evaluation endpoint
- Docker Compose deployment
- Prometheus metrics
- Grafana monitoring
- k6 benchmarks
- GitHub Actions CI
- AWS ECR image delivery
- AWS EC2 multi-node deployment
- AWS ALB ingress
- AWS ElastiCache Redis integration
- AWS SSM-based administration
- Gateway failure validation
- Gateway recovery validation
- HTTPS/TLS and stronger ingress security
- Authentication and authorization for management APIs
- AWS Secrets Manager / Parameter Store integration
- Highly available Redis with automatic failover
- CloudWatch alarms and centralized operational alerting
- WAF and edge protection
- Infrastructure as Code
- Adaptive / demand-based allocation strategies
- Expanded policy-management APIs
- Persistent policy/configuration storage
- Kubernetes deployment
- Distributed tracing
- Autoscaling and adaptive capacity management
| Area | Status |
|---|---|
| Core traffic-control engine | ✅ Validated |
| Local quota enforcement | ✅ Validated |
| Redis distributed coordination | ✅ Validated |
| Lease lifecycle | ✅ Validated |
| Expired-capacity reclamation | ✅ Validated |
| Concurrent allocation safety | ✅ Validated |
| Docker deployment | ✅ Validated |
| 3-node runtime | ✅ Validated |
| GitHub Actions CI | ✅ Validated |
| Prometheus | ✅ Validated |
| Grafana | ✅ Validated |
| Single-node benchmark | ✅ Validated |
| Distributed benchmark | ✅ Validated |
| Gateway failure handling | ✅ Validated |
| Gateway recovery | ✅ Validated |
| AWS multi-node deployment | ✅ Validated |
| AWS gateway failure/recovery | ✅ Validated |
Yash Chauhan
This repository does not currently declare a finalized open-source license.
Add a LICENSE file and update this section before publishing the project for external reuse.