Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
5556159
refactor(rust sdk): part 1
daflyinbed Jul 2, 2026
b76401e
fix(rust sdk): align grpc codec/consumer with server key spellings an…
daflyinbed Jul 2, 2026
859b138
feat(rust sdk): add self-contained e2e test suite with bundled docker…
daflyinbed Jul 2, 2026
ff29daa
fix(rust sdk): make rocketmq compose profile boot on selinux/low-mem …
daflyinbed Jul 2, 2026
c5477be
test(rust sdk): switch e2e suite to the rocketmq compose profile
daflyinbed Jul 2, 2026
2cfb148
fix(rust sdk): create e2e topics as form-encoded, not JSON
daflyinbed Jul 2, 2026
82a8338
fix(rust sdk): use per-call gRPC timeouts, not a channel-wide cap
daflyinbed Jul 2, 2026
bc53f55
fix(rust sdk): relocate rocketmq jars so the storage plugin's shadow …
daflyinbed Jul 3, 2026
c43e3f6
fix(rust sdk): make request/reply replies correlate with the original…
daflyinbed Jul 3, 2026
3b756eb
fix(rust sdk): add graceful shutdown and axum-style stream driver
daflyinbed Jul 3, 2026
60639f3
Merge pull request #2 from daflyinbed/feature/rust-sdk-http
daflyinbed Jul 4, 2026
2f7b023
feat(rust-sdk): tcp (#3)
daflyinbed Jul 6, 2026
fed181f
feat(rust-sdk): TCP auto-reconnect and CloudEvents support
daflyinbed Jul 7, 2026
ca9d23a
fix(rust sdk): route one-way publish to publishOneWay RPC and check s…
daflyinbed Jul 7, 2026
8226009
chore: add asf license header and cleanup
Jul 8, 2026
6ef8f3a
refactor: more idiomatic
Jul 8, 2026
b2ce983
refactor: interval
Jul 8, 2026
d5ddcbb
refactor(rust-sdk): redesign consumer API — separate RPC from connect…
Jul 8, 2026
eb30206
test(rust-sdk): use eventmesh:jdk11-test image and harden e2e suite
daflyinbed Jul 8, 2026
50cca57
fix(rust-sdk): set datacontenttype=application/cloudevents+json for T…
daflyinbed Jul 9, 2026
89e3458
feat(rust-sdk): concurrent dispatch in gRPC stream receive loop
Jul 13, 2026
baf6c38
fix(rust-sdk): prevent silent hang of gRPC subscribe_stream on curren…
daflyinbed Jul 13, 2026
30e5e78
fix(rust-sdk): address all 8 release-blocking review findings
daflyinbed Jul 13, 2026
1581bb3
fix(rust-sdk): strict is_success, TCP CloudEvent validation, unsubscr…
daflyinbed Jul 15, 2026
74aab8c
fix(rust-sdk): prevent TCP ghost writes, slow-consumer drops, driver-…
daflyinbed Jul 15, 2026
72ff7e4
fix(rust-sdk): redact secrets in Debug, strict HTTP retCode, (topic,u…
daflyinbed Jul 15, 2026
5299217
fix(rust-sdk): bound gRPC resubscribe waits
daflyinbed Jul 15, 2026
1ac1963
feat(sdk-rust): add OpenMessaging and CloudEvents APIs
daflyinbed Jul 15, 2026
e4ae43c
fix(rust-sdk): preserve request reply metadata
daflyinbed Jul 15, 2026
ed4e25b
feat(rust): add catalog and workflow clients
daflyinbed Jul 15, 2026
3c3b6db
refactor: rust
daflyinbed Jul 16, 2026
712b816
refactor: remove v2
daflyinbed Jul 16, 2026
0554c3b
Refactor Rust SDK transports to preserve message dialects
daflyinbed Jul 16, 2026
3d80800
Strengthen Rust E2E publish assertions
daflyinbed Jul 16, 2026
3e83af7
Add Java/Rust interop E2E test wiring
daflyinbed Jul 16, 2026
c03a3e2
Add gRPC webhook consumers and per-call request timeouts
daflyinbed Jul 16, 2026
5b2eaf0
Polish Rust SDK APIs and release checks
daflyinbed Jul 16, 2026
a92b906
Fix request reply and TCP E2E readiness
daflyinbed Jul 16, 2026
5c6c475
Refactor event mesh components
daflyinbed Jul 17, 2026
9f9c5f4
Expand Rust SDK integration coverage
daflyinbed Jul 17, 2026
1ac95e4
Document Rust SDK API, features, and verification workflow
daflyinbed Jul 17, 2026
6371128
Make generated gRPC modules crate-private
daflyinbed Jul 22, 2026
ac2e233
update
daflyinbed Jul 22, 2026
67e940e
Remove obsolete files and code
daflyinbed Jul 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions eventmesh-sdks/eventmesh-sdk-java/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,10 @@ dependencies {
testImplementation "org.mockito:mockito-inline"
testImplementation "org.mockito:mockito-junit-jupiter"
}

tasks.register('interopPeerClasspath') {
dependsOn testClasses
doLast {
println("INTEROP_PEER_CLASSPATH=${sourceSets.test.runtimeClasspath.asPath}")
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.eventmesh.client.interop;

import org.apache.eventmesh.client.grpc.config.EventMeshGrpcClientConfig;
import org.apache.eventmesh.client.grpc.consumer.EventMeshGrpcConsumer;
import org.apache.eventmesh.client.grpc.consumer.ReceiveMsgHook;
import org.apache.eventmesh.client.grpc.producer.EventMeshGrpcProducer;
import org.apache.eventmesh.common.EventMeshMessage;
import org.apache.eventmesh.common.enums.EventMeshProtocolType;
import org.apache.eventmesh.common.protocol.SubscriptionItem;
import org.apache.eventmesh.common.protocol.SubscriptionMode;
import org.apache.eventmesh.common.protocol.SubscriptionType;

import java.util.Collections;
import java.util.Optional;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

/** Small host-process peer used by the Rust SDK interop E2E suite. */
public final class GrpcInteropPeer {

private GrpcInteropPeer() {
}

public static void main(String[] args) throws Exception {
if (args.length < 3) {
throw new IllegalArgumentException("usage: publish|consume host topic [content]");
}
String operation = args[0];
String host = args[1];
String topic = args[2];
EventMeshGrpcClientConfig config = EventMeshGrpcClientConfig.builder()
.serverAddr(host).serverPort(10205).env("env").idc("idc").sys("java-interop")
.producerGroup("java-interop-producer").consumerGroup("java-interop-consumer")
.userName("eventmesh").password("eventmesh").build();

if ("publish".equals(operation)) {
try (EventMeshGrpcProducer producer = new EventMeshGrpcProducer(config)) {
producer.publish(EventMeshMessage.builder().topic(topic).content(args[3]).build());
}
System.out.println("INTEROP_PUBLISHED");
return;
}
if (!"consume".equals(operation)) {
throw new IllegalArgumentException("unknown operation: " + operation);
}

CountDownLatch delivered = new CountDownLatch(1);
EventMeshGrpcConsumer consumer = new EventMeshGrpcConsumer(config);
consumer.registerListener(new ReceiveMsgHook<EventMeshMessage>() {
@Override
public Optional<EventMeshMessage> handle(EventMeshMessage message) {
System.out.println("INTEROP_RECEIVED=" + message.getTopic() + "\t" + message.getContent());
delivered.countDown();
return Optional.empty();
}

@Override
public EventMeshProtocolType getProtocolType() {
return EventMeshProtocolType.EVENT_MESH_MESSAGE;
}
});
consumer.init();
consumer.subscribe(Collections.singletonList(new SubscriptionItem(topic, SubscriptionMode.CLUSTERING, SubscriptionType.ASYNC)));
if (!delivered.await(30, TimeUnit.SECONDS)) {
throw new IllegalStateException("timed out waiting for message");
}
consumer.close();
}
}
175 changes: 175 additions & 0 deletions eventmesh-sdks/eventmesh-sdk-rust/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
# AGENTS.md — EventMesh Rust SDK

Cargo crate `eventmesh` (`edition = "2021"`, **MSRV 1.86.0**). Speaks the
EventMesh **gRPC**, **HTTP**, and **TCP** protocols. gRPC wire format is
CloudEvents-protobuf; the simple `EventMeshMessage` model is converted at the
gRPC boundary by `codec.rs`. HTTP wire format is
`application/x-www-form-urlencoded` with JSON payloads in the `content` field,
mirroring the Java SDK. TCP uses the EventMesh binary wire protocol
(length-prefixed frames with `"EventMesh"` magic) and supports automatic
reconnect with exponential backoff plus CloudEvents JSON interop.

This crate is **not** part of the Gradle build and has **no GitHub Actions
CI**. All verification is local. The parent repo `AGENTS.md` covers the
docker-compose profiles and runtime ports; this file covers the Rust workflow.

## Build prerequisite: `protoc`

`build.rs` invokes `tonic-build`, which shells out to the `protoc` compiler at
**build time**. `prost-build` finds `protoc` automatically when it is on
`PATH`; only set the `PROTOC` env var to point at a binary that is **not** on
`PATH`. (`brew install protobuf`, `apt-get install protobuf-compiler`, etc. all
put `protoc` on `PATH`, so no prefix is needed.)

```bash
cargo build --features full
```

## Feature matrix

| Feature | Notes |
|---|---|
| `grpc` | gRPC transport — publish, batch, request-reply, stream/webhook subscribe. |
| `http` | HTTP transport — `HttpProducer`, managed `HttpConsumer`, external `WebhookRegistration`, and webhook codec helpers. Uses reqwest + axum. |
| `tcp` | TCP transport — `TcpProducer`, `TcpConsumer`, native binary wire protocol. Auto-reconnect with exponential backoff. CloudEvents interop behind `cloud_events`. |
| `cloud_events` | Native `cloudevents::Event` interop (gRPC, HTTP, and TCP). |
| `tls` | TLS on the gRPC channel. |
| `full` | `grpc` + `http` + `tcp` + `cloud_events` + `tls`. Use this for clippy/test so every code path compiles. |
| `e2e` | Gates the live-server integration suite (`tests/e2e/`). A plain `cargo test` never touches Docker. |

## Verification (the order the README mandates)

```bash
cargo fmt --check
cargo clippy --no-default-features --lib -- -D warnings
cargo clippy --features full --all-targets -- -D warnings
cargo test --features full
cargo doc --features full --no-deps
```

Clippy runs with **`-D warnings`** (warnings are errors here). There is no
`rustfmt.toml`/`clippy.toml` — defaults apply. To run a single test binary:
`cargo test --features full --test codec_test`.

## Documentation ownership

- `README.md` is the user-facing overview: installation, feature selection,
shared behavior, and links to the right entry point.
- Crate and public-item rustdoc in `src/` is the API reference. Update it with
every public API or observable behavior change; include feature and lifecycle
requirements where relevant.
- `examples/README.md` classifies runnable examples and their exact feature
flags. Keep it aligned with `[[example]]` entries in `Cargo.toml`.
- `CONTRIBUTING.md` owns contributor prerequisites and verification; this file
owns agent-facing architecture and repository workflow.

## Generated proto code (do not hand-edit)

`build.rs` compiles `proto/eventmesh-{service,cloudevents}.proto` into `OUT_DIR`
at build time, generating **client stubs only** (`build_server(false)`), with
`--experimental_allow_proto3_optional`. The generated tree is pulled in via
`src/proto_gen.rs` → `tonic::include_proto!("...")`; it is **not** checked in.
Add convenience aliases in `proto_gen.rs`, not in the generated module.

## Architecture notes

- `src/lib.rs` is `#![deny(unsafe_code)]` — no `unsafe` anywhere.
- `src/transport/mod.rs` defines `Publisher` as an **async-fn-in-trait**
(Rust 1.86). It is therefore **not object-safe** — use the concrete
`GrpcProducer` / `TcpProducer` / `HttpProducer` directly, never `dyn`.
The subscribe side has **no trait** — each transport exposes its own
consumer type (`GrpcStreamConsumer`, `GrpcWebhookConsumer`, `TcpConsumer`,
`HttpConsumer`) with transport-specific `subscribe` / `unsubscribe` /
`subscribe_webhook` methods, a background receive loop (where applicable),
and `wait_for_shutdown()` for clean exit.
- `src/transport/grpc/codec.rs` is the `EventMeshMessage` ↔ CloudEvents-protobuf
bridge for the gRPC transport.
- `src/transport/http/codec.rs` is the `EventMeshMessage` ↔ form-urlencoded +
JSON bridge for the HTTP transport. The wire format is
`application/x-www-form-urlencoded` (not JSON bodies); payloads go in the
`content` form field as JSON strings.
- `src/config/grpc.rs` — `GrpcClientConfig` + fluent builder.
- `src/config/http.rs` — `HttpClientConfig` + fluent builder. Accepts
semicolon/comma-separated `host:port[:weight]` server lists; uses the shared
`LoadBalanceSelector` from `common/loadbalance.rs`.
- `src/transport/http/webhook.rs` — **internal** axum handler (`WebhookHandler`
+ `WebhookState`) used only by `WebhookServer`. Not part of the public API.
- `src/transport/http/server.rs` — built-in `WebhookServer` (axum). Managed
consumers bind it before registration and run it in a background task.
- `src/transport/tcp/connection.rs` — TCP engine: `establish()` does TCP +
HELLO handshake; `run()` is an outer reconnect loop that calls `io_loop()`
(the inner select! over read/write/heartbeat). When `ReconnectConfig::enabled`
(default `true`), the outer loop re-establishes the connection with
exponential backoff after I/O errors. `take_reconnect_rx()` delivers a
notification on each successful reconnect so consumers can replay
subscriptions.
- `src/transport/tcp/message.rs` — package builders for `EventMeshMessage`
(`build_message_package`) and CloudEvents (`build_cloud_event_package`,
behind `cloud_events`). CloudEvents bodies use `protocoltype=cloudevents` and
are serialized as `application/cloudevents+json` raw bytes (matching the Java
runtime's codec path).
- `src/config/tcp.rs` — `TcpClientConfig` + `ReconnectConfig` + fluent builders.
- Public `TcpConfig` keeps Java-compatible timeout classes separate: TCP
connect (1s default), protocol control (20s), business request/response
(20s), heartbeat interval (30s), and reconnect backoff. Rust heartbeats and
GOODBYE remain fire-and-forget.
- `src/common/` — `ProtocolKey`, status codes, constants, `LoadBalanceSelector`
shared across transports.

### HTTP transport specifics

- The public HTTP consumer binds and runs an axum callback server in the
background, then owns registration, heartbeat, and shutdown as one lifecycle.
The lower-level `WebhookRegistration` is client-only (like the Java SDK) for
applications that host their own endpoint. Two receiving modes remain:
1. **Managed consumer** — `HttpClient::consumer` binds before registration,
preventing connection-refused startup races.
2. **Custom endpoint** — the user hosts any HTTP server (axum, actix, hyper,
…) and decodes pushes with the **public codec helpers** in
`src/transport/http/codec.rs`: `parse_push_body`,
`PushMessageRequestBody::to_event_mesh_message`, and `WebhookReply`
(the `{"retCode": 0}` ack). See the `http_consumer_custom` example.
- There is intentionally **no public `WebhookHandler`/`WebhookLayer`/`WebhookState`**
type. The old "register the SDK's handler on your own Router" mode was
removed; users who embed the webhook in their own app write the handler
themselves on top of the codec utilities.
- Runtime routing: the EventMesh HTTP server has two routing mechanisms —
path-based (new-style handlers, checked first by URI prefix match) and
code-header-based (old-style, checked by the `code` header when no path
matches). Because this SDK sends `application/x-www-form-urlencoded` bodies,
**all** operations (publish, subscribe, unsubscribe, heartbeat) use
code-header-based routing via the root path `/` (`uri::ROOT`). Posting to a
path-based handler (e.g. `/eventmesh/subscribe/local`) with a form body breaks
body decoding: the `topic` form field is parsed as a string and cannot be
deserialized as `List<SubscriptionItem>`.
- Heartbeat interval: 30s (mirrors the Java SDK), spawned as a background
tokio task tied to a `CancellationToken`.

## End-to-end tests (`tests/e2e/`)

Run with: `cargo test --features e2e`.

- The harness (`tests/e2e/runtime.rs`) **auto-starts the `rocketmq` docker-compose
profile** and tears it down at process exit. Set `EVENTMESH_E2E_EXTERNAL=1` to
point at a server you started yourself.
- If neither Docker nor a reachable server is found, tests **fail by default**.
Set `EVENTMESH_E2E_ALLOW_SKIP=1` only for an intentional local skip; release
verification must not set it.
- Each test generates a unique topic + consumer group (monotonic counter + nanos
timestamp). gRPC and HTTP cases may run in parallel; TCP cases take a shared
async test lock and run serially because the runtime's route refresh and
RocketMQ rebalance cycles are shared even when topics are distinct.
- **Standalone (in-memory) broker limitations:** a topic must be created via the
admin API **and** a consumer subscribed *before* any publish, and it does not
implement request/reply. The harness warms topics automatically. Because the
suite strictly verifies every advertised operation, use a runtime profile that
supports request/reply when running the complete release suite.
- Topic creation hits the admin API at `POST /topic`, which expects
**`application/x-www-form-urlencoded`** (not JSON).

## Conventions

- **Apache license header is required on every `.rs` file** — copy the header
block from a neighbor (e.g. `build.rs`). Consistent with the rest of the repo.
- Builders use the `Option<T> field + fluent setter + consuming build()` idiom
(see `GrpcClientConfigBuilder`); mirror it for new config types.
65 changes: 65 additions & 0 deletions eventmesh-sdks/eventmesh-sdk-rust/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Contributing to the EventMesh Rust SDK

This guide covers `eventmesh-sdks/eventmesh-sdk-rust`. Repository-wide Apache EventMesh contribution requirements still apply.

## Prerequisites

- Rust 1.86.0 or newer (the crate MSRV)
- `protoc` on `PATH` for builds that enable `grpc`, `full`, or `e2e`
- Docker and a compatible EventMesh runtime only for live end-to-end tests

`build.rs` invokes `tonic-build`; generated protobuf code lives in `OUT_DIR` and must not be edited or committed.

## Local checks

Run these before submitting a Rust SDK change:

```bash
cargo fmt --check
cargo clippy --no-default-features --lib -- -D warnings
cargo clippy --features full --all-targets -- -D warnings
cargo test --features full
cargo doc --features full --no-deps
```

Use `cargo test --features full --test codec_test` for the codec test binary. Examples are feature-gated in `Cargo.toml`; compile the one you changed with its documented `cargo run --example ... --features ...` command, or compile all supported paths with `cargo check --examples --features full`.

## End-to-end tests

The e2e suite is opt-in so a normal `cargo test` never requires Docker:

```bash
cargo test --features e2e
```

The harness starts the `rocketmq` docker-compose profile unless `EVENTMESH_E2E_EXTERNAL=1` points it at an already running runtime. An absent runtime is a failure by default. `EVENTMESH_E2E_ALLOW_SKIP=1` is only for an intentional local skip and must not be used for release verification.

The destructive TCP reconnect test is ignored by default because it restarts the compose-managed runtime:

```bash
cargo test --features e2e --test e2e \
tcp_reconnect_replays_subscription_after_runtime_restart -- --ignored
```

## Documentation responsibilities

Keep each document in its intended layer.

| Change | Update |
| --- | --- |
| Installation, feature choice, or common behavior | `README.md` |
| Public type, method, feature-gated API, or behavior | rustdoc in `src/` |
| Runnable workflow or transport use | the matching file in `examples/` and `examples/README.md` |
| Validation, e2e, or contributor workflow | this file and `AGENTS.md` when agent guidance changes |

Public rustdoc should state feature requirements, ownership/lifecycle rules, and error or acknowledgement behavior where relevant. Prefer an executable doctest when it has no runtime dependency; otherwise mark the snippet `rust,ignore` and point users to a runnable example.

## Code conventions

- Add the Apache license header to every new `.rs` file.
- Keep `#![deny(unsafe_code)]` intact; unsafe Rust is not permitted.
- Mirror the established consuming builder style for configuration additions.
- Keep transport wire formats behind the public v2 API. Do not expose private legacy adapters merely to reuse an implementation detail.
- `Publisher` uses async functions in traits and is not object-safe; use concrete protocol producers instead of `dyn Publisher`.

See [AGENTS.md](AGENTS.md) for architecture details and precise e2e behavior.
Loading