feat(data-pipeline): add span events FFI by lloeki · Pull Request #2301 · DataDog/libdatadog

lloeki · 2026-07-30T11:02:37Z

AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated.

What does this PR do?

Adds owned span-event construction and atomic attachment through the data-pipeline FFI. It supports typed scalar attributes, homogeneous typed arrays, empty arrays, event ordering, and UTF-8 validation.

Motivation

The Rust trace model supports span events, but native exporter consumers cannot populate them. This fills the span-event portion of the payload parity work tracked by APMSP-3219.

Additional Notes

Stack position: L3, current PR: DataDog/libdatadog PR 2301.

Complete expected review and merge order for one libdatadog release:

Level	libdatadog PR	Matching dd-trace-rb consumer
L0	DataDog/libdatadog PR 2302	DataDog/dd-trace-rb PR 6131
L1	DataDog/libdatadog PR 2303	DataDog/dd-trace-rb PR 6130
L2	DataDog/libdatadog PR 2305	DataDog/dd-trace-rb PR 6129
L3 (current)	DataDog/libdatadog PR 2301	DataDog/dd-trace-rb PR 6134
L4	DataDog/libdatadog PR 2304	DataDog/dd-trace-rb PR 6132
L5	DataDog/libdatadog PR 2300	DataDog/dd-trace-rb PR 6133

L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. L4 and L5 remain drafts while their prerequisite APIs are reviewed and rebased; their matching consumer PRs are also drafts pending the single libdatadog/libdatadog-rb release.

L0 tracks APMSP-3830; L1-L5 implement the payload work tracked by APMSP-3219.

How to test the change?

Prior local validation completed successfully:

nix develop --command cargo test -p libdd-data-pipeline-ffi passed 58 tests, including scalar values, homogeneous typed arrays, empty arrays, ordering, ownership, and atomic failure coverage.
nix develop --command cargo clippy -p libdd-data-pipeline-ffi --all-targets -- -D warnings passed.
nix develop .#nightly --command cargo fmt --all -- --check passed.
nix develop --command env LIBDATADOG_VENDOR_OVERRIDE=/tmp/opencode/native-span-events-ruby bundle exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/send_traces_spec.rb spec/datadog/tracing/transport/native/transport_spec.rb spec/datadog/tracing/transport/traces_spec.rb spec/datadog/tracing/span_event_spec.rb spec/datadog/tracing/span_operation_spec.rb passed 123 native and producer examples, including native wire output and legacy JSON fallback.
Targeted Standard and Steep checks for the native transport and span-event paths passed.

Expose an owned span event builder through the C API. Native tracers can set string, boolean, integer, and double attributes, including homogeneous arrays, before consuming the event into a span. Copy string inputs and preserve event insertion order. Invalid or null handles return errors without attaching a partial event.

Verify that every typed array setter accepts an empty array and records it as an array value rather than dropping the attribute. This preserves the caller's type information even when no elements are present. Also verify that invalid UTF-8 in scalar and array string values returns `InvalidInput` without partially mutating the detached event.

github-actions · 2026-07-30T11:04:44Z

📚 Documentation Check Results

⚠️ 1346 documentation warning(s) found

📦 `libdd-data-pipeline-ffi` - 1346 warning(s)

Updated: 2026-07-30 12:05:26 UTC | Commit: 1361bac | missing-docs job results

github-actions · 2026-07-30T11:06:02Z

🔒 Cargo Deny Results

⚠️ 3 issue(s) found, showing only errors (advisories, bans, sources)

📦 `libdd-data-pipeline-ffi` - 3 error(s)

Show output

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:78:1
   │
78 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   │   └── libdd-data-pipeline-ffi v38.0.0
                     │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │   │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── (dev) libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-common-ffi v38.0.0
                     │   │   └── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:228:1
    │
228 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   │   └── libdd-data-pipeline-ffi v38.0.0
      │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
      │   │   │   ├── libdd-dogstatsd-client v4.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── (dev) libdd-data-pipeline-ffi v38.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-common-ffi v38.0.0
      │   │   └── libdd-data-pipeline-ffi v38.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── libdd-data-pipeline-ffi v38.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:298:1
    │
298 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0
                  └── libdd-data-pipeline-ffi v38.0.0

advisories FAILED, bans ok, sources ok

Updated: 2026-07-30 12:06:56 UTC | Commit: 1361bac | dependency-check job results

pr-commenter · 2026-07-30T11:11:17Z

Benchmarks

Comparison

Candidate

Candidate benchmark details

Baseline

Baseline benchmark details

datadog-datadog-prod-us1 · 2026-07-30T11:16:22Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 83.69%
• Overall Coverage: 74.84% (+0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 127a5d1 | Docs | Datadog PR Page | Give us feedback!}

dd-octo-sts · 2026-07-30T11:37:56Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	8.01 MB	8.01 MB	0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	87.20 MB	87.34 MB	+.16% (+151.41 KB) 🔍

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	98.45 MB	98.60 MB	+.15% (+153.19 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.78 MB	10.78 MB	+.03% (+4.23 KB) 🔍

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	26.06 MB	26.10 MB	+.15% (+42.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	89.18 KB	92.42 KB	+3.63% (+3.24 KB) ⚠️
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	188.08 MB	188.43 MB	+.18% (+360.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	979.11 MB	980.11 MB	+.10% (+1022.90 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.47 MB	8.48 MB	+.14% (+13.00 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	89.18 KB	92.42 KB	+3.63% (+3.24 KB) ⚠️
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	25.05 MB	25.08 MB	+.12% (+32.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	49.83 MB	49.90 MB	+.13% (+69.38 KB) 🔍

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	22.62 MB	22.66 MB	+.16% (+39.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	90.58 KB	93.88 KB	+3.63% (+3.29 KB) ⚠️
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	192.55 MB	192.96 MB	+.21% (+416.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	970.38 MB	971.37 MB	+.10% (+1017.03 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.54 MB	6.55 MB	+.14% (+10.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	90.58 KB	93.88 KB	+3.63% (+3.29 KB) ⚠️
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	26.91 MB	26.94 MB	+.14% (+40.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	47.43 MB	47.49 MB	+.14% (+68.20 KB) 🔍

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	77.70 MB	77.83 MB	+.16% (+134.40 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.92 MB	8.93 MB	+.13% (+12.00 KB) 🔍

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	93.26 MB	93.39 MB	+.14% (+135.44 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.85 MB	10.86 MB	+.10% (+11.17 KB) 🔍

lloeki · 2026-07-30T11:49:01Z

Draft dd-trace-rb example consumer: DataDog/dd-trace-rb#6134.

lloeki added 2 commits July 29, 2026 18:35

lloeki requested a review from a team as a code owner July 30, 2026 11:02

lloeki added the AI Generated PR largely written by AI tools label Jul 30, 2026

github-actions Bot added the data-pipeline label Jul 30, 2026

lloeki mentioned this pull request Jul 30, 2026

Add native span event export DataDog/dd-trace-rb#6134

Draft

lloeki force-pushed the lloeki/native-span-events branch from 344d10b to 127a5d1 Compare July 30, 2026 12:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(data-pipeline): add span events FFI - #2301

feat(data-pipeline): add span events FFI#2301
lloeki wants to merge 2 commits into
mainfrom
lloeki/native-span-events

lloeki commented Jul 30, 2026 •

edited by atlassian Bot

Loading

Uh oh!

github-actions Bot commented Jul 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 30, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 30, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 Bot commented Jul 30, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

Uh oh!

dd-octo-sts Bot commented Jul 30, 2026 •

edited

Loading

Uh oh!

lloeki commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

lloeki commented Jul 30, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Additional Notes

How to test the change?

Uh oh!

github-actions Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Documentation Check Results

📦 libdd-data-pipeline-ffi - 1346 warning(s)

Uh oh!

github-actions Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Cargo Deny Results

📦 libdd-data-pipeline-ffi - 3 error(s)

Uh oh!

pr-commenter Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Comparison

Candidate

Baseline

Uh oh!

datadog-datadog-prod-us1 Bot commented Jul 30, 2026 • edited by datadog-prod-us1-4 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

lloeki commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lloeki commented Jul 30, 2026 •

edited by atlassian Bot

Loading

github-actions Bot commented Jul 30, 2026 •

edited

Loading

📦 `libdd-data-pipeline-ffi` - 1346 warning(s)

github-actions Bot commented Jul 30, 2026 •

edited

Loading

📦 `libdd-data-pipeline-ffi` - 3 error(s)

pr-commenter Bot commented Jul 30, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Jul 30, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

dd-octo-sts Bot commented Jul 30, 2026 •

edited

Loading