Skip to content

Feature/tls keylog - #34

Draft
heshaoqiong-tuya wants to merge 4 commits into
masterfrom
feature/tls-keylog
Draft

Feature/tls keylog#34
heshaoqiong-tuya wants to merge 4 commits into
masterfrom
feature/tls-keylog

Conversation

@heshaoqiong-tuya

Copy link
Copy Markdown
Collaborator

feat(common): opt-in TLS key log for Wireshark decryption

What & why

All SDK-to-cloud traffic (IoT-DNS, ATOP HTTPS, MQTT, RTC/TAI) runs over TLS, so a
packet capture shows only ciphertext. Debugging "the cloud says my field is wrong" or
"the downstream frame won't parse" needs the plaintext application-layer data.

This PR adds an opt-in, off-by-default TLS key log in common/tls.c: during the
handshake it exports the session secrets in NSS SSLKEYLOGFILE format, which Wireshark
reads to decrypt the same capture. Unlike terminating TLS at a proxy, this changes
neither the device's connection target nor its certificate. Because the facility lives in
common/tls.c, enabling it once covers every channel (MQTT / ATOP / IoT-DNS / RTC-TAI).

API (common/tls.h)

  • tls_keylog_open_file(path) / tls_keylog_close_file() — write key-log lines to a
    file. Compiled in by default on POSIX and ESP-IDF; elsewhere opt in with
    -DTLS_KEYLOG_FILE_SINK=1.
  • tls_set_keylog_handler(fn, ctx) — route each line to your own sink (UART, log server,
    ring buffer).
  • Process-wide and off by default: enable it once before the first TLS connection; no
    per-connection setup.
  • A runtime debug switch only; enabling it emits one LOG_WARN
    ([tls] key logging ENABLED) — the only sign in production logs that a debug switch
    shipped.
  • The export side needs mbedTLS 3.x; on 2.x the sink installs but nothing is exported.

Design notes

  • Lock-free, snapshotted per connection: tls_connect() snapshots the current sink
    into the tls_t on the connecting thread, and the export callback reads only its own
    copy. Every f_export_keys call site is inside the handshake, and tls.c never enables
    renegotiation / KeyUpdate export, so the "read at handshake time" invariant tls.h
    documents holds — swap the sink only while no tls_connect() is in flight.
  • Fixed, self-checked line bound: TLS_KEYLOG_LINE_SIZE is derived from
    MAX_LABEL(40) and MAX_SECRET(64) (= 236), with a _Static_assert on the longest
    label and a runtime check against the real secret_len before every write — an
    over-long line is dropped, never truncated into a silently undecryptable log.
  • The file is the secret, hardened accordingly: on POSIX, open(2) with
    O_NOFOLLOW|O_CLOEXEC and mode 0600 from creation (no fopen+fchmod 0666 window); the
    stream is unbuffered (a buffer holding secrets can't be wiped, and an unflushed line
    won't decrypt), and a stream that can't be made unbuffered is refused; each line is
    fsync'ed so a mid-session reset still leaves a decryptable capture; the stack line
    copy is wiped with mbedtls_platform_zeroize; a write failure is logged once
    (LOG_ERROR).
  • Portability: pal_t has no file interface, so the file sink uses libc
    fopen/fputs, compiled in only where a writable filesystem is the norm (this
    "deliberate gap" is documented in AGENTS.md rule 5) — a bare newlib port without
    _open/_close stubs still links tls.o.

Output format

Which lines appear depends on the negotiated version: TLS 1.2 (what iot-client pins)
emits one CLIENT_RANDOM line per connection; TLS 1.3 (what the RTC/TAI server may
negotiate) emits the handshake and application traffic secrets.

Tests

Two new tests in modules/iot-client/test/mqtt_test.c (need a local test broker + CA;
skipped when no certificate is loaded):

  • test_keylog_handler_captures_client_random: one TLS 1.2 handshake yields exactly one
    well-formed CLIENT_RANDOM line (48-byte master secret → 96 hex chars); asserts the
    key logging ENABLED warning; clearing the sink stops further lines.
  • test_keylog_file_sink: the file is written with exactly one line, a second open is
    refused, replacing the sink leaves the file closable twice (idempotent), and the mode is
    0600 on POSIX; the file is unlink'ed on pass or fail so no keys are left behind.

@heshaoqiong-tuya
heshaoqiong-tuya marked this pull request as draft September 7, 2026 10:24
heshaoqiong-tuya and others added 4 commits September 7, 2026 18:58
The Unreleased entry said (#32), the number the fix was expected to get. It
merged as #31 (squash commit 3556669) and #32 does not exist yet, so the one
link from the one-line entry back to its reasoning pointed at nothing.

Also adds the trailing newline AGENTS.md was missing. Whitespace only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Export TLS session secrets in NSS SSLKEYLOGFILE format so a packet capture
of the SDK's cloud traffic can be decrypted in Wireshark. Opt-in and off by
default; enabling it covers every channel (MQTT, ATOP, IoT-DNS, RTC/TAI) at
once, since the facility lives in common/tls.c.

- tls_keylog_open_file() / tls_keylog_close_file() write lines to a file
  (compiled in on POSIX and ESP-IDF, or with TLS_KEYLOG_FILE_SINK=1);
  tls_set_keylog_handler() routes them to a custom sink.
- Process-wide, snapshotted per connection: tls_connect() reads the sink on
  the connecting thread, so the export callback is lock-free and every call
  site is inside the handshake.
- The line bound is derived and self-checked; over-long lines are dropped,
  never truncated into an undecryptable log.
- The file is the secret: 0600 from creation, O_NOFOLLOW|O_CLOEXEC, unbuffered
  and fsync'ed per line, stack copy zeroized, write failure logged once.
- Runtime debug switch only; enabling it logs a LOG_WARN. Export needs
  mbedTLS 3.x. pal_t has no file interface, so the file sink uses libc I/O —
  a deliberate gap recorded in AGENTS.md rule 5.

Docs: docs-site/docs/guides/tls-keylog.md covers the Wireshark setup.
Tests: mqtt_test.c gains handler-capture and file-sink cases (skipped
without a CA cert).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SDK_VERSION agentic-kit_0.4.0 -> agentic-kit_0.5.0-dev, what
`tools/bump_version next --minor` produces. v0.4.0 is tagged at 238045b.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant