Skip to content

Add Opus and DTLS-SRTP/ICE-lite support for Meta WhatsApp calling - #842

Open
mab3321 wants to merge 1 commit into
livekit:mainfrom
mab3321:feature/meta-whatsapp-dtls-srtp
Open

mab3321 wants to merge 1 commit into
livekit:mainfrom
mab3321:feature/meta-whatsapp-dtls-srtp

Conversation

@mab3321

@mab3321 mab3321 commented Sep 10, 2026

Copy link
Copy Markdown

Summary

Adds opt-in Opus and inbound WebRTC-style DTLS-SRTP support, including ICE-lite interoperability required by Meta WhatsApp Business Calling.

Changes

  • add an opt-in Opus codec with configurable bitrate, complexity, FEC, and expected packet loss
  • parse DTLS fingerprints, setup roles, RTCP mux, and ICE credentials/candidates from inbound SDP
  • generate DTLS-SRTP/ICE answer attributes while keeping legacy RTP and SDES-SRTP paths unchanged
  • establish the controlling ICE connection for ICE-lite peers before DTLS
  • derive SRTP/SRTCP keys from the negotiated DTLS profile
  • close ICE, mux, SRTP, SRTCP, and DTLS transports without blocking call teardown
  • add unit coverage for Opus negotiation and Meta-style DTLS/ICE SDP
  • document the new opt-in configuration

Configuration

enable_opus: true
dtls_srtp:
  enabled: true
  handshake_timeout: 10s

Both features default to disabled.

Validation

  • Docker image builds against the current upstream main
  • go test ./pkg/sip -count=1 passes
  • verified real inbound Meta WhatsApp calls with bidirectional Opus audio over ICE/DTLS-SRTP
  • verified remote hangup completes promptly after transport teardown
  • verified an ordinary Twilio PSTN call continues through the existing RTP path with zero packet loss and sub-millisecond SIP media processing

@mab3321
mab3321 requested a review from a team as a code owner September 10, 2026 11:12
@CLAassistant

CLAassistant commented Sep 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 27.94457% with 312 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.69%. Comparing base (0460b40) to head (c5caffb).
⚠️ Report is 368 commits behind head on main.

Files with missing lines Patch % Lines
pkg/sip/dtls_srtp.go 0.00% 225 Missing ⚠️
pkg/sip/dtls_sdp.go 65.32% 24 Missing and 19 partials ⚠️
pkg/sip/media_codecs_opus.go 48.14% 14 Missing ⚠️
pkg/sip/service.go 20.00% 10 Missing and 2 partials ⚠️
pkg/sip/media_port.go 43.75% 5 Missing and 4 partials ⚠️
pkg/sip/media_pipeline.go 0.00% 2 Missing and 2 partials ⚠️
pkg/sip/inbound.go 50.00% 2 Missing and 1 partial ⚠️
pkg/config/config.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #842      +/-   ##
==========================================
+ Coverage   65.25%   65.69%   +0.44%     
==========================================
  Files          51       46       -5     
  Lines        6588     8938    +2350     
==========================================
+ Hits         4299     5872    +1573     
- Misses       1915     2514     +599     
- Partials      374      552     +178     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mab3321
mab3321 force-pushed the feature/meta-whatsapp-dtls-srtp branch from c5caffb to fc373f0 Compare September 14, 2026 12:10
@mab3321

mab3321 commented Sep 14, 2026

Copy link
Copy Markdown
Author

Implemented the four requested review fixes in fc373f0:

  • DTLS/ICE re-INVITE: renegotiation now includes fingerprint, setup role, ICE credentials, and all candidates. Unchanged session refreshes reuse the active transport SDP parameters; changed transport state rebuilds the transport.
  • Cancelable teardown: the session-scoped context is created synchronously and canceled first by Close(). ICE gathering/checks, DTLS handshake, publication, readiness, and startup waits derive from it. Resource publication and cleanup are race-safe, including Close racing ICE publication, DTLS completion, SRTP creation, or readiness signaling.
  • Multiple ICE candidates: all supported component-1 UDP/IPv4 candidates are retained with metadata, deduplicated, and added before connectivity checks.
  • Opus settings: the pinned media-sdk has no encoder configuration API, so unsupported non-default bitrate, complexity, FEC, and packet-loss settings now fail explicitly instead of being silently ignored.

Compatibility is covered by tests proving ordinary RTP/AVP and SDES-SRTP do not enter DTLS/ICE state. No transcoding was introduced.

Server-side validation completed: focused DTLS/ICE/Opus regressions, existing re-INVITE and late-offer tests, race-sensitive DTLS teardown tests, legacy RTP/SDES isolation under the race detector, and the full ./pkg/sip package. The final full package run passed.

Operational diagnosis: the affected call had bidirectional DTLS-SRTP media until inbound RTP stopped, followed by a media timeout and a five-minute teardown delay. The re-INVITE transport reuse and cancel-safe teardown changes directly cover those failure modes.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

4 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

Comment thread pkg/sip/dtls_sdp.go
Comment on lines +155 to +156
if ufrag, hasUfrag := mediaAttribute(m, "ice-ufrag"); hasUfrag {
pwd, hasPwd := mediaAttribute(m, "ice-pwd")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Session-level ICE credentials bypass ICE

Session-level ICE credentials make parseDTLSOffer treat the offer as non-ICE. DTLS targets the media address directly, so ICE-dependent calls lose media.

Learn more

ICE credentials can appear at session level and be overridden at media level. This parser only looks for media-level ice-ufrag and ice-pwd, unlike its fingerprint and setup handling. A conforming offer with session-level credentials therefore produces out.ice == nil. The transport then skips connectICE and starts DTLS against the SDP media address, which does not work when media requires ICE nomination.

Example: An offer contains session-level a=ice-ufrag:meta1 and a=ice-pwd:secret, plus a media-level candidate. The answer omits ICE attributes and the service sends DTLS directly; the expected behavior is an ICE-controlled connection followed by DTLS.

Recommended fix: Resolve each ICE credential from the media attributes first, then fall back to the session attributes. Reject offers where only one resolved credential exists, and parse media candidates whenever resolved credentials are present.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread pkg/sip/dtls_srtp.go
Comment on lines +457 to +459
func (m *dtlsMux) Close() error {
m.once.Do(func() { close(m.closed); _ = m.dtls.Close(); _ = m.srtp.Close(); _ = m.srtcp.Close() })
return nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Rebuilt DTLS transports retain stale readers

A non-ICE transport rebuild leaves readLoop blocked on the reusable UDP socket. The stale reader competes with the replacement mux and discards its packets.

Learn more

A DTLS mux owns a goroutine blocked in conn.Read. Closing only its endpoint channels does not interrupt that read or wait for the goroutine. A changed fingerprint, setup role, or DTLS mode causes configure to close the old pipeline and immediately build another mux over the same udpConn. Both read loops then consume the same UDP socket, but the old loop routes packets to closed endpoints and drops them.

Example: A direct DTLS call receives a re-INVITE with a new fingerprint. The old mux remains blocked while the new pipeline starts. Subsequent DTLS or SRTP datagrams can be consumed and discarded by the old mux instead of reaching the new handshake.

Recommended fix: Give dtlsMux.Close an ownership-aware way to interrupt conn.Read, and wait for readLoop to exit before reusing the socket. For the durable udpConn, use its soft-close/deadline mechanism and reopen it only after the old mux has stopped; do not close the underlying OS socket during rebuilds.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants