Skip to content

tcp: implement the RFC 4015 response to spurious loss recovery - #14458

Closed
davidbell217 wants to merge 1 commit into
google:masterfrom
davidbell217:rfc4015-spurious-recovery-response
Closed

davidbell217 wants to merge 1 commit into
google:masterfrom
davidbell217:rfc4015-spurious-recovery-response

Conversation

@davidbell217

@davidbell217 davidbell217 commented Aug 27, 2026 •

Copy link
Copy Markdown

Fixes #14102 (cc @nybidari).

Implements the RFC 4015 congestion control response to spurious recovery, per the notes on the issue:

  • Capture: pipe_prev is captured at all five recovery entry points (detectLoss, RACK reorder timer, TLP loss, RTO expiry) before HandleLossDetected()/HandleRTOExpired() cut Ssthresh — at enterRecovery() it's already too late. SndCwnd stands in for FlightSize (both packet-counted; Outstanding is already decimated by the ACKs that trigger detection). A never-reduced Ssthresh (InitialSsthresh) is not restored: it holds no pipe estimate, and restoring it re-created the collapse on our fixture.
  • Response: on exit from a recovery flagged spurious, Ssthresh = pipe_prev (only if that raises it) and SndCwnd = FlightSize + min(bytes_acked, IW). cwnd is not restored directly — a direct restore burst-collapsed 24/24 runs, as warned in the issue.
  • Both exit paths, one site: every recovery (fast/SACK via leaveRecovery, pure RTO directly) ends at the Open transition in handleRcvdSegment, so the response applies exactly there — after the ACK-removal loop, where Outstanding is the true FlightSize and the allowed send is bounded by IW. The capture is consumed on use, so it applies at most once per episode.
  • State: one new plain savable int on sender; nothing added to TCPSenderState.
  • Tests: TestDetectSpuriousRecoveryWithRTO and TestSACKDetectSpuriousRecoveryWithDupACK now drive the connection out of recovery and assert Ssthresh/SndCwnd are restored. Both fail without the fix (Ssthresh = 2, want = 10, one per exit path).

One enabling detection fix: detectSpuriousRecovery now ignores ACKs with no Timestamps option (!SendTSOk || TSEcr == 0). Previously TSEcr = 0 compared as < RetransmitTS and marked a genuine recovery spurious — harmless while only a metric consumed the flag, but TestSACKRecovery (whose crafted ACKs carry no TS option) fails once a response does.

Benchmark, on the fixture from #14102 (195 ms RTT, ±15 ms jitter, 3% of packets delayed +30 ms, no loss; 24 runs/arm; #14092's fix applied in both arms; pinned = run exceeds 15 s):

pinned median
detection only 4/24 10.8–11.0 s
+ RFC 4015 response 0/24 ~3 s

Episodes still fire either way; the response stops them compounding. A clean fixture (3% reorder, no jitter) shows no regression.

Scope: RFC 4015 specifies the response for spurious timeouts; applying it to fast/SACK recovery too matches netstack's detection, which deliberately doesn't differentiate. Step (8) (SND.NXT <- SND.MAX), steps (10)–(11) (RTT/RTO restoration), and the ECE guard (netstack has no ECN congestion response) are out of scope.

@davidbell217
davidbell217 force-pushed the rfc4015-spurious-recovery-response branch from be53c9a to 190e04d Compare August 27, 2026 20:56

@kerumeto kerumeto left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the nice description and clean code. Just a few small comments to address from my side.

Thanks for catching the missing Timestamps check!

Comment thread pkg/tcpip/transport/tcp/snd.go Resolved
Comment thread pkg/tcpip/transport/tcp/snd.go Resolved
Comment thread pkg/tcpip/transport/tcp/snd.go Resolved
@davidbell217
davidbell217 force-pushed the rfc4015-spurious-recovery-response branch from 190e04d to 88cc357 Compare September 5, 2026 06:07
RFC 3522 Eifel detection sets sender.spuriousRecovery and the
SpuriousRecovery metrics, but nothing consumed the signal: leaveRecovery
unconditionally deflated SndCwnd to Ssthresh, so a recovery the sender
itself proved spurious still cost a permanent multiplicative decrease,
and under sustained path jitter the sender ratchets its throughput down
episode after episode.

Capture pipe_prev (RFC 4015 Section 3 step (0)) at every loss-recovery
entry point before the congestion controller reduces Ssthresh, and on
exit from a recovery detected spurious restore Ssthresh to it and
slow-start back from FlightSize + min(bytes_acked, IW) (step (9))
instead of keeping the deflation. The response is applied at the single
point every recovery exits through — the Open transition in
handleRcvdSegment — after the ACK-removal loop, when Outstanding is the
true FlightSize, so the send it allows is bounded by IW. The capture is
consumed on use, so the response applies at most once per episode.

detectSpuriousRecovery now ignores ACKs carrying no Timestamps option:
a TSEcr of zero previously compared as smaller than RetransmitTS and
marked a genuine recovery spurious, which was harmless while the flag
only fed a metric but wrong once a response consumes it.

Extend the two spurious-recovery e2e tests to drive the connection out
of recovery on each exit path and assert Ssthresh/SndCwnd are restored;
both extensions fail without the fix.

Fixes google#14102

Assisted-by: Claude Code
@davidbell217
davidbell217 force-pushed the rfc4015-spurious-recovery-response branch from 88cc357 to 1af4ae8 Compare September 5, 2026 06:28

@kerumeto kerumeto left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, @nybidari might have a few comments

Comment thread pkg/tcpip/transport/tcp/snd.go Resolved
adios-404 added a commit to adios-404/netstack-rack-repro that referenced this pull request Oct 31, 2026
Adds results/2026-10-31/reorder: 20 MB fetches with Tailscale's netstack settings on gVisor master, alone and with google/gvisor#14101, google/gvisor#14458 or both, under the seeded reordering mode, with RACK off and on and with and without client timestamps.
@adios-404

Copy link
Copy Markdown
Contributor

@nybidari @kerumeto, here are the benchmark numbers requested on #14102. With RACK off, Tailscale's next default (tailscale/tailscale#22092), reordering costs netstack far more than kernel TCP (tailscale/tailscale#21651). In a two-stack harness with Tailscale's settings (CUBIC, SACK, host GSO, RACK off), a 1 ms, 1 Gbit/s path and 1% of data packets held 200 µs, the median 20 MB fetch on master aa767e4 (16 interleaved runs per build):

build 20 MB fetch vs master (benchstat)
master 1.30 s
+ #14101 1.31 s ~ (p = 0.79)
+ #14458 0.258 s −80.2% (p < 0.001)
+ both 0.254 s −80.5% (p < 0.001)

Without reordering, all four take 0.17 s. With RACK on, #14458 takes it from 1.14 to 0.195 s. Without client timestamps, no gain (1.33 vs 1.32 s, p = 0.89). Flags and raw output: https://github.com/adios-404/netstack-rack-repro/tree/68b789fcf/results/2026-10-31/reorder.

Only tcp_sack_test.go conflicts, since #16601 made TestDetectSpuriousRecoveryWithRTO table-driven. Rebased, all 11 TCP test packages pass (go test). Its timestamp check in detectSpuriousRecovery is now redundant with #16601's.

@davidbell217, will you rebase it, or are you happy for me to open a rebased copy with you kept as the author?

@tamird tamird left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing detector can miss a delayed original here. With RACK off, the first advancing ACK echoed the original packet’s timestamp (4294967290), which preceded the first repair (994). Because the timestamp wrapped, this branch didn’t run and ssthresh stayed at 2 packets instead of returning to 10.

The same thing happened without wrap after a second timeout cleared retransmitTS. Changing only the comparison then restored 10 for a genuine second repair. Keeping the first repair’s timestamp for the whole recovery episode and comparing in serial order distinguished the original from both repairs in all eight packet traces. A separate detector fix would cover both cases for this response.

Review by Codex on behalf of @tamird.

tamird added a commit to tamird/gvisor that referenced this pull request Nov 4, 2026
Reuse the existing Eifel GNU coordinator, remote platform and packet
owner. Compare the exact credited downstream sender against a control
that omits only its cancellation block, with one shared three-case test.
Keep original native exits, fresh remote TestResults, exact named cases
and bounded packet observations; publish only the typed receipt.

The existing generated-launcher shebang correction is identical in both
arms. This fork-only qualification does not claim Linux TCP equivalence
or change the scope of google#14458.
@davidbell217

Copy link
Copy Markdown
Author

Superseded by #17022. Thanks @adios-404 for carrying this forward. Please continue review there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants