test(dotnet): enable ot.th/ot.rv tracestate sampling scenarios [dotnet@milan.garnier/ot.th]

[MilanGarnier]

Enables tests/test_otel_tracestate_sampling.py entries (APMAPI-2171)

dd-trace-dotnet's implementation is on milan.garnier/ot.th:

[datadog-prod-us1-4]

 

⚠️ Warnings

🚦 7 Pipeline jobs failed

Testing the test | System Tests (dotnet, prod) / End-to-end #1 / poc 1     

🔧 Fix in code. This looks caused by changes in this PR.

Test run failed. AssertionError: 'th' should be erased when a force-keep overrides an inherited drop decision at tests/test_otel_tracestate_sampling.py:540

🧪 4 Tests failed · 🎯 related to the job error above

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_01.test_emit_ot_on_probability_decision[poc] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:bf63561b7ca482e3;t.dm:-5;t.ksr:0.01;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_01 object at 0x7f2d47e6b560>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_5.test_emit_ot_on_probability_decision[poc] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:adcf6ff78000f2e1;t.dm:-3;t.ksr:0.5;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_5 object at 0x7fb1415dbce0>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

🎯 tests.test_otel_tracestate_sampling.Test_ForceKeepClearsTh.test_force_keep_overrides_inherited_drop_decision[poc] from system_tests_suite  

AssertionError: th should be erased when a force-keep overrides an inherited drop decision
assert 'th' not in {'rv': '65cd67504a538e', 'th': 'e6666666666668'}

self = <tests.test_otel_tracestate_sampling.Test_ForceKeepClearsTh object at 0x7fc5bce145c0>

    def test_force_keep_overrides_inherited_drop_decision(self):
        """Upstream already decided to drop (a full th/rv pair); the local force-keep still clears th but forwards the inherited rv unchanged."""
        assert self.dropped_request.status_code == 200
        ot = _parse_ot(_outbound_tracestate(self.dropped_request))
    
...

View all failed tests

Testing the test | System Tests (dotnet, prod) / End-to-end #1 / uds 1     

🔧 Fix in code. This looks caused by changes in this PR.

Test run failed. AssertionError: 'th' should be erased when a force-keep overrides an inherited drop decision at tests/test_otel_tracestate_sampling.py:540.

🧪 3 Tests failed · 🎯 related to the job error above

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_01.test_emit_ot_on_probability_decision[uds] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:716ce1daf9159cf3;t.dm:-5;t.ksr:0.01;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_01 object at 0x7fd583fd11c0>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_5.test_emit_ot_on_probability_decision[uds] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:93cb3fc57f39487d;t.dm:-3;t.ksr:0.5;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_5 object at 0x7f13a0d421b0>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

🎯 tests.test_otel_tracestate_sampling.Test_ForceKeepClearsTh.test_force_keep_overrides_inherited_drop_decision[uds] from system_tests_suite  

AssertionError: th should be erased when a force-keep overrides an inherited drop decision
assert 'th' not in {'rv': '65cd67504a538e', 'th': 'e6666666666668'}

self = <tests.test_otel_tracestate_sampling.Test_ForceKeepClearsTh object at 0x7f61d1224ad0>

    def test_force_keep_overrides_inherited_drop_decision(self):
        """Upstream already decided to drop (a full th/rv pair); the local force-keep still clears th but forwards the inherited rv unchanged."""
        assert self.dropped_request.status_code == 200
        ot = _parse_ot(_outbound_tracestate(self.dropped_request))
    
...

Testing the test | System Tests (dotnet, prod) / End-to-end #2 / poc 2     

🔧 Fix in code. This looks caused by changes in this PR.

Test run failed due to assertion errors related to 'ot' tracestate member not being emitted properly under various probability decisions.

🧪 6 Tests failed · 🎯 related to the job error above

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_1.test_emit_ot_on_probability_decision[poc] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:846ffa8ef37e78bf;t.dm:-3;t.ksr:0.1;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_1 object at 0x7fc150dcc560>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_2.test_emit_ot_on_probability_decision[poc] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:5a3ec218f553181c;t.dm:-3;t.ksr:0.2;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_2 object at 0x7f1be0e04170>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

🎯 tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_99.test_emit_ot_on_probability_decision[poc] from system_tests_suite  

AssertionError: trace_id=1: no ot= tracestate member emitted on a probability decision
assert 'ot' in Tracestate(dd=s:2;p:4afd827eb687b748;t.dm:-3;t.ksr:0.99;t.ts:02)

self = <tests.test_otel_tracestate_sampling.Test_EmitOtOnProbabilityDecision_Rate0_99 object at 0x7f7bb050c0e0>

    def test_emit_ot_on_probability_decision(self):
        for req, trace_id, expected_rv, expected_th, expected_sampled in self.requests:
            assert req.status_code == 200, f"trace_id={trace_id}: /make_distant_call failed"
    
            tracestate = _outbound_tracestate(req)
...

View all failed tests

View all 7 failed jobs.

📋 Copy prompt for your agent

CI on my pull request is failing. Help me find and fix the root cause of each failing job below — they were flagged as caused by changes in this PR, so focus on the diff. For each job, explain the failure and propose a fix.

Branch: milan.garnier/enable-otel-th-rv-dotnet-manifest

Testing the test | System Tests (dotnet, prod) / End-to-end #1 / poc 1
Commit: 7a4dce9c7ac22aa67b2e2c641761c818eab088c7
Error (code / test):
1 failed test. AssertionError: 'th' should be erased when a force-keep overrides an inherited drop decision at tests/test_otel_tracestate_sampling.py:540
CI job: https://github.com/DataDog/system-tests/actions/runs/31598782758/job/94384096724

Testing the test | System Tests (dotnet, prod) / End-to-end #1 / uds 1
Commit: 7a4dce9c7ac22aa67b2e2c641761c818eab088c7
Error (code / test):
1 failed test. AssertionError: 'th' should be erased when a force-keep overrides an inherited drop decision at tests/test_otel_tracestate_sampling.py:540.
CI job: https://github.com/DataDog/system-tests/actions/runs/31598782758/job/94384096650

Testing the test | System Tests (dotnet, prod) / End-to-end #2 / poc 2
Commit: 7a4dce9c7ac22aa67b2e2c641761c818eab088c7
Error (code / test):
4 failed tests due to assertion errors related to 'ot' tracestate member not being emitted properly under various probability decisions.
CI job: https://github.com/DataDog/system-tests/actions/runs/31598782758/job/94384096658

Plus 1 more failing job not shown here.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 0 passed on retry

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ac90d12 | Docs | Datadog PR Page | Give us feedback!}

[github-actions]

CODEOWNERS have been resolved as:

manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet