fix: mark xfail test_groundedness_e2e_string_documents

[akihikokuroda]

Pull Request

Issue

Description

Work around test failure. Mark xfail a test failing caused by citation response differences between CPU and GPU.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

• Component
• Requirement
• Sampling Strategy
• Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

[planetf1]

One suggestion I can't anchor to the diff: the sibling test test_groundedness_e2e_complex_response just above uses assert isinstance(result.as_bool(), bool) precisely because grounding verdicts are hardware-dependent. If this test is primarily checking that string documents are accepted as input rather than pinning a specific verdict, the same looser assertion would make it deterministic without needing xfail at all.

[planetf1]

Wondering if this swap is intentional — @require_gpu(min_vram_gb=8) and @pytest.mark.xfail do quite different things. require_gpu is a skip gate; on a CPU-only runner it returns skipif(True, ...) so the test never executes, which is what all eight sibling tests in this file do. xfail is a result modifier — the test still runs, it just tolerates a failure. After this change this becomes the only test in the file that will load ibm-granite/granite-4.0-micro and run inference on a CPU-only runner.

Suggest keeping both. Worth filing an issue for the CPU/GPU divergence and linking it in the reason too — bare xfails without an issue reference tend to get missed in test-health sweeps:

Suggested change

	@pytest.mark.xfail(reason="CPU/GPU response differences")
	@require_gpu(min_vram_gb=8)
	@pytest.mark.xfail(strict=False, reason="CPU/GPU citation response differences — see #NNNN")

[akihikokuroda]

This requirement processes the citation response in detail. It can not loosen the condition. I restored require_gpu as suggested.

[planetf1]

Just to clarify — strict=False doesn't change what the test does. It only affects what happens on an unexpected pass, and since there's no global xfail_strict = true in pyproject.toml the behaviour is the same either way. No action needed.

[planetf1]

For reference: #1291 tracks this — it covers test_groundedness_e2e_string_documents alongside context-attribution and requirement_check_alora. PR #1292 fixes the underlying groundedness parser bug, but some GPU non-determinism remains so the xfail is still the right call for now.

[planetf1]

On the as_bool() suggestion: #1291 confirms this test is asserting a specific grounded verdict, not just that string documents are accepted as input, so the looser assertion wouldn't work here. The xfail is the right approach. That suggestion is withdrawn.

[planetf1]

The critical fix is in — @require_gpu restored, skip gate is intact. Comments addressed above.