Skip to content

Fantom's shouldCollectCoverage answers true on every second call for the same benchmark: FANTOM_BENCHMARK_FILENAME_RE has the g flag #60052

Open

Description

@YevheniiKotyrlo

Description

private/react-native-fantom/runner/coverageUtils.js declares FANTOM_BENCHMARK_FILENAME_RE = /[Bb]enchmark-itest\./g (L16), and shouldCollectCoverage calls .test() on it (L23). With the g flag, .test() starts from the lastIndex of the previous match, so asked three times for the same benchmark path with collectCoverage: true, shouldCollectCoverage answers [false, true, false].

runner.js calls shouldCollectCoverage once per config, inside for (const testConfig of testConfigs) (L300, L387-L391), and passes the answer to the bundle's collectCoverage (L402), LLVM_PROFILE_FILE (L438) and the tester's enableCoverage (L444). So in a --coverage run, every other config of a benchmark with several configs runs with coverage. getFantomTestConfigs.js declares the same pattern without the flag (L75, #56423).

Steps to reproduce

  1. In a checkout of main (6144835) after yarn install, put coverageBenchmark-test.js from the reproducer below in private/react-native-fantom/runner/__tests__/ and run yarn jest private/react-native-fantom/runner/__tests__/coverageBenchmark-test.js; or run node repro.cjs <path to the checkout> from the reproducer, which copies that test in, runs it, removes it, and exits 1 while the defect shows.
const globalConfig = {collectCoverage: true};

// The runner asks once per config of a test, always with the same path.
function askThreeTimes(testPath) {
  return [1, 2, 3].map(() =>
    shouldCollectCoverage(testPath, '', globalConfig),
  );
}

Expected: askThreeTimes('View-benchmark-itest.js') is [false, false, false], and askThreeTimes('View-itest.js') stays [true, true, true].
Actual: askThreeTimes('View-benchmark-itest.js') is [false, true, false].

React Native Version

main at 6144835 (nightly)

Affected Platforms

Build - Windows

Output of npx @react-native-community/cli info

Not applicable: this is Fantom's runner in React Native's own repository. The reproducer's run:
react-native 61448351cc2 (2026-11-16), Windows_NT 10.0.26300, node v26.12.1

Stacktrace or Logs

react-native 61448351cc2 (2026-11-16), Windows_NT 10.0.26300, node v26.12.1
private/react-native-fantom/runner/coverageUtils.js: const FANTOM_BENCHMARK_FILENAME_RE = /[Bb]enchmark-itest\./g;
copied coverageBenchmark-test.js to private/react-native-fantom/runner/__tests__/
$ yarn jest private/react-native-fantom/runner/__tests__/coverageBenchmark-test.js
removed private/react-native-fantom/runner/__tests__/coverageBenchmark-test.js
jest exited 1
  failed: never collects coverage for a benchmark, however often it is asked
  passed: collects coverage for a test that is not a benchmark
shouldCollectCoverage('View-benchmark-itest.js', '', {collectCoverage: true}), asked 3 times: [false, true, false]
shouldCollectCoverage('View-itest.js', '', {collectCoverage: true}), asked 3 times: [true, true, true]
SUMMARY: shouldCollectCoverage answers [false, true, false] for the same benchmark path asked 3 times with collectCoverage: true; expected [false, false, false]

MANDATORY Reproducer

https://gist.github.com/YevheniiKotyrlo/4bda8e7c8af665c0639274ee15918482 — node repro.cjs <path to a react-native checkout> exits 1 while a benchmark gets true on any of the three calls, and 0 once it gets false every time and the other test true every time.

Screenshots and Videos

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions