Use SciMLTesting v1.2 (folder-based run_tests)#89
Merged
ChrisRackauckas merged 1 commit intoJun 14, 2026
Conversation
Replace the hand-written GROUP-dispatch ladder in test/runtests.jl with a single SciMLTesting.run_tests() call in folder-discovery mode. - runtests.jl becomes `using SciMLTesting; run_tests()`. - Core: the inline @safetestset blocks move verbatim to top-level test/core_tests.jl (Core = top-level test/*.jl, main test env). - QA: test/qa/qa_tests.jl stays in its sub-env (test/qa/Project.toml), discovered + activated automatically (runs only for GROUP=QA, never under All — matches the old `if GROUP == "QA"` branch). - Root Project.toml: add SciMLTesting (+compat "1") and SafeTestsets to test deps; drop Pkg (only the removed activate_qa_env used it). - test/qa/Project.toml: add SciMLTesting + SafeTestsets (qa_tests.jl runs under a @safetestset wrapper in the qa sub-env). - test_groups.toml unchanged. Behavior-preserving: GROUP=Core/All run the same Core tests, GROUP=QA runs qa_tests.jl in the qa env. Verified end-to-end on Julia 1.11 (Core 105036/105036 pass, QA 46/46 pass). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChrisRackauckas-Claude
force-pushed
the
sciml-testing-rollout
branch
from
ad40ee2 to
83edb93
Compare
ChrisRackauckas-Claude
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the hand-written
GROUP-dispatch ladder intest/runtests.jlwith a singleSciMLTesting.run_tests()call in v1.2 folder-discovery mode.What changed
test/runtests.jl→ exactly:Folder layout (folder name matches the
test_groups.tomlkey, case-insensitively):test/*.jl. The inline@safetestsetblocks that previously lived in theif GROUP == "All" || GROUP == "Core"branch move verbatim (dedented) intotest/core_tests.jl. Core runs in the main test env (no activation). Runs forGROUP=CoreandGROUP=All, as before.test/qa/with its existingtest/qa/Project.toml.qa_tests.jlis discovered and run in its sub-env (activate_group_envdevelops the repo root + instantiates). A declared sub-env means QA runs only forGROUP=QAand never underAll— matching the originalif GROUP == "QA"branch exactly.Project.toml:SciMLTestingto[extras]/[targets].test/[compat] = "1"and keepSafeTestsets/StableRNGs/Test.Pkgfrom[extras]/[targets].test/[compat]— it was only used by the removedactivate_qa_envhelper (grepoftest/confirms no other use); the v1.2 harness owns all Pkg ops.test/qa/Project.toml:SciMLTesting(compat"1") andSafeTestsets(compat"0.0.1, 0.1") —qa_tests.jlruns under a@safetestsetwrapper in the qa sub-env.test/test_groups.tomlis unchanged.Verification (local, Julia 1.11)
TOML.parsefileon everyProject.toml+test_groups.toml;Meta.parseallonruntests.jl+core_tests.jl+qa_tests.jl(no:errornodes).core_tests.jlis byte-identical to the dedented original Core block (17@safetestsetblocks).core_tests.jl, QA →qa/qa_tests.jl(sub-env), All → Core only.run_tests():GROUP=Core→ 105036/105036 pass;GROUP=QA→ 46/46 pass (exercises the qa sub-env activation path).Behavior-preserving: the exact set of tests run under each
GROUPvalue is unchanged.Ignore until reviewed by @ChrisRackauckas.