REX Memory Tooling (Sanitizers, Valgrind, Memcheck)¶
This document is the single source of guidance for using sanitizers and Valgrind memcheck in REX. It covers how to configure, run, and interpret results, plus what to keep in mind when developing or triaging memory issues.
When to use which tool¶
- Sanitizers (ASan/LSan/UBSan): Fast feedback during development, great for catching UAF, OOB, and UB.
- Valgrind memcheck: Slower but precise leak reporting and call stacks; good for leak triage and hard-to-repro bugs.
Use separate build directories for normal, sanitizer, and Valgrind builds to avoid flag conflicts.
REX_ENABLE_UNINITIALIZED_FIELD_TESTS defaults to ON only when CMake finds both executable Valgrind and its development headers; otherwise it defaults to OFF. testUninitializedFields uses Valgrind client requests as definedness probes, which become no-ops outside Valgrind, but compiling the executable still requires those headers. Explicitly selecting the suite with -DREX_ENABLE_UNINITIALIZED_FIELD_TESTS=ON without complete Valgrind support is a hard configuration error. Native source CI installs the dependency and therefore keeps the availability-derived suite enabled. Standard nightly images omit optional Valgrind tooling, so the suite remains disabled there without an architecture switch. Finding Valgrind does not enable CTest MemCheck or its extended timeouts; MemCheck is enabled only by an explicit WITH-VALGRIND setting.
Build and configure¶
Normal build (no memory instrumentation)¶
cmake -S . -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build -j"$(nproc)"
Sanitizer build¶
Sanitizers require libclang-cpp (LLVM 23). Use Debug to keep ROSE_ASSERT active during memory checks. If ROSE_SANITIZERS is empty, the build defaults to address;leak.
cmake -S . -B build-sanitizer -DCMAKE_BUILD_TYPE=Debug \
-DENABLE-SANITIZER=ON -DROSE_SANITIZERS="address;leak;undefined"
cmake --build build-sanitizer -j"$(nproc)"
Valgrind/memcheck build¶
Configure with Valgrind paths so CTest can drive memcheck. Use Debug so assertions stay enabled.
cmake -S . -B build-valgrind -DCMAKE_BUILD_TYPE=Debug -DWITH-VALGRIND=/usr
cmake --build build-valgrind -j"$(nproc)"
If Valgrind lives elsewhere, use: - -DWITH-VALGRIND=/path/to/prefix or - -DWITH-VALGRIND-BIN=/path/to/bin -DWITH-VALGRIND-INCLUDE=/path/to/include
Running tests¶
Standard tests¶
ctest --test-dir build --output-on-failure
Sanitizer-labeled tests¶
ctest --test-dir build-sanitizer -L sanitizer -LE death -j"$(nproc)" --output-on-failure
Valgrind memcheck (recommended subset first)¶
ctest --test-dir build-valgrind -T memcheck -R "<regex>" -LE death -j"$(nproc)" --output-on-failure
Use ctest -N -R "<regex>" to list tests before running memcheck.
LSan and external runtime allocations¶
For the whole-codebase audit, the external-issue policy and evidence register are in audit issue #885 and its attached external-exceptions.md register. Use stock LLVM 23 by default. First correct REX integration through supported APIs and ownership. Narrow external-report suppressions require origin and correctness evidence, an inline register reference, and explicit recheck/removal conditions. Neither external ownership nor an existing suppression justifies hiding a REX defect or a correctness-affecting issue. Dependency forks are not required. Full audit validation runs all eligible tests, beyond the scheduled memory CI selections below.
Some toolchain/runtime components allocate process-lifetime memory that LSan cannot attribute cleanly. To keep sanitizer runs actionable, REX ships scripts/rex-suppressions-for-lsan. Weekly sanitizer CI exports this via LSAN_OPTIONS. If you run tests manually, set: LSAN_OPTIONS=suppressions=<path-to>/scripts/rex-suppressions-for-lsan.
Weekly sanitizer CI scales both timeout layers by four. The ROSE_TEST_TIMEOUT_SCALE environment variable controls REX's shell and Perl test harnesses, while CTest's explicit 6,000-second limit covers direct test executables that never enter those harnesses. Both boundaries remain hard errors; the larger limit changes neither the selected tests nor their failure handling.
The weekly complete Debug suite applies the same policy with a twofold scale: 1,800 seconds for REX harness commands and a 3,000-second CTest limit for direct executables. This retains a finite failure boundary for every test while covering the measured 14-minute outliner cases without depending on a single-digit percentage timing margin.
How memcheck works in REX¶
When configured with Valgrind, CTest uses the build’s memcheck settings: - CTEST_MEMORYCHECK_COMMAND uses the configured wrapper to launch Valgrind. - Options include leak checking, all leak kinds, --error-exitcode=1, and --trace-children=yes. - Valgrind suppressions live in scripts/rex-suppressions-for-valgrind. - Reviewed utility exceptions require package-matched debug symbols for exact allocation frames. The external register records tested package versions, build IDs and source ownership. Missing symbols leave reports visible; do not broaden the patterns to compensate. Weekly CI installs exact-version packages from Ubuntu's signed debug archive and verifies each ELF build ID with scripts/ci-verify-memcheck-symbols before tests. This uses Ubuntu's documented debug-package installation and fails on missing or mismatched companions. Interpreter teardown uses PERL_DESTRUCT_LEVEL=2, inherited from the configured wrapper.
Because child tracing is enabled, helper processes (Perl, Python, shell scripts) can be checked too. This can surface non-ROSE leaks; those should be handled via suppressions when they are clearly external and process-lifetime only.
The wrapper sends Valgrind output through a FIFO to one reader that writes CTest's report. Traced processes can reopen that stream without erasing earlier diagnostics. A regular file, even with %p or %p.%n in its name, can lose an error when the same process execs another instrumented program. The wrapper waits for the report stream to finish and fails on immediate Valgrind error markers, including errors from a child whose parent returns zero and errors before an exec that produces no final summary. Collection failures also fail the test. CTest can otherwise exit zero despite recording memory defects. rex_ctest_valgrind_wrapper_contract exercises these boundaries, concurrent children, and preservation of the command's own failure status. See AUD-038 evidence (nightly-outliner-timeout.md).
CTest's PASS_REGULAR_EXPRESSION can override even the wrapper's nonzero exit. The configured post-Memcheck hook therefore checks the completed dashboard's XML defect counts independently. Any reported defect or unreadable/incomplete dashboard fails the Memcheck command. rex_ctest_memcheck_result_contract proves that output matching cannot accept a memory defect and that subsequent clean runs and show-only inventories remain usable. The hook does not change test expectations, selections, or suppressions.
The weekly workflow runs the reviewed astInterface|testQuery|rex MemCheck selection in eight shards. Before executing a shard, scripts/run_ctest_name_set.py expands it to the transitive CTest DEPENDS and fixture closure. A producer can therefore never land in a different job from its selected consumer. Required support tests may execute in more than one shard. The runner derives numeric selectors from CTest's MemCheck-mode registry, after CTEST_CUSTOM_MEMCHECK_IGNORE is applied; normal-registry indices are not valid in that filtered dashboard registry. A single hosted job was measured at only 895 of 1,749 tests when GitHub enforced its six-hour limit. Registration-index residues are not used: regularly generated test families occupy equally spaced index blocks and can therefore cluster their most expensive variants in a single job. Stable name hashing makes shard membership independent of CMake registration order. The eight-way split preserves all direct coverage while keeping each independent job below that hard ceiling, including the measured parser-heavy tests that need more than three hours under Valgrind. Each shard is capped at eight build and MemCheck workers. Hosted runners currently provide fewer CPUs, while the cap prevents large local act matrices from multiplying host concurrency once per container.
Full libstdc++ map and regex integration specimens remain in regular, sanitizer, and local full CTest, but an exact, reviewed set of their individual REX-tool variants is excluded from scheduled Memcheck. The regex variants crossed the five-and-a-half-hour per-test boundary. The map variants each spent multiple hours in the same frontend traversal, and the containing ACT shard exceeded GitHub's six-hour job boundary before completing. Each such boundary has a bounded replacement using the same REX executable and AST operation: the lazy-system-header fixture is a real Clang system header, the normalization fixtures exercise both transformation drivers, and the nested template-argument frontend fixture avoids unrelated standard-library input. The complete AST-interface <regex> query also remains in regular and sanitizer CTest; Memcheck runs the same query executable against an additional bounded basic_regex instantiation. Every performance exclusion must retain an equivalent bounded instrumented test.
The same rule applies to two older integration families whose useful operation is buried behind unrelated frontend volume. interfaceFunctionCoverage over standard-library headers took more than four hours under Valgrind. Its bounded replacement runs that exact executable over templates, inheritance, loops, switches, calls, and mutable expressions without library headers. The two anonymous-tag move-declaration checks each transit roughly 12,000 preprocessed lines through three serialized tool modes. Their bounded replacement runs the same three modes and hard-checks both generated-source syntax and the absence of leaked anonymous-tag identities. The original inputs remain mandatory in regular, sanitizer, and local full CTest.
CTest also contains compiler-only contract fixtures whose primary command is the exact C, C++, or Fortran compiler selected at configure time. The memcheck wrapper recognizes only those canonical compiler paths and executes them without Valgrind because no REX process exists in those tests. Compiler child processes and test-harness utilities are excluded through the child-trace list. All other primary commands remain instrumented. Do not add a program to either boundary merely to make a report disappear: first prove that the process is external to REX, and keep the boundary tied to the configured tool or an exact harness utility.
Executable-script tests are also kept inside the instrumentation boundary. The wrapper resolves a valid shebang to its real interpreter before starting Valgrind, then passes the script as the interpreter's first source argument. This preserves Linux shebang semantics while avoiding multicall dispatch based on Valgrind's synthetic primary-process name. An env shebang must name one interpreter without options or assignments; malformed or ambiguous shebangs are hard errors rather than uninstrumented fallbacks. With child tracing enabled, both the interpreter and every REX process launched by the script are checked.
Nested executable scripts cross the same boundary after the primary process starts. CMake probes /usr/bin/env under Valgrind for split arguments and the synthetic interpreter argv[0], rejecting incompatible dispatch at configuration. Older uutils releases reject that name; uutils 0.10 passes the traced probe and the nested released/leaking controls. Weekly CI keeps Ubuntu's GNU coreutils alternative for the affected distribution. No script test escapes child tracing.
Interpreting memcheck results¶
rex_symbol_table_allocation_failure replaces global new/delete to throw at successive allocation boundaries and verify that failed index publication preserves ownership and lookups. Valgrind replaces those operators itself, so this probe runs in native CTest and is explicitly excluded from Memcheck. rex_symbol_table_ownership compiles the same specimen without allocator replacement and checks successful insertion, reindexing and removal under Memcheck. AST JSON tests retain their normal allocator and memory coverage. Remove this exclusion if allocation faults can be injected through the actual index allocator under Memcheck without changing compiler behavior.
Memcheck failures can be either memory issues or functional failures:
- Functional failure: The test exits non-zero or hits
ROSE_ASSERT, but Valgrind showsERROR SUMMARY: 0 errors. - Memory defect: Valgrind reports errors or leaks and CTest lists “defects.”
Always inspect build-valgrind/Testing/Temporary/MemoryChecker.<#>.log for the exact cause.
Death tests intentionally abort to validate hard invariants, which leaves “still reachable” memory behind. To keep memcheck and sanitizer runs actionable, exclude them with -LE death and keep them in the normal (non-memcheck) test suite.
Leak kinds in Valgrind¶
definite,indirect,possible: Almost always real leaks and must be fixed in ROSE code.reachable(aka “still reachable”): Memory that is still referenced at exit. This can be:- A real issue in ROSE (e.g., pools or caches not released after AST teardown).
- External/runtime process-lifetime allocations (e.g.,
/usr/bin/perl, LLVM managed statics).
If “still reachable” comes from ROSE-owned structures, treat it as a real issue. If it is from external runtimes and cannot be cleaned up safely, suppress it explicitly in scripts/rex-suppressions-for-valgrind.
Triage workflow in an actively developed tree¶
- Run the tests normally (no memcheck). Fix functional failures first.
- Run memcheck only on known-passing tests with
-R "<regex>". - Inspect memcheck logs to distinguish memory defects from functional failures.
- Fix ROSE leaks at the root cause (prefer CFE fixes when the bug originates there).
- Only add narrow suppressions for independently reproduced third-party or process-lifetime allocations outside ROSE. Anchor them to exact third-party frames and the REX integration boundary.
- Keep all Valgrind suppressions in
scripts/rex-suppressions-for-valgrindand require root-cause justification for each entry.
Development guidelines for memory correctness¶
- Do not hide memory issues by “fixing” tests or weakening assertions.
- Use sanitizers for rapid feedback during development and memcheck for leak triage.
Ownership boundaries (AST, attributes, non-AST)¶
AST nodes (Sage/ROSE IR)¶
- AST nodes are owned by the AST and its memory pools; do not
deletethem directly. - When replacing a subtree, use
SageInterface::replaceExpressionorSageInterface::replaceStatement. The original node becomes detached. ForreplaceStatement, you must delete the old node to avoid leaks. ForreplaceExpression, the old node is deleted by default; passkeepOldExp=trueto take ownership and manage its deletion manually. To delete a detached subtree, useSageInterface::deleteAST(or its wrapperSageInterface::deepDelete). Note that this only deletes the AST nodes; you must handle any dangling symbols or types that result. Avoid rawset_*on child pointers unless you also delete the old subtree. - If you intentionally detach nodes (set pointers to
NULL), you must delete the detached subtree or transfer ownership to a well-defined owner. - AST teardown is the final owner boundary; do not rely on process exit to clean ROSE-owned nodes.
AstAttributeMechanism (attached attributes)¶
- Always implement
AstAttribute::getOwnershipPolicy()in custom attributes. CONTAINER_OWNERSHIP: container owns and deletes on replace/clear.NO_OWNERSHIP: attribute is leaked and its memory is reclaimed only on process exit. This policy is not recommended.CUSTOM_OWNERSHIP: attribute class must manage its own cleanup.- Avoid
UNKNOWN_OWNERSHIP(it is treated as a warning and often leaks). - Do not store raw owning pointers inside attributes without RAII or explicit cleanup.
Non-AST objects (utilities, frontends, tool glue)¶
- Prefer RAII (
std::unique_ptr,std::vector,std::string) for owned objects. - Raw pointers in APIs must be non-owning views; document who owns the lifetime.
- For C interop, keep allocation pairs explicit and matched (
malloc/freeornew/delete), and isolate the boundary in a small helper/RAII wrapper. - Avoid global/process-lifetime caches unless you also provide explicit teardown.
CI policy¶
- Sanitizer and memcheck runs are too heavy for per-commit gating; they run in weekly CI only.
- Use Debug builds for sanitizer/memcheck to keep
ROSE_ASSERTactive during memory triage.
Common pitfalls¶
- Running memcheck in a non-Valgrind build does nothing (CTEST memorycheck is disabled).
- Mixing Valgrind and sanitizers in the same build is not supported; use separate build directories.
- Memcheck is slow; reduce the test set or run smaller subsets during iteration.