Skip to content

AXPY End-to-End Runbook with Clang/Flang

This note captures the steps, roadblocks, and fixes required to get a pure C++ AXPY example working end-to-end with the experimental Clang frontend in REX.

Toolchain & Build

  • Configured a fresh build-clang tree with clang-20, clang++-20, and flang as the backend compilers: 
    cmake -S . -B build-clang \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/lib/llvm-20/bin/clang \
  -DCMAKE_CXX_COMPILER=/usr/lib/llvm-20/bin/clang++ \
  -DCMAKE_Fortran_COMPILER=/usr/lib/llvm-20/bin/flang \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
  -DENABLE-C=ON -DENABLE-FORTRAN=ON
cmake --build build-clang -j$(nproc)
  • All invocations of rose-compiler require LD_LIBRARY_PATH to point at the freshly built build-clang/lib directory.

Source Example

  • Added tests/nonsmoke/functional/CompileTests/Cxx_tests/rex_test2025_issue160_axpy_system_headers.cpp as a C++ example that now exercises common standard headers (<array>, <numeric>, <cmath>) to validate language default initialisation.
  • The program initialises two fixed-size buffers with std::array, performs the AXPY loop, and returns 0/1 based on a relative-error check (no runtime I/O needed).

Frontend Issues & Fixes

  1. Missing language defaults for C++
  2. Symptom: Including even <cmath> produced thousands of errors like unknown type name 'bool' and missing __builtin_* intrinsics.
  3. Root cause: We were never initialising clang::LangOptions with C++ defaults; the invocation inherited C-like settings from CompilerInvocation::CreateFromArgs.

  4. Fix: Call clang::LangOptions::setLangDefaults with the host triple and lang_gnucxx17 (see src/frontend/CxxFrontend/Clang/clang-frontend.cpp:268). We collect any implicit include paths returned by that helper and append them back into the active PreprocessorOptions.

  5. Unhandled LinkageSpecDecl (extern "C"/"C++")

  6. Symptom: Declarations inside extern "C" could be dropped from the ROSE AST (unreachable / missing from the owning scope), and linkage information could be lost during unparsing.

  7. Fix: VisitLinkageSpecDecl now always attaches translated inner declarations to the current scope and preserves language linkage on those declarations (e.g., set_linkage("C") + setExtern() for extern "C" T f() forms). Brace-form extern "C" { ... } is preserved via the existing preprocessing-info mechanism (and marked via setExternBrace() on the contained declarations) so the unparser round-trips the original structure.

  8. Anonymous namespace handling

  9. Symptom: The translator asserted inside buildDefiningFunctionDeclaration_T when processing a file that declared helper routines inside namespace { ... }.
  10. Workaround: Keep helper routines at global scope and mark them static. This sidesteps unimplemented namespace support in the current frontend.
  11. Duplicate scope symbols triggering name-qualification warnings
  12. Symptom: nameQualificationSupport warned on almost every reference to block-local variables (notably for-loop indices).
  13. Root cause: The Clang bridge manually pushed a SgVariableSymbol into the current scope after calling SageBuilder::buildVariableDeclaration_nfi, but the builder already inserts the symbol. This double insertion inflated the scope's symbol table and fooled the unparser into thinking multiple declarations existed.
  14. Fix: Drop the extra insert_symbol and rely on SageBuilder to manage the symbol table. The unparser now uses the scope counts directly with no custom filtering, so only real ambiguities surface.

Validation Flow

  1. Run the translator to generate rose_rex_test2025_issue160_axpy_system_headers.cpp: 
    LD_LIBRARY_PATH=$PWD/build-clang/lib \
  ./build-clang/bin/rose-compiler tests/nonsmoke/functional/CompileTests/Cxx_tests/rex_test2025_issue160_axpy_system_headers.cpp
    Output appears at the repository root.
  2. Verify both the original and generated sources compile and execute cleanly: 
    clang++ tests/nonsmoke/functional/CompileTests/Cxx_tests/rex_test2025_issue160_axpy_system_headers.cpp -O2 -o /tmp/axpy_orig
clang++ rose_rex_test2025_issue160_axpy_system_headers.cpp -O2 -o /tmp/axpy_rose
/tmp/axpy_orig && /tmp/axpy_rose   # both exit 0

Limitations & Next Steps

  • Future hardening: it is still worthwhile to tighten the Clang bridge so loop induction variables are stored under the SgForInitStatement scope, but the immediate user-facing problem is gone.

Known Gaps

  • The frontend still struggles with large swaths of the C++ standard library (heavy template specialisations, allocator traits, etc.). The builtin/runtime mismatches are gone, but expect additional traversal failures until those visitors are implemented.

May 2025 Status & Next Steps

  • Bug fixed: the Clang driver now initialises LangOptions for C++ input, picking a GNU C++17 default, injecting implicit headers, and enabling builtin intrinsics. This unblocks <cmath>/__builtin_* usage and lines up with the intended Clang invocation (clang-frontend.cpp:267-395, clang-to-dot.cpp:264-325).
  • AXPY sample: the smoke test in tests/nonsmoke/functional/CompileTests/Cxx_tests/rex_test2025_issue160_axpy_system_headers.cpp now exercises <array>, <numeric>, <cmath>, and <cstddef>. It compiles and runs with stock clang++, but the current Clang→ROSE bridge still aborts when walking the resulting libstdc++ AST.
  • Translator gaps: namespace traversal, template declarations/specialisations, and many type/statement visitors remain stubs. With real system headers enabled the translator quickly encounters std::array scaffolding and dies. Skipping or generating opaque stand-ins for those nodes is the next prerequisite before we can ingest richer C++.
  • Proposed short-term plan
  • Teach VisitNamespaceDecl to record namespace shells while continuing to walk child declarations safely (no early aborts).
  • Add minimal handling for class/function/alias/var template decls: traverse the templated declaration, create an opaque SgType, and store symbols so later lookups succeed.
  • Extend record/typedef/type visitors to fall back to buildOpaqueType and unblock std::array basics; do the same for critical statement/expression visitors (DeclRefExpr, MemberExpr, CallExpr into template instantiations).
  • Re-run the AXPY sample end-to-end, then incrementally admit additional headers.
  • Risks: without better symbol management we risk leaking thousands of opaque placeholders or emitting unusable ASTs. Each fallback should log once (guarded by a debug knob) so production runs stay quiet.

Roadmap (Follow-Through Checklist)

The following sequence should be executed top-to-bottom for any future engineer continuing this work. Do not re-investigate the fixed language-default bug; the remaining tasks are strictly around AST translation hardening:

  1. Namespace scaffolding
  2. Finish implementation of VisitNamespaceDecl to emit an actual SgNamespaceDeclarationStatement while preserving child traversal.
  3. Verify we no longer produce SgNullStatement placeholders for namespace IR.
  4. Template declaration handling
  5. Implement stub visitors for class/function/type-alias/variable templates that fabricate opaque types + symbols in scope.
  6. Ensure specialisations (full/partial) are walked, even if they degrade to placeholders.
  7. Type fallback coverage
  8. Harden VisitRecordType, VisitTemplateSpecializationType, and VisitTypedefType to avoid assertions; emit buildOpaqueType or buildUnknownType with deterministic naming.
  9. Add exhaustive unit logging behind a debug flag so missing cases are easy to trace.
  10. Expression/statement wiring
  11. Handle the common expression forms produced by std::array and friends: DeclRefExpr, MemberExpr, CXXConstructExpr, CallExpr, etc.
  12. Reuse existing SageBuilder helpers where possible; skip unhandled nodes with buildNullExpression() rather than abort.
  13. Regression loop
  14. Re-run rose-compiler tests/nonsmoke/functional/CompileTests/Cxx_tests/rex_test2025_issue160_axpy_system_headers.cpp until it produces a translated file without crashing.
  15. Expand coverage gradually (<vector>, <algorithm>, etc.) using the same smoke-test pattern.
  16. Linkage specifications are flattened; follow-up work is needed if downstream analyses require exact extern metadata.