Skip to content

feat(extract): ingest AsciiDoc documentation (#2938) - #3089

Open
abhay-codes07 wants to merge 1 commit into
Graphify-Labs:v8from
abhay-codes07:feat/asciidoc-docs
Open

feat(extract): ingest AsciiDoc documentation (#2938)#3089
abhay-codes07 wants to merge 1 commit into
Graphify-Labs:v8from
abhay-codes07:feat/asciidoc-docs

Conversation

@abhay-codes07

Copy link
Copy Markdown
Contributor

Closes #2938.

The problem

.adoc was in no extension set, so a project whose documentation is AsciiDoc had none of it in the graph — the detector reported that only the PNGs beside the docs were ingested. _PROSE_EXTS already knew .adoc for word counting, which is how far it got.

The change

.adoc / .asciidoc join every place a document is decided:

And a structural pass, extractors/asciidoc.py, so the deterministic layer is on par with markdown's:

  • the file is a page node carrying the document title (= Title) and the header attributes (:author: …) as frontmatter, sanitized like markdown's;
  • section titles (==, ===, …) are heading nodes nested by level, file → contains → heading, heading → contains → subheading;
  • include::other.adoc[], xref:other.adoc[...] / xref:other#anchor[] (an extension-less xref names a sibling .adoc), link:<local path>[...] and the cross-document <<other.adoc#anchor,text>> form become references edges. The target id is minted from the resolved path — the same recipe as the target's own file node — so it merges into that node rather than spawning a ghost, and an existing target is stamped target_file for the incremental remap, exactly as markdown does (Incremental rebuild drops cross-file references edges between Markdown docs; GRAPHIFY_FORCE does not help in the hook (follow-up to #1018) #2211).

Skipped rather than guessed: external URLs, in-page anchors, non-document targets (diagram.png), and paths carrying an unresolved AsciiDoc attribute (include::{snippets}/gen.adoc[]). Delimited blocks (----, ...., ////, ++++, ====, ****, ____) and // comment lines are not parsed for headings or links, mirroring markdown's fenced-block handling.

What it looks like

guide.adoc (page: title="Operations Guide", frontmatter={author: "Ops Team"})
  ├─ contains → Deploy
  │    └─ contains → Rollback
  ├─ contains → Monitor
  ├─ references → runbook.adoc      (xref:runbook.adoc[...] and xref:runbook#steps[] — one edge)
  ├─ references → alerts.adoc       (include::parts/alerts.adoc[] and <<parts/alerts.adoc#p1,…>>)
  └─ references → checklist.md      (link:checklist.md[...])
checklist.md ─ references → guide.adoc   (markdown → asciidoc now resolves)

Tests

tests/test_asciidoc.py — 21 tests: both extensions present in every set that decides a document (classification, dispatch, slicing, markdown linkability) and listed by detect(); the page node's title and attributes; section nesting; block and comment skipping; duplicate section titles; a title-less fragment; unreadable files; each link form producing one edge per target; the skip rules; the target_file stamp; and a corpus-level check that every reference — in both directions between markdown and AsciiDoc — merges into the linked document's real node with nothing dangling and no stamp leaking.

With the registrations reverted and only the extractor module kept, the 4 integration tests fail (the extractor's own unit tests rightly keep passing). test_detect, test_oversized_document_slicing, test_extractors_registry and the markdown/wikilink suites are unchanged (290 passed); the full suite matches the v8 baseline.

README's file-types table is updated.

`.adoc` was in no extension set, so a project documented in AsciiDoc had
none of it in the graph — only the PNGs beside the docs were ingested, and
the detector said so in passing.

`.adoc`/`.asciidoc` join every place a document is decided: DOC_EXTENSIONS
(classification and the semantic pass), the splittable-text set (an
oversized manual is sliced, not truncated), the callflow and hook-guard
extension lists, and markdown's linkable set (a `[text](./guide.adoc)`
link now resolves).

A structural pass, extractors/asciidoc.py, mirrors the markdown extractor
so the deterministic layer is on par: the file is a `page` node carrying
the document title and header attributes (as `frontmatter`), section
titles are `heading` nodes nested by level, and `include::`, `xref:`,
`link:<local path>` and the cross-document `<<file.adoc#anchor>>` form
become `references` edges minted from the resolved path, so they merge
into the linked document's own node (stamped `target_file` for the
incremental remap, like markdown). External URLs, in-page anchors,
non-document targets and paths carrying an unresolved attribute
(`{docdir}/x.adoc`) are skipped rather than guessed; delimited blocks and
comment lines are not parsed for headings or links.
Copilot AI lite review requested due to automatic review settings August 25, 2026 18:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Formal verification. 1 change(s) tested, no difference found (not proven).


Graphify review — findings

Adds an AsciiDoc extractor that mirrors the markdown one: extract_asciidoc turns a .adoc/.asciidoc file into a page node with nested heading nodes per section title, and mints references edges from include::, xref:, link:, and <<...>> targets whose ids match the linked document's own node so they merge rather than spawn ghosts. External URLs, in-page anchors, unresolved-attribute paths, and delimited block bodies (listing/literal/comment/passthrough/example/sidebar/quote) are skipped, and extension-less xrefs resolve to a sibling .adoc. Registers the new extensions across DOC_EXTENSIONS, the extract dispatch, hook source exts, and the HTML viewer's document-type detection, and documents the new format support in the README.

Worth a look

  • .asciidoc files are not included in hook source extensionsgraphify/cli.py:74 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Delimited block closes on any same-character delimiter lengthgraphify/extractors/asciidoc.py:49 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • _block_delim matches any line of same char length >=4 including heading underlines / horizontal rulesgraphify/extractors/asciidoc.py:52 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2520 functions depend on the 620 functions this change touches.

Health — this change adds coupling hotspots:

  • new: extract() — 500 callers, 42 callees
  • new: _rebuild_code() — 98 callers, 50 callees
  • new: detect() — 110 callers, 15 callees
  • new: save_manifest() — 40 callers, 11 callees
  • new: extract_files_direct() — 17 callers, 20 callees
  • new: extract_xaml() — 19 callers, 17 callees
  • new: extract_corpus_parallel() — 26 callers, 11 callees
  • new: extract_js() — 83 callers, 3 callees
  • …and 49 more — each is listed as a finding

Verification — 2520 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 2473 function(s) in the blast radius were not formally verified this run

Formal verification

No difference found (not proven): No behavior difference found in normalize\_node (not a proof).

The verifier ran both versions of normalize\_node on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

· 1 grounded finding(s) anchored inline below; 56 more finding(s) on lines outside this diff (see the check run).

return Path(os.path.normpath(str(candidate)))


def extract_asciidoc(path: Path) -> dict:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressionextract_asciidoc()

fans out to 8 callees (efferent coupling); 11 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Detect and digest asciidoc / adoc documentation

2 participants