Skip to content

fix(autocomplete): suggest interactive option names#50

Open
autocarl wants to merge 6 commits into
yllibed:mainfrom
autocarl:agent/issue-43-interactive-option-completion-20260708212315
Open

fix(autocomplete): suggest interactive option names#50
autocarl wants to merge 6 commits into
yllibed:mainfrom
autocarl:agent/issue-43-interactive-option-completion-20260708212315

Conversation

@autocarl

@autocarl autocarl commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add interactive autocomplete option candidates for --/partial --flag tokens.
  • Reuse the same option sources as shell completion: static/global output options, custom global options, and route OptionSchema.KnownTokens.
  • Add regression coverage for route/global option suggestions and partial option-prefix filtering.

Fixes #43

Test Plan

  • dotnet test src/Repl.Tests/Repl.Tests.csproj -c Release --filter 'FullyQualifiedName~Given_InteractiveAutocomplete_OptionCandidates'
  • dotnet test src/Repl.Tests/Repl.Tests.csproj -c Release --filter 'FullyQualifiedName~Given_InteractiveAutocomplete'
  • dotnet test src/Repl.IntegrationTests/Repl.IntegrationTests.csproj -c Release --filter 'FullyQualifiedName~Given_Completions'
  • dotnet test src/Repl.slnx -c Release

Add interactive autocomplete option candidates from global and route option sources so --flag prefixes behave like shell completion.\n\nFixes yllibed#43
@autocarl autocarl marked this pull request as ready for review July 8, 2026 21:32
@carldebilly

Copy link
Copy Markdown
Member

@codex please review

chatgpt-codex-connector[bot]

This comment was marked as resolved.

autocarl

This comment was marked as resolved.

…parser semantics

Review-panel round on the initial commit — all three field repros fixed
red-first, plus the structural findings:

- One option-candidate source (OptionTokenCompletionSource) now feeds both
  the interactive engine and the shell-completion engine: the previous
  copy-pasted lists could only drift, and a new global option had to be
  added twice. Composed tokens (--alias, --output:format) only allocate
  when they can match the typed prefix.
- Prior tokens are normalized through InvocationOptionParser exactly like
  the shell path: option VALUES are consumed ("--channel beta" no longer
  leaves a stray positional that hides route options) and the bare "--"
  end-of-options separator suppresses option suggestions past it.
- The committed command prefix goes through the router''s own
  ResolveUniquePrefixes expansion, and route-prefix matching mirrors
  RouteResolver''s terminal-segment alias rule: alias and unique-prefix
  invocations ("i bib-overalls") now surface the route''s options.
- Route options are offered as soon as the command WORDS are typed (only
  dynamic argument segments may be missing): "install --" suggests
  --force, matching the parser''s options-anywhere acceptance.
- IsOptionPrefixToken gates suggestions: a single dash surfaces short
  aliases (-f) while signed numeric literals (-42) stay positional,
  mirroring the invocation parser; token classification uses the same
  predicate so short flags no longer paint as invalid.
- Dynamic completion providers are skipped when the current token is an
  option prefix — provider values are parameter values, not option names.
- OptionSchema.KnownTokens is materialized once (was a recomputed LINQ
  projection hit per keystroke per route).
- Tests: regression coverage for every repro above (valued option, alias,
  single dash, missing positionals, end-of-options, negative number,
  provider pollution, cross-source dedup); neutral test data.
@carldebilly

Copy link
Copy Markdown
Member

Review round on the initial commit (local multi-perspective review + field repros) — fixes pushed in dc6997b, every behavioral fix red-first. Full suite green (1091 tests).

Fixed

  • Copy-paste vs reuse: the option-candidate enumeration was a line-for-line clone of the shell-completion engine's. Both engines now consume one shared source (OptionTokenCompletionSource), so the two surfaces cannot drift and a new global option is added once. Composed tokens (--alias, --output:format) only allocate when they can match the typed prefix.
  • Valued options broke route matching (install x --channel beta -- lost --force): prior tokens are now normalized through InvocationOptionParser exactly like the shell path — option values are consumed, and a bare -- (POSIX end-of-options) suppresses option suggestions past it.
  • Aliases/unique prefixes (i x --): the committed prefix goes through the router's own ResolveUniquePrefixes expansion, and route-prefix matching mirrors RouteResolver's terminal-segment alias rule — autocomplete now agrees with what would execute.
  • Options before positionals: install -- suggests --force; route options appear as soon as the command words are typed (only dynamic segments may be missing), matching the parser's options-anywhere acceptance.
  • Single-dash aliases: a new IsOptionPrefixToken gate surfaces -f-style aliases from -, while signed numeric literals (-42) stay positional (same rule as the parser). Token classification uses the same predicate, so short flags no longer paint as invalid.
  • Provider pollution: dynamic completion providers are skipped when the current token is an option prefix.
  • Perf: OptionSchema.KnownTokens materialized once (was a recomputed LINQ projection per keystroke per route); hot-path LINQ/interpolations removed; dead token.Length >= 2 dropped.
  • Tests: regression coverage for each case above + cross-source dedup pin; neutral test data (public repo).

Considered, deliberately unchanged

  • Quoted option-like tokens ("--channel") are treated as options — identical to what execution does after tokenization, so autocomplete stays consistent with the parser rather than diverging.
  • Interactive shows parameter placeholders on an empty token (options appear from the first -), while shell completion lists options on empty input after a terminal route — an intentional UX difference between a live menu and a dump-style completion list.
  • The token-classification pass predates this PR and rebuilds prefixes per token (quadratic on very long pasted lines, hosted sessions included), and suggestion rendering does not strip control characters from the raw prefix; both are pre-existing surfaces worth their own issue rather than a rider on this PR.

Option suggestions share AutocompleteSuggestionKind.Parameter, so the
live hint could shortcut to "Param skillName" while the menu listed
option names. In option-prefix mode the parameter shortcut is skipped
and the hint renders the option alternatives.
chatgpt-codex-connector[bot]

This comment was marked as resolved.

…eview round 2)

- SECURITY: both completion engines parsed prior tokens with the legacy
  parser profile, whose AllowResponseFiles default meant a @file token
  triggered server-side file reads (UNC probes included) — per keystroke
  on remote-controlled lines in hosted sessions. A shared
  CompletionParsingOptions profile (response files off) now feeds both
  engines; the execution pipeline already gated expansion on
  !isInteractiveSession for the same reason.
- Single-dash short flags are dropped from the prefix only BEFORE the
  POSIX "--" separator; after it they are positional arguments and keep
  their place in the route prefix (exact-route providers fire again).
- The shell gate is now the same IsOptionPrefixToken as the interactive
  menu: "-" surfaces short aliases in shell completion too, and two
  parity tests pin interactive == shell candidates for "--" and "-"
  inputs — a divergent reimplementation on either side now fails.
- Unique-prefix/alias expansion is bounded by the deepest route/context
  template: the per-index candidate derivation no longer scales with the
  token count of an adversarial line; the misleading "menu shows the
  competing alternatives" comment now states what really happens to an
  ambiguous committed shorthand.
- The autocomplete literal/alias rule delegates to
  RouteResolver.IsLiteralMatch (comparison parameterized) instead of
  mirroring it; ReplCaseSensitivity.ToStringComparison() replaces the
  ternary re-derived at every call site; GlobalOptionParser reuses the
  shared IsSignedNumericLiteral.
- docs/shell-completion.md documents the two deliberate divergences that
  remain (empty-token options, enum values).
@carldebilly

Copy link
Copy Markdown
Member

Second review round on a1eaafa (multi-perspective panel) — fixes pushed in 168d644, red-first throughout. Full suite green (1096 tests).

Fixed

  • Security (the round's blocker): both completion engines normalized prior tokens with the legacy parser profile, whose AllowResponseFiles default meant a @file prior token triggered server-side File.Exists/File.ReadAllText — per keystroke on remote-controlled lines in hosted sessions (UNC probes / NTLM leak vector on Windows), the very exposure the execution pipeline gates with !isInteractiveSession. A shared CompletionParsingOptions (response files off) now feeds both engines; pinned by a test proving a real @file prior token is never expanded.
  • Post--- positionals: single-dash flags are dropped from the route prefix only before the POSIX separator; after it they are positionals and keep their segment position (red observed by guard inversion).
  • Parity pinned: two tests run BOTH engines on the same app and assert identical candidates for -- and - inputs — the shared token source was only half the guarantee, the gates now agree too (shell adopts IsOptionPrefixToken, so -f works in shell completion as well).
  • Bounded expansion: unique-prefix/alias expansion is capped at the deepest route/context template — per-keystroke cost no longer scales with the token count of an adversarial line; the misleading "menu shows the competing alternatives" comment now states the real behavior of an ambiguous committed shorthand.
  • No more mirrors: the autocomplete literal/alias rule delegates to RouteResolver.IsLiteralMatch (comparison parameterized); ReplCaseSensitivity.ToStringComparison() replaces five hand-rolled ternaries; GlobalOptionParser reuses the shared IsSignedNumericLiteral.
  • docs/shell-completion.md documents the two deliberate divergences that remain (empty-token options, enum values).

Analyzed, deliberately unchanged

  • The permissive parser's greedy value consumption (e.g. config --verbose set swallowing set) is more permissive than execution, not less: at execution time an undeclared option interposed before the end of the command words breaks route resolution too (only GlobalOptionParser knows arities, and only for declared global options). Completion stays consistent with both the shell path and execution's outcome.
  • The three case-sensitivity regimes (router hardcodes OrdinalIgnoreCase, Autocomplete.CaseSensitive, Parsing.OptionCaseSensitivity) predate this PR and span the whole routing surface — worth a dedicated issue rather than a rider here.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

…guess (review round 3)

Prior-token normalization no longer guesses option arities with the
generic parser; it mirrors execution's resolve-then-parse order:

- The command prefix is built by stripping global options with the
  parser that knows their arities (GlobalOptionParser: --no-logo takes
  no value, --output: does), then resolving the route. A matched route
  contributes exactly its own segments and leaves every route option in
  its trailing tokens — so a valued short alias (-c beta) or a valueless
  global flag before the command can never corrupt the prefix.
- Route options are sourced from the single route RouteResolver would
  actually select (fixes overloaded routes offering the wrong overload's
  options) and only when every required positional is filled (a resolved
  match with no leftover positional) — accepting a suggestion always
  yields a command that runs cleanly. Per maintainer decision, options
  wait until required arguments are satisfied.
- The dynamic completion provider is skipped for an option-prefix token
  only before the POSIX "--" separator; after it a dash token is a
  positional and the provider runs.
- Shell completion mirrors all of the above (route-first prefix, "--"
  termination state, valued short-alias handling), keeping the two
  surfaces consistent.

Every behavioral fix is covered by a red-first regression test; full
suite green (1102).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f39a0f88b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Repl.Core/Autocomplete/AutocompleteEngine.cs Outdated
Comment thread src/Repl.Core/ShellCompletion/ShellCompletionEngine.cs
Comment thread src/Repl.Core/Internal/Options/OptionTokenCompletionSource.cs Outdated
Comment thread src/Repl.Core/Autocomplete/AutocompleteEngine.cs Outdated
…case (review round 4)

Removes the last divergences from execution's resolve-then-parse order:

- BuildCommandPrefix no longer strips the bare "--" separator or drops
  dash-prefixed tokens before resolving. Route resolution runs before
  option parsing (as CommittedInputResolver does), so "--" and short
  tokens like "-prod" bind to route segments as positional values —
  "remote -prod status" / "remote -- status" now keep the terminal
  literal suggestion. The no-match fallback keeps the tokens as typed
  instead of second-guessing which are options.
- Shell completion mirrors the same: "--" only records the end-of-options
  state, tokens stay as typed for route resolution.
- Route-option candidates are filtered against OptionSchema.Entries with
  each entry's own effective case sensitivity, not the flattened
  KnownTokens under one global comparison — a case-insensitive option is
  offered for a differently-cased prefix even under a case-sensitive
  global default (pinned via the shared source, since a per-option
  override is otherwise not expressible through the attribute).
- Shell enum-value completion recognizes short option aliases: a pending
  option is detected with the single-dash gate, so "-m <TAB>" offers the
  enum values like "--mode" does.

All red-first; full suite green (1106).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae605dbb7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// positional is filled: a resolved match with no leftover positional means the
// options belong to a command that runs cleanly. This is the same resolution
// execution performs, so autocomplete never offers an option the run would reject.
if (app.Resolve(commandPrefix) is { RemainingTokens.Count: 0 } match)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Delay route options past optional arguments

Fresh evidence is the optional-segment case: for a route like run {profile?} with a [ReplOption] bool force, the prefix run -- satisfies this check and offers --force, but executing the completed run --force does not leave --force in match.RemainingTokens; RouteResolver binds it to the optional {profile} before option parsing. Route option candidates should wait until optional positionals are disambiguated, or routing needs to skip option tokens before these suggestions are safe.

Useful? React with 👍 / 👎.

Comment on lines +1440 to +1442
if (string.Equals(values[i], "--", StringComparison.Ordinal))
{
optionsTerminated = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Defer -- termination until after route resolution

When an earlier bare -- is consumed by the route as a positional, such as remote {name} status with input remote -- status --, this raw scan has already set OptionsTerminated. The route is terminal and execution would parse a completed trailing --force from the route's remaining tokens, but autocomplete suppresses all option candidates because it treats the positional -- as the separator; track the separator only among tokens left after the matched route/global parsing boundary.

Useful? React with 👍 / 👎.


foreach (var alias in options.Output.Aliases.Keys)
{
TryAddComposed("--", alias, currentTokenPrefix, comparison, dedupe, results);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match output aliases case-insensitively

With the default case-sensitive option setting, this filters output aliases using an ordinal comparison, but GlobalOptionParser resolves --json and other output aliases through OutputOptions.Aliases, whose dictionary is case-insensitive. In that default configuration, a user typing --J/--JSON can execute the uppercase alias once completed, but neither interactive nor shell completion offers --json; output alias candidates should use the alias resolver's case-insensitive semantics rather than the general option-token setting.

Useful? React with 👍 / 👎.

Comment on lines +103 to +104
var prefix = stripped as string[] ?? [.. stripped];
if (app.Resolve(prefix, activeGraph.Routes) is { } match)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve unique prefixes in shell completion

Execution expands unique command prefixes before route matching, but this shell-completion path resolves the stripped tokens directly. For a route install {name}, app i pkg -- (where i is the unique prefix for install) runs after prefix expansion, while shell completion sees no terminal route here and omits install's route options; expand unique prefixes against the captured graph before calling app.Resolve.

Useful? React with 👍 / 👎.

if (!AutocompleteEngine.IsGlobalOptionToken(previousToken))
// A single dash is enough: short option aliases (e.g. "-m") take values too, and the
// schema resolves them like any other token below.
if (!AutocompleteEngine.IsOptionPrefixToken(previousToken))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ignore route-bound dash tokens for enum values

Because this now treats any previous single-dash token as a pending option, shell completion can offer enum values for a dash-prefixed route argument that was already consumed by routing. For example, with deploy {target} and an option-only enum alias -m, app deploy -m has target = -m; accepting a suggested enum value leaves it as trailing positional text rather than the value for -m. Resolve the pending option from the matched route's remaining option tokens, not from the raw prior command tokens.

Useful? React with 👍 / 👎.

@carldebilly

Copy link
Copy Markdown
Member

@codex please review again

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: ae605dbb7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Interactive Tab completion never suggests option names (--flags)

2 participants