fix: end a command at newline unless the line is continued - #53
Open
dpezto wants to merge 3 commits into
Open
Conversation
Commands previously accepted their arguments across a raw newline, so
invalid input like a bare 'set' followed by 'title "foo"' on the next
line parsed as one valid statement. Gnuplot ends a command at end of
line unless the line ends with a backslash; semicolons separate
commands on one line.
Every cmd_* rule now gates its argument tail behind the zero-width
same-line token _gval_sep (or _gval_tail where the tail can start with
a range block: plot, splot, vfill, fit, stats). The gate declines at a
newline, semicolon, comment, or EOF, so the orphaned next line becomes
a statement-level ERROR; backslash continuation still works. The
_set_multiplot/_unset_multiplot pair carries a mandatory gate so the
multiplot_block GLR branch can shift the same token and keep its
dynamic-precedence tie. cmd_undefine is unchanged: its token.immediate
tail was already newline-bounded. The scanner's gate whitelist gains
'{' so a complex literal ('print {1,2}') still binds on the same line.
A bare command with no arguments ('set', 'plot') now parses as a valid
empty node; rejecting it is left to downstream tooling. Remaining gaps
(assignment right-hand side, 'set for' tails) are recorded in
docs/deferred.md.
Cost: parser.c +35,297 bytes (+0.14 %), +29 states, TOKEN_COUNT and
LARGE_STATE_COUNT flat. Corpus grows 230 to 240.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ncat
Inside a generic option body, a word immediately followed by a dot is
string concatenation on a variable ('set print pal.".gp"'), but the
keyword tables claimed the word anyway: 'pal' matched the palette row,
and error recovery inserted a zero-width identifier to rebuild the
expression, leaving a missing node in an otherwise valid tree.
Both scanner paths now decline on an adjacent dot: the gate word-peek
emits the plain separator so the word stays a value, and scan_keywords
refuses every keyword table in generic bodies, mirroring the existing
constant-word denylist. Spaced leading-dot numbers ('at graph .5',
'rotate by .5') are unaffected because their dot is not adjacent to the
word; the eccentric adjacent form 'set size.5' still parses through the
bespoke option head, matching gnuplot's tokenizer.
The zero-width node predates the newline-gate work; it went unnoticed
because 'tree-sitter parse' prints such nodes without a MISSING label,
so grep-based oracle checks never saw it. Scanner-only change: parse
tables are untouched.
The comment justified the \w+_ prefix with 'set fit prefix', which does not exist (probed on 6.0.4: unrecognized option). Fit parameters are plain variables with no prefix system; the user-selectable prefix comes from 'stats ... name' and its synonym 'prefix'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A bare newline no longer lets a command absorb the next line as its arguments.
setfollowed bytitle "foo"on the next line used to parse as one validcmd_set; gnuplot rejects both lines independently. Commands now end at a newline, semicolon, comment, or EOF unless the line ends with a backslash.Mechanism: every
cmd_*rule gates its argument tail behind the existing zero-width same-line scanner token_gval_sep(_gval_tailwhere the tail can begin with a range block: plot, splot, vfill, fit, stats). When the gate declines, the command reduces to a bare node and the orphaned next line surfaces as a statement-level ERROR, matching the shape the corpus already documents forreplot. The_set_multiplot/_unset_multiplotpair carries a mandatory gate so themultiplot_blockGLR branch shifts the same token and keeps its dynamic-precedence tie.cmd_undefineis untouched; itstoken.immediatetail was already newline-bounded. The scanner's gate whitelist gains{so complex literals (print {1,2}) still bind on the same line.A command with no arguments (
set,plot) now parses as a valid empty node; rejecting it is left to downstream tooling. Remaining same-line gaps (assignment right-hand side,set fortails) are recorded in docs/deferred.md.Numbers: parser.c +35,297 bytes (+0.14 %), STATE 11616 to 11645, LARGE_STATE_COUNT down 1, TOKEN_COUNT unchanged. Corpus 230 to 240, all green; the synthetic oracle and external validation file parse with no new ERROR or MISSING nodes, including their heavy backslash-continuation sections. keywords.json regenerates with no diff; node-types.json changes are only required-to-optional flips on command children.