Scaffold NSE support

[lionel-]

Progress towards #1338

This PR adds preliminary support for NSE functions that create a local scope, like local() or test_that(). Data-masking functions like with() are also supported, although only for the effect of opening a scope, not for the effect of injecting a data frame in the scope.

The semantic index is created in two passes per scan unit to resolve a chicken and egg problem: to recognise an NSE function while we're building the index, we need to resolve the function callee: is it locally bound, perhaps to a function with NSE annotations? If it's locally unbound, does it come from an imported file?

The two passes are structured by scan units, which are either the file scope, or a lazy scope. Eager scopes are just traversed linearly without creating a new unit.

We already had a similar pre-scan to collect bound symbols for lazy scopes. This first pass now has more responsibilities. The two passes design is forward-looking and takes into account that file imports from source() (and library() in a future PR) calls may themselves be masked by assignments, which may or may not have local effects if they happen within a nested NSE scope.

NSE functions are parameterised along two axes:

• NseScope: Either Current (evalq(), rlang::on_load()) or Nested (local(), test_that()).

• NseTiming: Either Eager (local(), test_that()) or Lazy (rlang::on_load(), shiny::observe())

Each combination gets distinct handling:

• Current + Eager: Same as regular evaluation. The reason this combination exists is that in the future we'll support the envir argument of evalq(), at which point it will stop falling into the Current bucket depending on its arguments.

• Nested + Eager: Pushes a scope. Scanned inline during the eager linear scan. Uses of enclosing names get a point-in-time snapshot, unlike the accumulated union a lazy scope gets. That's the main difference with lazy scopes like functions.

• Current + Lazy: It's a special scan unit whose definitions are deferred, just like <<-

  x <- 1
  on_load({ x <- 2 })   # routed to file scope, deferred
  x                     # sees {1, 2}: deferred def didn't shadow `x <- 1`

• Nested + Lazy: Pushes a scope, special scan unit. Uses refer to an enclosing snapshot, exactly like function scopes.

The linear scanner decides whether a function is NSE by resolving symbols eagerly. This is always correct in an eager context, but is only a guess in a lazy context, like inside a function or Shiny reactive. If a binding comes in later and contradicts that guess, we can't order these two states of the world.

f <- function() local({ x <- 1 })
local <- identity

The position that we take from now on is that this is an ambiguous program that should be linted. We record the ambiguity in LazyShadowAmbiguity { name, range } and log it later on. In a future PR, we'll surface in diagnostics.

That's a general principle that we'll apply for all effects, for instance this will produce a lint:

function() source("path.R")
source <- identity

Imported effects are now handled by new ImportsResolver methods. The PR also introduces an effect registry where we'll start documenting effects of known functions from base and other important packages (rlang, shiny, etc).

Positron Release Notes

New Features

• Improved support for symbol navigation and symbol renaming within well known NSE functions like test_that().

Bug Fixes

• N/A

[DavisVaughan]

I've looked at everything except for builder_nse.rs, which I'll do on Monday, but here are some starting comments

[DavisVaughan]

Do you think this one really needs to be a trait method?

It feels like this is the definitive implementation for this function, and it doesn't use self at all.

Like, could it just be a "free" function that queries this external effect registry?

[lionel-]

The registry is a crutch, but ideally the effects should come from annotations, and then we do need the imports resolver.

The method will probably evolve over time though. Some thoughts:

• Effects parsing belongs in oak_semantic, so the oak_db impl would have to call oak_semantic to parse decls
• Maybe instead we resolve to a definition and parse the decls from there. In this case the trait method is no longer about effects at all.

[DavisVaughan]

Isn't there a 3rd possibility? Like Parent maybe? I'm thinking of some environment that is passed through multiple layers before finally being used. This is a bad example but kind of demonstrates the idea:

x <- "foo"

fn <- function(env = caller_env()) {
  fn2(env)
}

fn2 <- function(env = caller_env()) {
  x <- "bar"

  # `env` here isn't really Current, it's multiple layers up
  eval(quote(x), envir = env)
}

fn()
#> [1] "foo"

[lionel-]

I'm not sure yet how this will play out exactly, but this struct will change completely when we introduce environment capture. This will require some thinking.

[DavisVaughan]

Neat entry! macro to get a static lifetime

[DavisVaughan]

Nit:

I have some gut feeling that pub nse here should really be pub arguments or something?

Like when use use Effect::nse it seems like its because you're wrapping argument info.

Or maybe it's that a function can (eventually) contribute 3 ish kinds of effects:

• NSE
• Attach
• Assign

And it just so happens to be that the only way it can contribute an NSE effect is via these arguments? That's fine then, I guess.

[lionel-]

Yep this becomes arguments in an ulterior PR. In addition to Nse (which will become EvalQ), an argument might be Quote or Eval.

[DavisVaughan]

And what happens when you start incorporating the envir argument?

[lionel-]

It'll keep working as is when the environment is nested or current. What happens in the other cases depends on what the environment resolves to (unknown, data-masked, etc).

[DavisVaughan]

"Should be" makes me think we have a bug, is it?

[lionel-]

More like a known limitation, but I've just refactored things to lift it.