feat: workspace symbols warmup (Phase 2) by AlexCannonball · Pull Request #158 · coder3101/protols

AlexCannonball · 2026-08-11T20:24:01Z

Improvements related to Phase 2 from #130

AlexCannonball · 2026-08-11T20:39:17Z

At the moment still a very early attempt to implement eager warmup + lazy query evaluation from #130 (comment)

TODO list from the top of my head:

Add progress to the warmup function
Get rid of the temporary debug ProtoLanguageServer::cache field
Implement the CacheEntry state change methods
spawn_blocking to get metamodel for document(s)
Event listeners concept from Proposal: High-Performance Architecture Refactor (Zero-FFI Queries, Pre-Computed Metadata) #130 (comment)
FS watchers to handle things like git checkout
handle adding and removing Workspace Folders, with nested folders optimizations

AlexCannonball · 2026-08-25T19:53:43Z

Quick update on Phase 2 refactoring:

I'm currently pushing all heavy synchronous workloads (Tree-sitter processing, WalkDir, protoc, and clang-format) completely out of the main current_thread executor.

As expected, introducing async and graceful cancellation has deeply "infected" the entire codebase.

Right now, the IDE is reporting 100+ cascading type mismatch and lifetime errors.

I am currently grinding through these compiler errors layer by layer to align all the type boundaries and will continue pushing updates to this PR as soon as the core compilation passes. Still on it!

coder3101 · 2026-09-21T02:59:50Z

Are you still working on this?

AlexCannonball · 2026-09-21T03:59:49Z

Hello @coder3101

Yes, I do. Currently I'm trying to switch LSP notification handling to async. The order of notifications must be preserved and async-lsp only supports synchronous notification handler. So my attempt is to use the unbounded mpsc channel (synchronous tx, async handlers for rx).

AlexCannonball · 2026-10-06T17:30:24Z

@coder3101 Hello, I've updated the draft to share some progress.

async-ification takes more time than I expected. Still in the middle of deep refactoring, still has compilation errors, but now I feel that some concepts look more realistic:

Notification worker.
- Add the client capabilities for the file watcher registration.
Cancellation topology.
src/state/mod.rs the method query_documents_batch looks promising: batching, cancellation, shared futures, progress reporting and the resulting futures stream (hopefully it unlocks the partial result feature).

I will also comment some interesting findings in separate line comments.

So far I'm breaking down src/lsp.rs into distinct submodules for proper LSP request handling (some are already placed in src/server/request/).

src/state/mod.rs needs a cleanup as well. Will do it later.

I understand that currently import and grammar diagnostics probably won't work. First, I'd like to compile and test the metamodel cache state and the notification edge cases.

AlexCannonball · 2026-10-06T17:54:51Z

-            .output()
-            .ok()?;
+        let wait_future = child.wait_with_output();
+        let timeout_result = tokio::time::timeout(CLANG_FORMAT_TIMEOUT, wait_future).await;


Improvements:

async, not blocking the main thread

using stdin input, so a temp file isn't needed

added timeout protection

AlexCannonball · 2026-10-06T17:55:59Z

-
-pub use clang::ClangFormatter;
-
-pub trait ProtoFormatter: Sized {


Haven't discovered this trait purpose.

Could have been used for unifying other formatters like buf, clangd etc.

Thank you! As I remember, There were some troubles with the async trait. I'll keep this open and will try to restore the trait after the cache and async-ification are stabilized.

AlexCannonball · 2026-10-06T17:59:31Z

    // Run protoc and capture output
-    match cmd.output() {
-        Ok(output) => {
+    match tokio::time::timeout(PROTOC_TIMEOUT, cmd.kill_on_drop(true).output()).await {


not blocking the main thread

timeout protection

AlexCannonball · 2026-10-06T18:00:26Z

+
 pub struct ProtoLanguageServer {
    pub client: ClientSocket,
-    pub(crate) log_handle: log::LogReloadHandle,


Moved this to the notification worker.

coder3101 · 2026-10-07T03:38:35Z

    }
 }

+const CLANG_FORMAT_TIMEOUT: Duration = Duration::from_secs(2);


Should be configurable from config file, on slower systems with large file 2s might be too low.

Maybe just increasing the timeout from 2 seconds to 5–10 seconds will be enough?

if not configurable, setting to 5 second is fine, given its async anyways.

Thank you, I've set it to 5: 51e4c92

I'm not sure about the setting(s) because having too many config options isn't always good for users. Also it increases the complexity:

Validation and boundaries

The main TOML config

Client settings

CLI args

ENV

Anyways, we can add a setting later as a follow-up, if needed.

coder3101 · 2026-10-07T03:48:35Z

-    file_path: &str,
-    include_paths: &[String],
-) -> Vec<Diagnostic> {
+const PROTOC_TIMEOUT: Duration = Duration::from_secs(2);


Same as ClangFormatter Timeout

coder3101 · 2026-10-07T03:52:21Z

-    pub shutdown_received: bool,
+    pub configs: Arc<RwLock<WorkspaceProtoConfigs>>,
+    pub shutdown_token: CancellationToken,
+    notification_tx: UnboundedSender<Notification>,


Why Unbounded?

Thank you, it's a very good question. I guess I was too optimistic, like, "if everything works fine, the unbounded channel will never consume too many memory"😀

I've changed it to a fail-fast bounded sender, please take a look: 44d7cbf

https://docs.rs/async-lsp/latest/async_lsp/router/struct.Router.html#method.notification

Also, since async-lsp routes notifications through a synchronous API handler, we can't legitimately .await on a full bounded channel without completely stalling the main events loop or introducing heavy overengineering. A fail-fast try_send combined with an explicit WouldBlock error seems to be the most pragmatic and reliable approach here.

A fail-fast strategy for the notification worker channel.

Set `clang-format` and `protoc` run timeout to 5 seconds.

Subscribe to the client file watchers and skip reporting progress if the client doesn't support `window/workDoneProgress/create`

AlexCannonball added 2 commits October 6, 2026 17:05

feat: workspace symbols warmup

f6ec20e

WIP notification worker

3a4bdcf

AlexCannonball force-pushed the feat/workspace-symbols-warmup branch from 74ed283 to 3a4bdcf Compare October 6, 2026 17:06

AlexCannonball commented Oct 6, 2026

View reviewed changes

Comment thread src/document/syntax.rs

AlexCannonball commented Oct 6, 2026

View reviewed changes

Comment thread src/main.rs

AlexCannonball commented Oct 6, 2026

View reviewed changes

coder3101 reviewed Oct 7, 2026

View reviewed changes

AlexCannonball added 4 commits October 7, 2026 17:47

fix: typos

6b3350a

refactor: channel instead of unbounded_channel

44d7cbf

A fail-fast strategy for the notification worker channel.

fix: increase the child process timeout

51e4c92

Set `clang-format` and `protoc` run timeout to 5 seconds.

Add the client capabilities to state

7179b40

Subscribe to the client file watchers and skip reporting progress if the client doesn't support `window/workDoneProgress/create`


		pub use clang::ClangFormatter;

		pub trait ProtoFormatter: Sized {

Conversation

AlexCannonball commented Aug 11, 2026

Uh oh!

AlexCannonball commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexCannonball commented Aug 25, 2026

Uh oh!

coder3101 commented Sep 21, 2026

Uh oh!

AlexCannonball commented Sep 21, 2026

Uh oh!

AlexCannonball commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

AlexCannonball Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asharkhan3101 Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexCannonball commented Aug 11, 2026 •

edited

Loading

AlexCannonball commented Oct 6, 2026 •

edited

Loading

AlexCannonball Oct 6, 2026 •

edited

Loading

asharkhan3101 Oct 7, 2026 •

edited

Loading