Skip to content

Replaced server-wide queue with asyncio#124

Merged
MatejFrnka merged 4 commits into
developfrom
refactor/102-use-asyncio-in-routes
Jul 3, 2026
Merged

Replaced server-wide queue with asyncio#124
MatejFrnka merged 4 commits into
developfrom
refactor/102-use-asyncio-in-routes

Conversation

@MatejFrnka

@MatejFrnka MatejFrnka commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

CPU usage remained similar, RAM usage has improved slightly.
Green = original
Orange = #123
Blue = This PR

image

I only did one run for measuring: queue manager set to 10 parallel executions, 20 requests sent in total for each config
can do more if needed

closes #102

@MatejFrnka MatejFrnka changed the title Replaced even server-wide queue with asyncio Replaced server-wide queue with asyncio Jun 26, 2026
@MatejFrnka MatejFrnka changed the title Replaced server-wide queue with asyncio [DRAFT] Replaced server-wide queue with asyncio Jun 26, 2026
@MatejFrnka
MatejFrnka force-pushed the refactor/102-use-asyncio-in-routes branch from 6cfc6eb to c25f422 Compare June 26, 2026 16:32
@MatejFrnka
MatejFrnka force-pushed the refactor/102-use-asyncio-in-routes branch from bdbfdc8 to 438857f Compare June 30, 2026 07:46
Comment thread service/src/ai_document_plugin_service/api/routes.py
@MatejFrnka MatejFrnka changed the title [DRAFT] Replaced server-wide queue with asyncio Replaced server-wide queue with asyncio Jul 3, 2026
@MatejFrnka
MatejFrnka merged commit 1b6f7c9 into develop Jul 3, 2026
8 checks passed
@MatejFrnka
MatejFrnka deleted the refactor/102-use-asyncio-in-routes branch July 3, 2026 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use Asyncio instead of threadpool for parallel requests to openai client

1 participant