Skip to content

feat: add embedding.dimensions for LiteLLM models with configurable output size - #339

Open
fzowl wants to merge 2 commits into
cocoindex-io:mainfrom
fzowl:docs/voyage-models
Open

fzowl wants to merge 2 commits into
cocoindex-io:mainfrom
fzowl:docs/voyage-models

Conversation

@fzowl

@fzowl fzowl commented Nov 14, 2026 •

Copy link
Copy Markdown

Matryoshka embedding models (for example the Voyage voyage-4 family, voyage-code-4, voyage-code-3, voyage-3-large and voyage-3.5) can return smaller vectors, but there was no way to ask for one: the README says dimensions is a model-wide setting, yet no model-wide setting existed, and it is rejected in indexing_params / query_params. The docs also showed only voyage-4-large and voyage-code-3 as Voyage examples.

This adds embedding.dimensions (LiteLLM provider only). It is passed to every embedding call, so indexing, the size probe and queries always agree. Invalid values, and use with sentence-transformers, are rejected when the settings load. Because LiteLLM silently drops parameters a model does not support, the embedder now fails if the returned vectors do not have the configured size instead of indexing at the model's native size. Changing the value changes the embedder's memo key, so the index is rebuilt on the next indexing run. EMBEDDINGS.md now lists the ten Voyage text embedding models that work through LiteLLM.

Tested with the live Voyage API through ccc init, ccc index, ccc search and ccc doctor: all ten models at their default size, eight Matryoshka models at 2048, 512 and 256 dimensions, an 80-file index (more than one embedding batch), and the failure cases (voyage-finance-2 with 512 and voyage-4-lite with 300 both surface Voyage's accepted-values message). uv run prek run --all-files passes, and so does the Docker e2e suite on macOS (the two Linux-only tests are skipped).

@fzowl fzowl changed the title docs: list the Voyage embedding models that work through LiteLLM feat: add embedding.dimensions for LiteLLM models with configurable output size Nov 14, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant