Add guide for using AI on your own GPU server privately

[koppor]

Follow-up to JabRef/jabref#16981

Pull Request Description

Summary

New page "AI on your own GPU server" walks researchers through running the language model on their group's server so PDF content never reaches a commercial AI provider. The existing local Ollama guide now warns that the API base URL is ignored unless "Customize expert settings" is checked, in which case requests silently go to OpenAI.

Steps to test

1. Open the preview of this PR and go to "AI functionality → AI on your own GPU server".
2. Follow the steps with an Ollama server and a JabRef development build.

Related issues and pull requests

Closes _____
Documents _____

AI usage

Claude Code (model claude-opus-5), AIL4.

Checklist

• I reviewed and take ownership of all content in this PR, including any AI-assisted text
• I opened JabRef and followed these instructions myself to confirm they still work (tested on version: _____)
• Any links I added or changed work (no 404s)
• Any images or tables I added display correctly
• I checked spelling/grammar

🤖 Generated with Claude Code

https://claude.ai/code/session_018T7VcBpJsgKMRhwWKNghia

[ThiloteE]

Why?! Just check the "Running a local language model" doc page?!
Why are we splitting this up into two competing documentation sub-pages?

[ThiloteE]

We already have https://docs.jabref.org/ai/local-llm#step-by-step-guide-for-ollama

[koppor]

The first one is the one for experts, the second one is a step-by-step instruction form a single source, centered to jabref and a remote GPU cluster. See the first steps in the documentaiton.

[ThiloteE]

Ok, you are right. I am sorry for the exclamation marks. Though, running it on a remote GPU cluster seems to be the one for experts, not the local setup, since the documentation to set up a remote instance is much longer. Yes, given that you want to introduce a difference between remote and local, it indeed might make sense to keep them on separate pages.

[ThiloteE]

I didn't think long enough about this before commenting.

[koppor]

I am not sure though... Maybe, also having it integrated with the existing page would make sense. Not sure, but I would like to have some guidance on remote usage without "thinking" on my side ^^

[ThiloteE]

Outdated model.

[koppor]

What do you propose then?

[ThiloteE]

The one you were hosting during JabCon wasn't bad.
https://ollama.com/library/granite4.2:8b
The average gamer currently has a GPU with 12GB of VRAM, according to the steam hardware survey and this model fits in there, so this is what I would recommend for now.

[koppor]

Switched to qwen3.8:27b and updated the model list in 033f395. Quick check on full papers from the chocolate demo library: qwen3.8:27b answered all questions correctly; gpt-oss:20b too, but faster, so it stays as the alternative.

[koppor]

@ThiloteE I know, this is now duplicate information...

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Document private AI on local hardware and GPU servers

📝 Documentation ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Replaces local-model documentation with a privacy-focused computer and GPU server guide.
• Documents secure Ollama access, model sizing, JabRef settings, safeguards, and troubleshooting.
• Redirects old links and refreshes AI documentation navigation and references.

Diagram

graph TD
  A["JabRef PDF"] --> B["Local embeddings"] --> C["Relevant PDF text"] --> D{"Model location"}
  D -->|"Own computer"| F["Ollama API"] --> G["Local model"]
  D -->|"GPU server"| E["SSH tunnel"] --> F
  C -.->|"blocked"| H["Commercial APIs"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Authenticated HTTPS endpoint

• ➕ Supports persistent access for multiple researchers without maintaining individual SSH sessions.
• ➕ Can centralize authentication, certificates, logging, and network policy.
• ➖ Requires reverse-proxy, certificate, and identity-management administration.
• ➖ Exposes a network service and increases the consequences of configuration mistakes.

2. VPN-only server access

• ➕ Provides encrypted access without exposing Ollama publicly.
• ➕ Can offer a smoother persistent connection for regular server users.
• ➖ Requires institutional VPN infrastructure or additional client software.
• ➖ May route more traffic than necessary and needs administrator-managed access controls.

3. JabRef provider allowlist

• ➕ Would prevent commercial requests inside JabRef without modifying the system hosts file.
• ➕ Avoids blocking commercial providers for unrelated applications.
• ➖ Requires application code changes and distribution in a future JabRef release.
• ➖ Does not help users running existing JabRef versions.

Recommendation: The documented SSH tunnel is the best default because it uses commonly available tooling, encrypts traffic, and avoids exposing Ollama's unauthenticated port. Institutions with managed infrastructure may prefer an authenticated HTTPS endpoint or VPN, while a future JabRef-level provider allowlist would provide a more targeted safety mechanism than hosts-file blocking.

Files changed (6) +214 / -5

Documentation (5) +213 / -5

SUMMARY.mdExpose the private AI guide in navigation +1/-1

Expose the private AI guide in navigation

• Replaces the former local language model navigation entry with the expanded guide for personal computers and GPU servers.

en/SUMMARY.md

README.mdUpdate the AI section guide reference +2/-2

Update the AI section guide reference

• Points the AI documentation landing page to 'private-llm-server.md' instead of the removed local-model page.

en/ai/README.md

ai-providers-and-api-keys.mdClarify privately hosted OpenAI-compatible providers +1/-1

Clarify privately hosted OpenAI-compatible providers

• Updates the local-model discussion to cover personal computers and GPU servers, explain OpenAI-compatible configuration, and refine hardware and model-capability guidance.

en/ai/ai-providers-and-api-keys.md

how-to-enable-and-use-ai-features.mdLink setup guidance to the private AI guide +1/-1

Link setup guidance to the private AI guide

• Redirects the external-provider explanation to the new private computer and GPU server documentation.

en/ai/how-to-enable-and-use-ai-features.md

private-llm-server.mdAdd comprehensive private AI server guide +208/-0

Add comprehensive private AI server guide

• Documents model and hardware selection, Ollama installation, encrypted SSH tunneling, JabRef expert settings, local embedding models, and end-to-end verification. It also warns about cloud-backed models and accidental commercial API requests, recommends provider blocking as a safety net, and covers related privacy settings and troubleshooting.

en/ai/private-llm-server.md

Other (1) +1 / -0

.gitbook.ymlRedirect the former local-model documentation URL +1/-0

Redirect the former local-model documentation URL

• Adds a GitBook redirect from 'ai/local-llm' to the new private server guide, preserving existing inbound links and bookmarks.

.gitbook.yml

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗