Skip to content

Studio: provider model loading controls#5645

Merged
rolandtannous merged 10 commits into
unslothai:mainfrom
Imagineer99:ux/studio-provider-model-loading-controls
May 20, 2026
Merged

Studio: provider model loading controls#5645
rolandtannous merged 10 commits into
unslothai:mainfrom
Imagineer99:ux/studio-provider-model-loading-controls

Conversation

@Imagineer99

Copy link
Copy Markdown
Collaborator

Summary

  • Prefill default Base URLs for Ollama and llama.cpp connection presets
  • Disable “Load available models” until required connection details are present
  • Keep vLLM Base URL user-provided, since there is no reliable universal default

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces native support for local LLM providers, specifically Ollama and llama.cpp, by adding configuration presets and specialized error handling. The backend now includes logic to rewrite provider-specific errors into user-friendly instructions and provides a fallback mechanism for listing Ollama models via its native API. The frontend was updated to handle these local providers by automatically setting default base URLs, hiding API key fields where they are not required, and supporting remote model catalog fetching. Review feedback correctly identified potential runtime vulnerabilities in the new backend logic where JSON responses from local providers are parsed without verifying they are dictionaries, which could lead to attribute errors.

Comment thread studio/backend/core/inference/external_provider.py Outdated
Comment thread studio/backend/core/inference/external_provider.py Outdated
@rolandtannous

Copy link
Copy Markdown
Contributor

@Imagineer99 how do you know at which port they started the service?
In my opinion, a better direction is that we suffix /v1 if not present , when they send requests to these local endpoints.

@Imagineer99

Imagineer99 commented May 20, 2026

Copy link
Copy Markdown
Collaborator Author

@Imagineer99 how do you know at which port they started the service? In my opinion, a better direction is that we suffix /v1 if not present , when they send requests to these local endpoints.

this is valid, we don't know the exact port of users service, in this pr we assume its most likely default port for llama.cpp and ollama.

suffix /v1 if not present is a good idea and I don't disagree with the direction change.

@Imagineer99 Imagineer99 marked this pull request as ready for review May 20, 2026 12:41
@Imagineer99

Copy link
Copy Markdown
Collaborator Author

@rolandtannous updated pr to remove pre-population of base URLs, and suffix /v1 if not present.
Tested locally.

rsd-darshan pushed a commit to rsd-darshan/unsloth that referenced this pull request Jun 3, 2026
* feat: add custom model v1/model loading

* fix: require base URL for local model catalog loading

* ux/studio-provider-model-loading-controls

* fix: normalize local provider base URLs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants