Studio: provider model loading controls#5645
Conversation
…com/Imagineer99/unsloth into feat/model-loading-custom-providers
There was a problem hiding this comment.
Code Review
This pull request introduces native support for local LLM providers, specifically Ollama and llama.cpp, by adding configuration presets and specialized error handling. The backend now includes logic to rewrite provider-specific errors into user-friendly instructions and provides a fallback mechanism for listing Ollama models via its native API. The frontend was updated to handle these local providers by automatically setting default base URLs, hiding API key fields where they are not required, and supporting remote model catalog fetching. Review feedback correctly identified potential runtime vulnerabilities in the new backend logic where JSON responses from local providers are parsed without verifying they are dictionaries, which could lead to attribute errors.
|
@Imagineer99 how do you know at which port they started the service? |
this is valid, we don't know the exact port of users service, in this pr we assume its most likely default port for llama.cpp and ollama. suffix /v1 if not present is a good idea and I don't disagree with the direction change. |
…ithub.com/Imagineer99/unsloth into ux/studio-provider-model-loading-controls
for more information, see https://pre-commit.ci
|
@rolandtannous updated pr to remove pre-population of base URLs, and suffix /v1 if not present. |
* feat: add custom model v1/model loading * fix: require base URL for local model catalog loading * ux/studio-provider-model-loading-controls * fix: normalize local provider base URLs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Summary