fix: prevent server crash when clients send 'tools' param (fixes #530)#538
Open
octo-patch wants to merge 1 commit intomicrosoft:mainfrom
Open
fix: prevent server crash when clients send 'tools' param (fixes #530)#538octo-patch wants to merge 1 commit intomicrosoft:mainfrom
octo-patch wants to merge 1 commit intomicrosoft:mainfrom
Conversation
…icrosoft#530) When clients like Open WebUI send a POST to /v1/chat/completions with a 'tools' parameter (for function calling), the llama-server binary crashes with SIGABRT instead of returning a proper error response. Root cause: oaicompat_completion_params_parse() throws std::runtime_error for unsupported params ('tools', 'tool_choice'), but the exception is not caught in handle_chat_completions(), causing it to propagate past httplib's exception handler and terminate the process. Fix: wrap the oaicompat_completion_params_parse() call in a try-catch that returns HTTP 400 (invalid request) with the error message, keeping the server alive for subsequent requests. Change: update 3rdparty/llama.cpp submodule to octo-patch/llama.cpp at commit 7525084 which contains the fix in examples/server/server.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #530
Problem
When clients like Open WebUI send a
POST /v1/chat/completionsrequest with atoolsparameter (standard OpenAI function-calling API), the llama-server binary crashes withSIGABRTinstead of returning a proper error response:Root Cause
In
examples/server/server.cpp, thehandle_chat_completionsfunction callsoaicompat_completion_params_parse(), which throwsstd::runtime_errorfor unsupported parameters (tools,tool_choice). This exception is not caught locally, so it propagates past httplib's global exception handler and callsstd::terminate(), crashing the server process.Solution
Wrap the
oaicompat_completion_params_parse()call in atry-catchblock that:std::exceptionthrown during parameter parsingThe change is minimal (7 lines in
examples/server/server.cpp) and does not affect normal request handling.Changes
3rdparty/llama.cppsubmodule toocto-patch/llama.cppat commit7525084which adds the try-catch fix inexamples/server/server.cppTesting
Verified that the fix pattern matches how other error conditions are handled in the same file (e.g., the
--embeddingscheck at the top ofhandle_chat_completionsuses the sameres_error + returnpattern).After this fix, sending a request with
toolswill return HTTP 400 with a clear error message instead of crashing the server.