test(e2e): add deepl conformance cases for ai-proxy#4121
Conversation
336d8e2 to
2aca065
Compare
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4121 +/- ##
=======================================
Coverage ? 50.70%
=======================================
Files ? 88
Lines ? 13398
Branches ? 0
=======================================
Hits ? 6794
Misses ? 6156
Partials ? 448 🚀 New features to boost your workflow:
|
The test doesn't pass. |
…test
The deepl conformance case's expected response body was invalid JSON:
`"finish_reason":null` sat outside the choice object but inside the
`choices` array (i.e. `...}},"finish_reason":null]` instead of
`...},"finish_reason":null}]`). The e2e HTTP comparator unmarshals the
expected body first, so the case always failed with "failed to unmarshall
ExpectedResponse body ... invalid character ':' after array element",
which is what surfaced as the higress-wasmplugin-test (GO) failure.
Move `"finish_reason":null` back inside the single choice object so the
document is well-formed. The corrected body now parses and is structurally
equal to the deepl provider's actual `chatCompletionResponse` output
(`index`, `message{name,role,content}`, `finish_reason:null`,
`logprobs:null`, `model`, `object`), with `created`/`usage` remaining in
`JsonBodyIgnoreFields` as before.
Verified by reproducing the provider's `responseDeepl2OpenAI` success
branch and comparing the marshaled output against the fixed expected body
(structural match), and with `gofmt -e` on the edited file.
Signed-off-by: 王越 <1939455790@qq.com>
|
Thanks for flagging the Root causeThe deepl conformance case's expected response body was invalid JSON. The The e2e HTTP comparator ( FixMoved Verification
The provider/YAML wiring (Ingress |
59c8fb9 to
40ce0f0
Compare
Ⅰ. Describe what this PR did
Add a non-streaming e2e conformance case for the
deeplprovider of the ai-proxy wasm plugin. The case sends an OpenAI-format chat completion request (modelFree) and verifies that ai-proxy transforms it to the DeepL/v2/translaterequest shape (text array +target_lang), routes to the Free hostapi-free.deepl.com, and converts the DeepL translation response back into an OpenAIchat.completion(withdetected_source_languagemapped tomessage.name).It reuses the same conformance scaffolding as the existing openai/hunyuan/etc. cases:
Ingresswasmplugin-ai-proxy-deepl(hostapi-free.deepl.com) pointing atllm-mock-service.matchRulewithtype: deepl,apiTokens: [fake_token],targetLang: EN(required bydeeplProviderInitializer.ValidateConfig).createdandusagefields.Ⅱ. Does this pull request fix one issue?
fixes #1723
Ⅲ. Why don't you add test cases (unit test/integration test)?
This PR itself adds the e2e conformance test case for the deepl provider.
Ⅳ. Describe how to verify it
Run the conformance suite against a cluster with the
llm-mock(latest image, which ships the deepl mock) service and theai-proxywasm plugin built:The
deepl case 1: non-streaming requestcase should pass.Ⅴ. Special notes for reviews
/v2/translateis non-streaming, so the deepl provider has no streaming path.model: "Free"(the deepl provider requiresFreeorPro;Freeselects theapi-free.deepl.comhost).targetLang: ENis mandatory in the provider config and is therefore set in thematchRule.JsonBodyIgnoreFields: ["created", "usage"]excludes the timestamp (time.Now()) and the always-nullusage(deepl does not populate token usage) from the comparison.Ⅵ. AI Coding Tool Usage Checklist (if applicable)
Please check all applicable items:
For new standalone features (e.g., new wasm plugin or golang-filter plugin):
design/directory in the plugin folderdesign/directoryFor regular updates/changes (not new plugins):
AI Coding Prompts (for regular updates)
AI Coding Summary