[MOSH-2320]: Update API docs for worker number exposure#240
[MOSH-2320]: Update API docs for worker number exposure#240jli-together wants to merge 1 commit intomainfrom
Conversation
✱ Stainless preview buildsThis PR will update the go openapi python terraform typescript Edit this comment to update them. They will appear in their respective SDK's changelogs. ✅ togetherai-terraform studio · code · diff
✅ togetherai-openapi studio · code · diff
✅ togetherai-typescript studio · code · diff
✅ togetherai-python studio · code · diff
✅ togetherai-go studio · code · diff
This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push. |
| type: integer | ||
| minimum: 1 | ||
| description: "Number of concurrent workers for inference requests. Overrides the default concurrency for this model. Useful for tuning throughput when using proxy endpoints (e.g. OpenRouter) or rate-limited external APIs." | ||
| example: 9 |
There was a problem hiding this comment.
the examples are a bit weird, maybe we can just say 5
Related to PR changes: https://github.com/togethercomputer/together-evaluation/pull/165